Add `tool_choice` parameter to `ChatCompletionClient` `create` and `create_stream` methods #6697

Copilot · 2025-06-19T03:50:03Z

Summary

Implements the tool_choice parameter for ChatCompletionClient interface as requested in #6696. This allows users to restrict which tools the model can choose from when multiple tools are available.

Changes

Core Interface

Core Interface: Added tool_choice: Tool | Literal["auto", "required", "none"] = "auto" parameter to ChatCompletionClient.create() and create_stream() methods
Model Implementations: Updated client implementations to support the new parameter, for now, only the following model clients are supported:
- OpenAI
- Anthropic
- Azure AI
- Ollama
LlamaCppChatCompletionClient currently not supported

Features

"auto" (default): Let the model choose whether to use tools, when there is no tool, it has no effect.
"required": Force the model to use at least one tool
"none": Disable tool usage completely
Tool object: Force the model to use a specific tool

ekzhu · 2025-06-20T12:15:05Z

@copilot can you continue working

…mplementations Co-authored-by: ekzhu <320302+ekzhu@users.noreply.github.com>

ekzhu

Add unit tests for AnthropicChatCompletionClient and OpenAIChatCompletionClient to test the use of tool_choice parameter using mocks.

python/packages/autogen-core/src/autogen_core/models/_model_client.py

…auto" Co-authored-by: ekzhu <320302+ekzhu@users.noreply.github.com>

Co-authored-by: ekzhu <320302+ekzhu@users.noreply.github.com>

Copilot · 2025-06-22T07:47:27Z

Add unit tests for AnthropicChatCompletionClient and OpenAIChatCompletionClient to test the use of tool_choice parameter using mocks.

Added comprehensive unit tests for both AnthropicChatCompletionClient and OpenAIChatCompletionClient to test the tool_choice parameter using mocks. Tests cover specific tool selection, "auto" mode, None mode, and validation errors. Commit 717c05b

@copilot can you continue working

Completed the implementation as requested. Updated the tool_choice parameter type, refactored all client implementations, and added comprehensive unit tests for OpenAI and Anthropic clients. All changes are now complete and committed.

python/packages/autogen-core/src/autogen_core/models/_model_client.py

Co-authored-by: ekzhu <320302+ekzhu@users.noreply.github.com>

ekzhu · 2025-06-30T04:43:20Z

@copilot update the PR description, do not modify the code.

Copilot · 2025-06-30T04:45:08Z

@copilot update the PR description, do not modify the code.

Updated the PR description to reflect the current implementation with the simplified tool_choice parameter type and behavior.

ekzhu · 2025-06-30T04:46:36Z

@copilot write me a new PR description, remove the existing one.

Copilot · 2025-06-30T04:49:10Z

@copilot write me a new PR description, remove the existing one.

Created a new PR description that accurately reflects the current implementation with the simplified tool_choice parameter design and comprehensive testing coverage.

codecov · 2025-06-30T05:12:42Z

Codecov Report

Attention: Patch coverage is 64.86486% with 39 lines in your changes missing coverage. Please review.

Project coverage is 80.25%. Comparing base (6f15270) to head (6bd777b).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
.../autogen_ext/models/anthropic/_anthropic_client.py	45.65%	25 Missing ⚠️
...t/models/llama_cpp/_llama_cpp_completion_client.py	22.22%	7 Missing ⚠️
...xt/src/autogen_ext/models/openai/_openai_client.py	87.50%	3 Missing ⚠️
...xt/models/replay/_replay_chat_completion_client.py	60.00%	2 Missing ⚠️
...els/semantic_kernel/_sk_chat_completion_adapter.py	66.66%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #6697      +/-   ##
==========================================
+ Coverage   79.85%   80.25%   +0.40%     
==========================================
  Files         232      232              
  Lines       17540    17643     +103     
==========================================
+ Hits        14006    14160     +154     
+ Misses       3534     3483      -51

Flag	Coverage Δ
unittests	`80.25% <64.86%> (+0.40%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

reqyou · 2025-07-18T14:41:58Z

Whats your plan with wiring it up to the AssistantAgent and down to the _call_llm?

ekzhu · 2025-07-18T17:14:33Z

Whats your plan with wiring it up to the AssistantAgent and down to the _call_llm?

It's already being used by AssistantAgent for the reflection step.

reqyou · 2025-07-23T07:23:18Z

That is all well and good, but what about the rest? When _tools is not empty, as a user, I would expect that either on model_client.create_stream(..) or model_client.create(..) in for example _call_llm, the tool_choice parameter is set to required by default, or that tool_choice is exposed as an assistant agent parameter to have better control and e.g. leave it to None to still allow the agent to decide whether to call the tool or just create a result.

So imho this MR only addresses half of the work.

ekzhu · 2025-07-23T16:54:14Z

That is all well and good, but what about the rest? When _tools is not empty, as a user, I would expect that either on model_client.create_stream(..) or model_client.create(..) in for example _call_llm, the tool_choice parameter is set to required by default, or that tool_choice is exposed as an assistant agent parameter to have better control and e.g. leave it to None to still allow the agent to decide whether to call the tool or just create a result.

So imho this MR only addresses half of the work.

That's rather strong assumption on the agent behaviour. I think it will also conflict with the max_tool_iteration parameter. If you set that parameter, then the tool calling loop only ends when the agent stops calling tool or reaching maximum iteration. If the tool choice is also set to required, then you end up with forced tool calling when it is not needed.

I think in many cases AssistantAgent is not suitable. It's already a kitchen sink. See my note on the documentation: https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/tutorial/agents.html#assistant-agent

Initial plan for issue

c980baa

Copilot AI assigned Copilot and ekzhu Jun 19, 2025

Copilot started work on behalf of ekzhu June 19, 2025 03:50 View session

Copilot AI requested a review from ekzhu June 19, 2025 03:50

Copilot started work on behalf of ekzhu June 20, 2025 12:15 View session

Add tool_choice parameter to ChatCompletionClient interface and all i…

8a2fb5a

…mplementations Co-authored-by: ekzhu <320302+ekzhu@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Add tool_choice to ChatCompletionClient create and create_stream method~~ Add tool_choice parameter to ChatCompletionClient create and create_stream methods Jun 20, 2025

Copilot finished work on behalf of ekzhu June 20, 2025 12:47

ekzhu requested changes Jun 22, 2025

View reviewed changes

python/packages/autogen-core/src/autogen_core/models/_model_client.py Outdated Show resolved Hide resolved

Copilot started work on behalf of ekzhu June 22, 2025 07:32 View session

Copilot AI and others added 2 commits June 22, 2025 07:44

Update tool_choice parameter to use Tool | Literal["auto"] | None = "…

d11e27d

…auto" Co-authored-by: ekzhu <320302+ekzhu@users.noreply.github.com>

Add unit tests for tool_choice parameter in OpenAI and Anthropic clients

717c05b

Co-authored-by: ekzhu <320302+ekzhu@users.noreply.github.com>

Copilot AI requested a review from ekzhu June 22, 2025 07:48

Copilot finished work on behalf of ekzhu June 22, 2025 07:48

ekzhu requested changes Jun 23, 2025

View reviewed changes

python/packages/autogen-core/src/autogen_core/models/_model_client.py Outdated Show resolved Hide resolved

Copilot started work on behalf of ekzhu June 23, 2025 13:05 View session

Update tool_choice documentation to clarify None behavior

760b82c

Co-authored-by: ekzhu <320302+ekzhu@users.noreply.github.com>

Copilot AI requested a review from ekzhu June 23, 2025 13:11

Copilot finished work on behalf of ekzhu June 23, 2025 13:11

Merge branch 'main' into copilot/fix-6696

ebf597f

ekzhu marked this pull request as ready for review June 23, 2025 13:13

ekzhu added 5 commits June 23, 2025 21:35

update

1e9a475

fix typing

5b0fb66

Merge remote-tracking branch 'origin/main' into copilot/fix-6696

3956f66

Merge remote-tracking branch 'origin/main' into copilot/fix-6696

841c087

type fix

9fce9c5

ekzhu added 11 commits June 29, 2025 23:34

use the parameter in azure ai model

a20f480

Merge branch 'main' into copilot/fix-6697

0b80b9e

Update

0702fe7

update azure ai

17befd3

fix test

f026d39

update

27e1b5b

wip

912d10c

Fix tests

68ac1a2

wip

9088b38

add ollama

dbe82fd

type

c89cecb

Copilot started work on behalf of ekzhu June 30, 2025 04:43 View session

Copilot finished work on behalf of ekzhu June 30, 2025 04:45

Copilot started work on behalf of ekzhu June 30, 2025 04:46 View session

Copilot finished work on behalf of ekzhu June 30, 2025 04:49

Fix tests

6bd777b

ekzhu approved these changes Jun 30, 2025

View reviewed changes

ekzhu merged commit c150f85 into main Jun 30, 2025
66 of 67 checks passed

ekzhu deleted the copilot/fix-6696 branch June 30, 2025 05:15

This was referenced Jun 30, 2025

Make reflection step work with various openrouter models #6695

Closed

Add tool_choice to ChatCompletionClient create and create_stream method #6696

Closed

Add tool_choice parameter to ChatCompletionClient create and create_stream methods #6697

Add tool_choice parameter to ChatCompletionClient create and create_stream methods #6697

Uh oh!

Conversation

Copilot AI commented Jun 19, 2025 • edited by ekzhu Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Core Interface

Uh oh!

ekzhu commented Jun 20, 2025

Uh oh!

ekzhu left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI commented Jun 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

ekzhu commented Jun 30, 2025

Uh oh!

Copilot AI commented Jun 30, 2025

Uh oh!

ekzhu commented Jun 30, 2025

Uh oh!

Copilot AI commented Jun 30, 2025

Uh oh!

codecov bot commented Jun 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

reqyou commented Jul 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ekzhu commented Jul 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

reqyou commented Jul 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ekzhu commented Jul 23, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Add `tool_choice` parameter to `ChatCompletionClient` `create` and `create_stream` methods #6697

Add `tool_choice` parameter to `ChatCompletionClient` `create` and `create_stream` methods #6697

Copilot AI commented Jun 19, 2025 •

edited by ekzhu

Loading

Copilot AI commented Jun 22, 2025 •

edited

Loading

codecov bot commented Jun 30, 2025 •

edited

Loading

reqyou commented Jul 18, 2025 •

edited

Loading

ekzhu commented Jul 18, 2025 •

edited

Loading

reqyou commented Jul 23, 2025 •

edited

Loading