KEMBAR78
Changes for evaluating remote workflows by AnuradhaKaruppiah · Pull Request #57 · NVIDIA/NeMo-Agent-Toolkit · GitHub
Skip to content

Conversation

@AnuradhaKaruppiah
Copy link
Contributor

@AnuradhaKaruppiah AnuradhaKaruppiah commented Apr 1, 2025

Change done:

  1. Added new endpoint generate/stream/full to stream the complete IntermediateStep. Sample usage -
curl --request POST   --url http://localhost:8000/generate/stream/full   --header 'Content-Type: application/json'   --data '{
    "input_message": "What is LangSmith?"
}'
  1. Use the generate/stream/full endpoint for evaluating remote workflows.

Sample Usage:

  1. Start server on the remote cluster with the base config.yml:
aiq serve --config_file=examples/simple/configs/config.yml 
  1. Run evaluation, against the remote endpoint, using a different config.yml that provides the dataset:
aiq eval --config_file=examples/simple/configs/eval_config.yml --endpoint http://localhost:8000

Closes #51

@AnuradhaKaruppiah AnuradhaKaruppiah requested a review from a team as a code owner April 1, 2025 20:15
@AnuradhaKaruppiah AnuradhaKaruppiah marked this pull request as draft April 1, 2025 20:16
@AnuradhaKaruppiah AnuradhaKaruppiah added feature request New feature or request non-breaking Non-breaking change labels Apr 1, 2025
Signed-off-by: Anuradha Karuppiah <anuradhak@nvidia.com>
Signed-off-by: Anuradha Karuppiah <anuradhak@nvidia.com>
1. Added new helper function generate_streaming_response_raw that provides raw
   IntermediateStep objects.
2. Added the new endpoint /generate/stream/full in the FastApiFrontEndPluginWorker
   class

The new endpoint will:
1. Accept the same input as the regular /generate/stream endpoint
2. Return raw IntermediateStep objects without any step adaptor translations
3. Include all intermediate steps, not just the filtered ones that the step
   adaptor would normally provide

Signed-off-by: Anuradha Karuppiah <anuradhak@nvidia.com>
Signed-off-by: Anuradha Karuppiah <anuradhak@nvidia.com>
Signed-off-by: Anuradha Karuppiah <anuradhak@nvidia.com>
Signed-off-by: Anuradha Karuppiah <anuradhak@nvidia.com>
Signed-off-by: Anuradha Karuppiah <anuradhak@nvidia.com>
Signed-off-by: Anuradha Karuppiah <anuradhak@nvidia.com>
Signed-off-by: Anuradha Karuppiah <anuradhak@nvidia.com>
Signed-off-by: Anuradha Karuppiah <anuradhak@nvidia.com>
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.

Signed-off-by: Anuradha Karuppiah <anuradhak@nvidia.com>
Signed-off-by: Anuradha Karuppiah <anuradhak@nvidia.com>
@AnuradhaKaruppiah AnuradhaKaruppiah marked this pull request as ready for review April 7, 2025 21:17
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

src/aiq/front_ends/fastapi/response_helpers.py:131

  • Ensure that 'AsyncIOProducerConsumerQueue' is imported to avoid a NameError. Please add the appropriate import statement if it is not defined within this module.
q: AsyncIOProducerConsumerQueue[AIQResponseSerializable] = AsyncIOProducerConsumerQueue()

Signed-off-by: Anuradha Karuppiah <anuradhak@nvidia.com>
@AnuradhaKaruppiah
Copy link
Contributor Author

/merge

@rapids-bot rapids-bot bot merged commit 8989f94 into NVIDIA:develop Apr 8, 2025
10 checks passed
liamy-nv pushed a commit to liamy-nv/AgentIQ that referenced this pull request Apr 9, 2025
Change done:
1. Added new endpoint `generate/stream/full` to stream the complete IntermediateStep. Sample usage -
```
curl --request POST   --url http://localhost:8000/generate/stream/full   --header 'Content-Type: application/json'   --data '{
    "input_message": "What is LangSmith?"
}'
```
2.  Use the `generate/stream/full` endpoint for evaluating remote workflows.


Sample Usage:
1.  Start server on the remote cluster with the base config.yml:
```
aiq serve --config_file=examples/simple/configs/config.yml 
```
2. Run evaluation, against the remote endpoint, using a different config.yml that provides the dataset:
```
aiq eval --config_file=examples/simple/configs/eval_config.yml --endpoint http://localhost:8000
```

Closes NVIDIA#51

Authors:
  - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah)

Approvers:
  - Eric Evans II (https://github.com/ericevans-nv)

URL: NVIDIA#57
ericevans-nv pushed a commit to ericevans-nv/agent-iq that referenced this pull request Apr 14, 2025
Change done:
1. Added new endpoint `generate/stream/full` to stream the complete IntermediateStep. Sample usage -
```
curl --request POST   --url http://localhost:8000/generate/stream/full   --header 'Content-Type: application/json'   --data '{
    "input_message": "What is LangSmith?"
}'
```
2.  Use the `generate/stream/full` endpoint for evaluating remote workflows.


Sample Usage:
1.  Start server on the remote cluster with the base config.yml:
```
aiq serve --config_file=examples/simple/configs/config.yml 
```
2. Run evaluation, against the remote endpoint, using a different config.yml that provides the dataset:
```
aiq eval --config_file=examples/simple/configs/eval_config.yml --endpoint http://localhost:8000
```

Closes NVIDIA#51

Authors:
  - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah)

Approvers:
  - Eric Evans II (https://github.com/ericevans-nv)

URL: NVIDIA#57
yczhang-nv pushed a commit to yczhang-nv/NeMo-Agent-Toolkit that referenced this pull request Apr 21, 2025
Change done:
1. Added new endpoint `generate/stream/full` to stream the complete IntermediateStep. Sample usage -
```
curl --request POST   --url http://localhost:8000/generate/stream/full   --header 'Content-Type: application/json'   --data '{
    "input_message": "What is LangSmith?"
}'
```
2.  Use the `generate/stream/full` endpoint for evaluating remote workflows.

Sample Usage:
1.  Start server on the remote cluster with the base config.yml:
```
aiq serve --config_file=examples/simple/configs/config.yml
```
2. Run evaluation, against the remote endpoint, using a different config.yml that provides the dataset:
```
aiq eval --config_file=examples/simple/configs/eval_config.yml --endpoint http://localhost:8000
```

Closes NVIDIA#51

Authors:
  - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah)

Approvers:
  - Eric Evans II (https://github.com/ericevans-nv)

URL: NVIDIA#57
Signed-off-by: Yuchen Zhang <134643420+yczhang-nv@users.noreply.github.com>
@AnuradhaKaruppiah AnuradhaKaruppiah deleted the remote-workflow branch May 6, 2025 00:48
yczhang-nv pushed a commit to yczhang-nv/NeMo-Agent-Toolkit that referenced this pull request May 8, 2025
Change done:
1. Added new endpoint `generate/stream/full` to stream the complete IntermediateStep. Sample usage -
```
curl --request POST   --url http://localhost:8000/generate/stream/full   --header 'Content-Type: application/json'   --data '{
    "input_message": "What is LangSmith?"
}'
```
2.  Use the `generate/stream/full` endpoint for evaluating remote workflows.

Sample Usage:
1.  Start server on the remote cluster with the base config.yml:
```
aiq serve --config_file=examples/simple/configs/config.yml
```
2. Run evaluation, against the remote endpoint, using a different config.yml that provides the dataset:
```
aiq eval --config_file=examples/simple/configs/eval_config.yml --endpoint http://localhost:8000
```

Closes NVIDIA#51

Authors:
  - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah)

Approvers:
  - Eric Evans II (https://github.com/ericevans-nv)

URL: NVIDIA#57
Signed-off-by: Yuchen Zhang <134643420+yczhang-nv@users.noreply.github.com>
AnuradhaKaruppiah added a commit to AnuradhaKaruppiah/oss-agentiq that referenced this pull request Aug 4, 2025
Change done:
1. Added new endpoint `generate/stream/full` to stream the complete IntermediateStep. Sample usage -
```
curl --request POST   --url http://localhost:8000/generate/stream/full   --header 'Content-Type: application/json'   --data '{
    "input_message": "What is LangSmith?"
}'
```
2.  Use the `generate/stream/full` endpoint for evaluating remote workflows.


Sample Usage:
1.  Start server on the remote cluster with the base config.yml:
```
aiq serve --config_file=examples/simple/configs/config.yml 
```
2. Run evaluation, against the remote endpoint, using a different config.yml that provides the dataset:
```
aiq eval --config_file=examples/simple/configs/eval_config.yml --endpoint http://localhost:8000
```

Closes NVIDIA#51

Authors:
  - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah)

Approvers:
  - Eric Evans II (https://github.com/ericevans-nv)

URL: NVIDIA#57
scheckerNV pushed a commit to scheckerNV/aiq-factory-reset that referenced this pull request Aug 22, 2025
Change done:
1. Added new endpoint `generate/stream/full` to stream the complete IntermediateStep. Sample usage -
```
curl --request POST   --url http://localhost:8000/generate/stream/full   --header 'Content-Type: application/json'   --data '{
    "input_message": "What is LangSmith?"
}'
```
2.  Use the `generate/stream/full` endpoint for evaluating remote workflows.


Sample Usage:
1.  Start server on the remote cluster with the base config.yml:
```
aiq serve --config_file=examples/simple/configs/config.yml 
```
2. Run evaluation, against the remote endpoint, using a different config.yml that provides the dataset:
```
aiq eval --config_file=examples/simple/configs/eval_config.yml --endpoint http://localhost:8000
```

Closes NVIDIA#51

Authors:
  - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah)

Approvers:
  - Eric Evans II (https://github.com/ericevans-nv)

URL: NVIDIA#57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature request New feature or request non-breaking Non-breaking change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEA]: Enable evaluation of remote workflows

3 participants