-
Notifications
You must be signed in to change notification settings - Fork 396
Changes for evaluating remote workflows #57
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Signed-off-by: Anuradha Karuppiah <anuradhak@nvidia.com>
Signed-off-by: Anuradha Karuppiah <anuradhak@nvidia.com>
Signed-off-by: Anuradha Karuppiah <anuradhak@nvidia.com>
1. Added new helper function generate_streaming_response_raw that provides raw IntermediateStep objects. 2. Added the new endpoint /generate/stream/full in the FastApiFrontEndPluginWorker class The new endpoint will: 1. Accept the same input as the regular /generate/stream endpoint 2. Return raw IntermediateStep objects without any step adaptor translations 3. Include all intermediate steps, not just the filtered ones that the step adaptor would normally provide Signed-off-by: Anuradha Karuppiah <anuradhak@nvidia.com>
Signed-off-by: Anuradha Karuppiah <anuradhak@nvidia.com>
Signed-off-by: Anuradha Karuppiah <anuradhak@nvidia.com>
Signed-off-by: Anuradha Karuppiah <anuradhak@nvidia.com>
Signed-off-by: Anuradha Karuppiah <anuradhak@nvidia.com>
Signed-off-by: Anuradha Karuppiah <anuradhak@nvidia.com>
Signed-off-by: Anuradha Karuppiah <anuradhak@nvidia.com>
Signed-off-by: Anuradha Karuppiah <anuradhak@nvidia.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.
Signed-off-by: Anuradha Karuppiah <anuradhak@nvidia.com>
Signed-off-by: Anuradha Karuppiah <anuradhak@nvidia.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (1)
src/aiq/front_ends/fastapi/response_helpers.py:131
- Ensure that 'AsyncIOProducerConsumerQueue' is imported to avoid a NameError. Please add the appropriate import statement if it is not defined within this module.
q: AsyncIOProducerConsumerQueue[AIQResponseSerializable] = AsyncIOProducerConsumerQueue()
Signed-off-by: Anuradha Karuppiah <anuradhak@nvidia.com>
|
/merge |
Change done:
1. Added new endpoint `generate/stream/full` to stream the complete IntermediateStep. Sample usage -
```
curl --request POST --url http://localhost:8000/generate/stream/full --header 'Content-Type: application/json' --data '{
"input_message": "What is LangSmith?"
}'
```
2. Use the `generate/stream/full` endpoint for evaluating remote workflows.
Sample Usage:
1. Start server on the remote cluster with the base config.yml:
```
aiq serve --config_file=examples/simple/configs/config.yml
```
2. Run evaluation, against the remote endpoint, using a different config.yml that provides the dataset:
```
aiq eval --config_file=examples/simple/configs/eval_config.yml --endpoint http://localhost:8000
```
Closes NVIDIA#51
Authors:
- Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah)
Approvers:
- Eric Evans II (https://github.com/ericevans-nv)
URL: NVIDIA#57
Change done:
1. Added new endpoint `generate/stream/full` to stream the complete IntermediateStep. Sample usage -
```
curl --request POST --url http://localhost:8000/generate/stream/full --header 'Content-Type: application/json' --data '{
"input_message": "What is LangSmith?"
}'
```
2. Use the `generate/stream/full` endpoint for evaluating remote workflows.
Sample Usage:
1. Start server on the remote cluster with the base config.yml:
```
aiq serve --config_file=examples/simple/configs/config.yml
```
2. Run evaluation, against the remote endpoint, using a different config.yml that provides the dataset:
```
aiq eval --config_file=examples/simple/configs/eval_config.yml --endpoint http://localhost:8000
```
Closes NVIDIA#51
Authors:
- Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah)
Approvers:
- Eric Evans II (https://github.com/ericevans-nv)
URL: NVIDIA#57
Change done:
1. Added new endpoint `generate/stream/full` to stream the complete IntermediateStep. Sample usage -
```
curl --request POST --url http://localhost:8000/generate/stream/full --header 'Content-Type: application/json' --data '{
"input_message": "What is LangSmith?"
}'
```
2. Use the `generate/stream/full` endpoint for evaluating remote workflows.
Sample Usage:
1. Start server on the remote cluster with the base config.yml:
```
aiq serve --config_file=examples/simple/configs/config.yml
```
2. Run evaluation, against the remote endpoint, using a different config.yml that provides the dataset:
```
aiq eval --config_file=examples/simple/configs/eval_config.yml --endpoint http://localhost:8000
```
Closes NVIDIA#51
Authors:
- Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah)
Approvers:
- Eric Evans II (https://github.com/ericevans-nv)
URL: NVIDIA#57
Signed-off-by: Yuchen Zhang <134643420+yczhang-nv@users.noreply.github.com>
Change done:
1. Added new endpoint `generate/stream/full` to stream the complete IntermediateStep. Sample usage -
```
curl --request POST --url http://localhost:8000/generate/stream/full --header 'Content-Type: application/json' --data '{
"input_message": "What is LangSmith?"
}'
```
2. Use the `generate/stream/full` endpoint for evaluating remote workflows.
Sample Usage:
1. Start server on the remote cluster with the base config.yml:
```
aiq serve --config_file=examples/simple/configs/config.yml
```
2. Run evaluation, against the remote endpoint, using a different config.yml that provides the dataset:
```
aiq eval --config_file=examples/simple/configs/eval_config.yml --endpoint http://localhost:8000
```
Closes NVIDIA#51
Authors:
- Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah)
Approvers:
- Eric Evans II (https://github.com/ericevans-nv)
URL: NVIDIA#57
Signed-off-by: Yuchen Zhang <134643420+yczhang-nv@users.noreply.github.com>
Change done:
1. Added new endpoint `generate/stream/full` to stream the complete IntermediateStep. Sample usage -
```
curl --request POST --url http://localhost:8000/generate/stream/full --header 'Content-Type: application/json' --data '{
"input_message": "What is LangSmith?"
}'
```
2. Use the `generate/stream/full` endpoint for evaluating remote workflows.
Sample Usage:
1. Start server on the remote cluster with the base config.yml:
```
aiq serve --config_file=examples/simple/configs/config.yml
```
2. Run evaluation, against the remote endpoint, using a different config.yml that provides the dataset:
```
aiq eval --config_file=examples/simple/configs/eval_config.yml --endpoint http://localhost:8000
```
Closes NVIDIA#51
Authors:
- Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah)
Approvers:
- Eric Evans II (https://github.com/ericevans-nv)
URL: NVIDIA#57
Change done:
1. Added new endpoint `generate/stream/full` to stream the complete IntermediateStep. Sample usage -
```
curl --request POST --url http://localhost:8000/generate/stream/full --header 'Content-Type: application/json' --data '{
"input_message": "What is LangSmith?"
}'
```
2. Use the `generate/stream/full` endpoint for evaluating remote workflows.
Sample Usage:
1. Start server on the remote cluster with the base config.yml:
```
aiq serve --config_file=examples/simple/configs/config.yml
```
2. Run evaluation, against the remote endpoint, using a different config.yml that provides the dataset:
```
aiq eval --config_file=examples/simple/configs/eval_config.yml --endpoint http://localhost:8000
```
Closes NVIDIA#51
Authors:
- Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah)
Approvers:
- Eric Evans II (https://github.com/ericevans-nv)
URL: NVIDIA#57
Change done:
generate/stream/fullto stream the complete IntermediateStep. Sample usage -generate/stream/fullendpoint for evaluating remote workflows.Sample Usage:
Closes #51