-
Notifications
You must be signed in to change notification settings - Fork 396
Add support for AWS Bedrock LLM Provider #238
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
3145f3e to
929f30e
Compare
* Fix broken anchor tags and external links in examples * I opened up NVIDIA#176 to prevent this in the future ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AIQToolkit/blob/develop/docs/source/advanced/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - David Gardner (https://github.com/dagardner-nv) Approvers: - Yuchen Zhang (https://github.com/yczhang-nv) URL: NVIDIA#177 Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
* support python 3.11 Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com> * fix doc Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com> * update pyproject.toml Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com> * update uv.lock Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com> * update README to specify Python version requirements Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com> * resolve comments Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com> * fix comments Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com> * remove support for python 3.13 Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com> --------- Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
* Adds temporary transitional packages for users which still have dependencies on the old package names. * Consolidate some CI code. ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AIQToolkit/blob/develop/docs/source/advanced/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - David Gardner (https://github.com/dagardner-nv) Approvers: - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah) URL: NVIDIA#181 Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
Feature: a tunable RAG evaluator
This PR adds a tunable RAG evaluator that allows for flexible evaluation of RAG workflows.
It includes a default scoring mechanism based on an expected answer description rather than a ground truth answer.
Key Points:
- Full control over judge LLM prompt
- Full control over scoring guidelines
- Everything is configurable from the config file
- No changes to dataset format other than answer format etc.
Usage Example:
```
eval:
evaluators:
custom_rag_evaluation:
_type: tunable_rag_evaluator
llm_name: nim_rag_eval_llm
default_scoring: false
default_score_weights:
coverage: 0.5
correctness: 0.3
relevance: 0.2
judge_llm_prompt: >
You are an intelligent evaluator that scores the generated answer based on the description of the expected answer.
The score is a measure of how well the generated answer matches the description of the expected answer based on the question.
Take into account the question, the relevance of the answer to the question and the quality compared to the description of the expected answer.
Rules:
- The score must be a float of any value between 0.0 and 1.0 on a sliding scale.
- The reasoning string must be concise and to the point. It should be 1 sentence and 2 only if extra description is needed. It must explain why the score was given and what is different between the generated answer and the expected answer.
```
## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AgentIQ/blob/develop/docs/source/advanced/contributing.md).
- We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license.
- Any contribution which contains commits that are not Signed-Off will not be accepted.
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.
Authors:
- https://github.com/liamy-nv
- David Gardner (https://github.com/dagardner-nv)
- Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah)
- Michael Demoret (https://github.com/mdemoret-nv)
Approvers:
- Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah)
URL: NVIDIA#110
Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
…DIA#184) Closes NVIDIA#183 This PR fixes errors in the CLI documentation. The section that describes the CLI commands to update locally configured remote registry channels is incorrect. The following command changes are implemented: `aiq configure registry update` is modified to `aiq configure channel update` ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AIQToolkit/blob/develop/docs/source/advanced/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - Matthew Penn (https://github.com/mpenn) Approvers: - David Gardner (https://github.com/dagardner-nv) URL: NVIDIA#184 Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AIQToolkit/blob/develop/docs/source/advanced/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - David Gardner (https://github.com/dagardner-nv) Approvers: - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah) URL: NVIDIA#187 Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
* support python 3.11 Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com> * fix doc Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com> * update pyproject.toml Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com> * update uv.lock Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com> * update README to specify Python version requirements Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com> * resolve comments Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com> * fix comments Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com> * add support of ImageContext and AudioContext to AIQChatRequest Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com> --------- Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
Rendered doc here: https://github.com/AnuradhaKaruppiah/oss-agentiq/blob/doc-typo/examples/simple_calculator/README.md#accuracy-evaluation ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AIQToolkit/blob/develop/docs/source/advanced/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah) Approvers: - David Gardner (https://github.com/dagardner-nv) URL: NVIDIA#188 Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
Closes This PR adds support tracing of AIQ intemediate steps and the underlying framework (Langchain, CrewAI, etc.) using Weights and Biases [Weave](https://weave-docs.wandb.ai) platform for better interactivity and off-the-shelf LLMOps life-cycle. As a user all you need to do is update the `config.yaml` with the following: ``` general: use_uvloop: true telemetry: tracing: weave: _type: weave entity: <your-wandb-entity> project: my-awesome-genai-app ``` An example trace timeline logged to Weave: <img width="1728" alt="image" src="https://github.com/user-attachments/assets/382d4926-b115-4514-a453-c58cf550a457" /> View more traces in this project: https://wandb.ai/ayut/simple_calculator/weave/traces?view=traces_default cc: @mdemoret-nv This PR was opened in favor of NVIDIA#135 ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AgentIQ/blob/develop/docs/source/advanced/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - Ayush Thakur (https://github.com/ayulockin) - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah) Approvers: - Dhruv Nandakumar (https://github.com/dnandakumar-nv) URL: NVIDIA#170 Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AIQToolkit/blob/develop/docs/source/advanced/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah) Approvers: - Yuchen Zhang (https://github.com/yczhang-nv) URL: NVIDIA#190 Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
…IDIA#163) * Note the previous name of the project to avoid confusing users. ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AIQToolkit/blob/develop/docs/source/advanced/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - David Gardner (https://github.com/dagardner-nv) Approvers: - Michael Demoret (https://github.com/mdemoret-nv) URL: NVIDIA#163 Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
changed `concurrency` to `max_concurrency` in config parameters of _evaluate.md_ ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AIQToolkit/blob/develop/docs/source/advanced/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - Soumili Nandi (https://github.com/soumilinandi) Approvers: - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah) URL: NVIDIA#191 Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
Closes ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AIQToolkit/blob/develop/docs/source/advanced/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah) Approvers: - Eric Evans II (https://github.com/ericevans-nv) URL: NVIDIA#195 Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
* support python 3.11 Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com> * fix doc Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com> * update pyproject.toml Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com> * update uv.lock Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com> * update README to specify Python version requirements Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com> * resolve comments Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com> * fix comments Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com> * updated simple example eval dataset Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com> * update the nim_rag_eval_llm config Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com> --------- Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
…ut.json (NVIDIA#198) Closes ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AIQToolkit/blob/develop/docs/source/advanced/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah) Approvers: - Dhruv Nandakumar (https://github.com/dnandakumar-nv) URL: NVIDIA#198 Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
…A#204) Closes NVIDIA#202 ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AIQToolkit/blob/develop/docs/source/advanced/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah) Approvers: - Eric Evans II (https://github.com/ericevans-nv) URL: NVIDIA#204 Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
This PR makes `opentelemetry` and `phoenix` as optional dependencies for Omniverse integration. - Moved the telemetry dependencies from AIQ core dependency to a separate group of optional dependency, which can be installed by `uv pip install -e '.[telemetry]'`. - Created `src/aiq/utils/optional_imports.py` to include optional import functionalities and dummy implementations. This module can be reused if we want to add more optional dependencies in the future. - When the configuration file contains `tracing` section but the telemetry dependencies are not installed, `TelemetryOptionalImportError` will be raised - Updated files that imports the optional dependencies to gracefully handle the case when those dependencies are not installed - Updated logging and documentation Closes [AIQ-1064](https://jirasw.nvidia.com/browse/AIQ-1064) ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AIQToolkit/blob/develop/docs/source/advanced/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - Yuchen Zhang (https://github.com/yczhang-nv) Approvers: - Michael Demoret (https://github.com/mdemoret-nv) URL: NVIDIA#167 Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
…#130) This pull request introduces the ability to extract and utilize user-defined attributes from incoming HTTP requests. These attributes can be accessed via the UserManagerAttributes class within the AIQContext and used in workflow tools. ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AgentIQ/blob/develop/docs/source/advanced/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - Eric Evans II (https://github.com/ericevans-nv) Approvers: - Michael Demoret (https://github.com/mdemoret-nv) URL: NVIDIA#130 Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
Steps to repro:
Install latest AiqToolKit from source and run the simple-calculator example:
```
aiq run --config_file examples/simple_cal
culator/configs/config.yml --input "What is the product of 3 and 7, and is it greater than the curre
nt hour?"
```
Following traceback is seen:
```
2025-05-05 11:18:30,130 - aiq.cli.commands.start - ERROR - Failed to initialize workflow
Traceback (most recent call last):
File "/home/devcontainers/dev/forks/agentiq/src/aiq/cli/commands/start.py", line 229, in invoke_subcommand
return asyncio.run(run_plugin())
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/devcontainers/dev/forks/agentiq/.venv/lib/python3.12/site-packages/nest_asyncio.py", line 30, in run
return loop.run_until_complete(task)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/devcontainers/dev/forks/agentiq/.venv/lib/python3.12/site-packages/nest_asyncio.py", line 98, in run_until_complete
return f.result()
^^^^^^^^^^
File "/home/devcontainers/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/python3.12/asyncio/futures.py", line 202, in result
raise self._exception.with_traceback(self._exception_tb)
File "/home/devcontainers/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/python3.12/asyncio/tasks.py", line 316, in __step_run_and_handle_result
result = coro.throw(exc)
^^^^^^^^^^^^^^^
File "/home/devcontainers/dev/forks/agentiq/src/aiq/cli/commands/start.py", line 227, in run_plugin
await front_end_plugin.run()
File "/home/devcontainers/dev/forks/agentiq/src/aiq/front_ends/console/console_front_end_plugin.py", line 74, in run
await self.run_workflow(session_manager)
File "/home/devcontainers/dev/forks/agentiq/src/aiq/front_ends/console/console_front_end_plugin.py", line 94, in run_workflow
runner_outputs = await asyncio.gather(*[run_single_query(query) for query in input_list])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/devcontainers/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/python3.12/asyncio/tasks.py", line 385, in __wakeup
future.result()
File "/home/devcontainers/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/python3.12/asyncio/tasks.py", line 314, in __step_run_and_handle_result
result = coro.send(None)
^^^^^^^^^^^^^^^
File "/home/devcontainers/dev/forks/agentiq/src/aiq/front_ends/console/console_front_end_plugin.py", line 84, in run_single_query
async with session_manager.session(user_input_callback=prompt_for_input_cli) as session:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/devcontainers/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/lib/python3.12/contextlib.py", line 210, in __aenter__
return await anext(self.gen)
^^^^^^^^^^^^^^^^^^^^^
File "/home/devcontainers/dev/forks/agentiq/src/aiq/runtime/session.py", line 100, in session
self.set_request_attributes(request)
File "/home/devcontainers/dev/forks/agentiq/src/aiq/runtime/session.py", line 127, in set_request_attributes
self._context.metadata._request.method = request.method
^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'method'
Error: 'NoneType' object has no attribute 'method'
```
This PR adds a sanity check to prevent request access unless it is present.
## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AIQToolkit/blob/develop/docs/source/advanced/contributing.md).
- We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license.
- Any contribution which contains commits that are not Signed-Off will not be accepted.
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.
Authors:
- Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah)
Approvers:
- Eric Evans II (https://github.com/ericevans-nv)
URL: NVIDIA#209
Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
This PR adds a new multi-agent example for alert triaging. Tested offline and verified the output to match expected labels on the test dataset (in the `data/` folder). Authors: - https://github.com/hsin-c - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah) Approvers: - David Gardner (https://github.com/dagardner-nv) URL: NVIDIA#193 Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
Closes ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AIQToolkit/blob/develop/docs/source/advanced/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - Eric Evans II (https://github.com/ericevans-nv) Approvers: - David Gardner (https://github.com/dagardner-nv) URL: NVIDIA#211 Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
* All examples, plugins and transitional packages now declare a version dependency on `aiqtoolkit` * Adds an `ci/release/update-version.sh` script to automate the updating of versions at release time. * Add a `ci/release/update_toml_dep.py` (called by `ci/release/update-version.sh`) to parse and update individual `pyproject.toml` files. * Add a new dev dependency on `tomlkit` (this was already present in the env as a transitive dep). * Ensure that nightly builds declare explicit dependencies on the nightly version number Closes NVIDIA#197 ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AIQToolkit/blob/develop/docs/source/advanced/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - David Gardner (https://github.com/dagardner-nv) Approvers: - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah) URL: NVIDIA#208 Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
This PR introduces a new example agent, profiler agent. Users can use this agent to chat with the profiling traces in their AgentIQ agent application and use it as an interactive profiling tool. https://github.com/user-attachments/assets/318f9a1e-fb5a-47dd-a9f4-32e088aebd51 This agent requires users to configure AgentIQ with profiling and telemetry features enabled, following [the documentation](https://docs.nvidia.com/agentiq/latest/guides/profiler.html) ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AgentIQ/blob/develop/docs/source/advanced/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - Zac Wang (https://github.com/zac-wang-nv) - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah) Approvers: - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah) URL: NVIDIA#120 Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
…example and fixed broken link in `README` (NVIDIA#210) - Re-generated `uv.lock` - Clean up `pyproject.toml` for profiler agent example and fix broken link in `README` ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AIQToolkit/blob/develop/docs/source/advanced/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - Yuchen Zhang (https://github.com/yczhang-nv) - Zac Wang (https://github.com/zac-wang-nv) - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah) Approvers: - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah) URL: NVIDIA#210 Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
Closes NVIDIA#151 ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AIQToolkit/blob/develop/docs/source/advanced/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - Hritik Raj (https://github.com/Hritik003) - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah) Approvers: - David Gardner (https://github.com/dagardner-nv) URL: NVIDIA#186 Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
Closes ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AIQToolkit/blob/develop/docs/source/advanced/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - Victor Yudin (https://github.com/VictorYudin) Approvers: - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah) URL: NVIDIA#215 Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
mdemoret-nv
approved these changes
May 29, 2025
|
@lvojtku Need your review on the docs changes |
…gentIQ into yuchen-fix-aiq-1213
Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
Signed-off-by: Yuchen Zhang <yuchenz@nvidia.com>
|
@lvojtku I'll get this merged after having your approval on the documentation changes |
|
/merge |
gfreeman-nvidia
pushed a commit
to gfreeman-nvidia/AIQToolkit
that referenced
this pull request
May 30, 2025
- Add support for AWS Bedrock LLM provider. The provider can be accessed by type `aws_bedrock`, and it is compatible with both `langchain` and `llamaindex` - Added unit tests for existing combinations of existing LLM providers and LLM frameworks. The unit tests are skipped by default, since each of them needs credential to interact with LLM models, but it should be updated and checked each time we add/modify the LLM provider/client to make sure they are still working - Fixed OpenAI + `llamaindex` LLM client that is not working Closes [AIQ-1213](https://jirasw.nvidia.com/browse/AIQ-1213) ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AIQToolkit/blob/develop/docs/source/advanced/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - Yuchen Zhang (https://github.com/yczhang-nv) - David Gardner (https://github.com/dagardner-nv) - https://github.com/liamy-nv - Matthew Penn (https://github.com/mpenn) - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah) - Ayush Thakur (https://github.com/ayulockin) - Soumili Nandi (https://github.com/soumilinandi) - Eric Evans II (https://github.com/ericevans-nv) - https://github.com/hsin-c - Zac Wang (https://github.com/zac-wang-nv) - Hritik Raj (https://github.com/Hritik003) - Victor Yudin (https://github.com/VictorYudin) - Dhruv Nandakumar (https://github.com/dnandakumar-nv) - Michael Demoret (https://github.com/mdemoret-nv) Approvers: - Michael Demoret (https://github.com/mdemoret-nv) URL: NVIDIA#238 Signed-off-by: Greg Freeman <gfreeman@nvidia.com>
ericevans-nv
pushed a commit
to ericevans-nv/agent-iq
that referenced
this pull request
Jun 3, 2025
- Add support for AWS Bedrock LLM provider. The provider can be accessed by type `aws_bedrock`, and it is compatible with both `langchain` and `llamaindex` - Added unit tests for existing combinations of existing LLM providers and LLM frameworks. The unit tests are skipped by default, since each of them needs credential to interact with LLM models, but it should be updated and checked each time we add/modify the LLM provider/client to make sure they are still working - Fixed OpenAI + `llamaindex` LLM client that is not working Closes [AIQ-1213](https://jirasw.nvidia.com/browse/AIQ-1213) ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AIQToolkit/blob/develop/docs/source/advanced/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - Yuchen Zhang (https://github.com/yczhang-nv) - David Gardner (https://github.com/dagardner-nv) - https://github.com/liamy-nv - Matthew Penn (https://github.com/mpenn) - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah) - Ayush Thakur (https://github.com/ayulockin) - Soumili Nandi (https://github.com/soumilinandi) - Eric Evans II (https://github.com/ericevans-nv) - https://github.com/hsin-c - Zac Wang (https://github.com/zac-wang-nv) - Hritik Raj (https://github.com/Hritik003) - Victor Yudin (https://github.com/VictorYudin) - Dhruv Nandakumar (https://github.com/dnandakumar-nv) - Michael Demoret (https://github.com/mdemoret-nv) Approvers: - Michael Demoret (https://github.com/mdemoret-nv) URL: NVIDIA#238 Signed-off-by: Eric Evans <194135482+ericevans-nv@users.noreply.github.com>
ericevans-nv
pushed a commit
to ericevans-nv/agent-iq
that referenced
this pull request
Jun 3, 2025
- Add support for AWS Bedrock LLM provider. The provider can be accessed by type `aws_bedrock`, and it is compatible with both `langchain` and `llamaindex` - Added unit tests for existing combinations of existing LLM providers and LLM frameworks. The unit tests are skipped by default, since each of them needs credential to interact with LLM models, but it should be updated and checked each time we add/modify the LLM provider/client to make sure they are still working - Fixed OpenAI + `llamaindex` LLM client that is not working Closes [AIQ-1213](https://jirasw.nvidia.com/browse/AIQ-1213) ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AIQToolkit/blob/develop/docs/source/advanced/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - Yuchen Zhang (https://github.com/yczhang-nv) - David Gardner (https://github.com/dagardner-nv) - https://github.com/liamy-nv - Matthew Penn (https://github.com/mpenn) - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah) - Ayush Thakur (https://github.com/ayulockin) - Soumili Nandi (https://github.com/soumilinandi) - Eric Evans II (https://github.com/ericevans-nv) - https://github.com/hsin-c - Zac Wang (https://github.com/zac-wang-nv) - Hritik Raj (https://github.com/Hritik003) - Victor Yudin (https://github.com/VictorYudin) - Dhruv Nandakumar (https://github.com/dnandakumar-nv) - Michael Demoret (https://github.com/mdemoret-nv) Approvers: - Michael Demoret (https://github.com/mdemoret-nv) URL: NVIDIA#238 Signed-off-by: Eric Evans <194135482+ericevans-nv@users.noreply.github.com>
ericevans-nv
pushed a commit
to ericevans-nv/agent-iq
that referenced
this pull request
Jun 3, 2025
- Add support for AWS Bedrock LLM provider. The provider can be accessed by type `aws_bedrock`, and it is compatible with both `langchain` and `llamaindex` - Added unit tests for existing combinations of existing LLM providers and LLM frameworks. The unit tests are skipped by default, since each of them needs credential to interact with LLM models, but it should be updated and checked each time we add/modify the LLM provider/client to make sure they are still working - Fixed OpenAI + `llamaindex` LLM client that is not working Closes [AIQ-1213](https://jirasw.nvidia.com/browse/AIQ-1213) ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AIQToolkit/blob/develop/docs/source/advanced/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - Yuchen Zhang (https://github.com/yczhang-nv) - David Gardner (https://github.com/dagardner-nv) - https://github.com/liamy-nv - Matthew Penn (https://github.com/mpenn) - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah) - Ayush Thakur (https://github.com/ayulockin) - Soumili Nandi (https://github.com/soumilinandi) - Eric Evans II (https://github.com/ericevans-nv) - https://github.com/hsin-c - Zac Wang (https://github.com/zac-wang-nv) - Hritik Raj (https://github.com/Hritik003) - Victor Yudin (https://github.com/VictorYudin) - Dhruv Nandakumar (https://github.com/dnandakumar-nv) - Michael Demoret (https://github.com/mdemoret-nv) Approvers: - Michael Demoret (https://github.com/mdemoret-nv) URL: NVIDIA#238 Signed-off-by: Eric Evans <194135482+ericevans-nv@users.noreply.github.com>
AnuradhaKaruppiah
pushed a commit
to AnuradhaKaruppiah/oss-agentiq
that referenced
this pull request
Aug 4, 2025
- Add support for AWS Bedrock LLM provider. The provider can be accessed by type `aws_bedrock`, and it is compatible with both `langchain` and `llamaindex` - Added unit tests for existing combinations of existing LLM providers and LLM frameworks. The unit tests are skipped by default, since each of them needs credential to interact with LLM models, but it should be updated and checked each time we add/modify the LLM provider/client to make sure they are still working - Fixed OpenAI + `llamaindex` LLM client that is not working Closes [AIQ-1213](https://jirasw.nvidia.com/browse/AIQ-1213) ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AIQToolkit/blob/develop/docs/source/advanced/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - Yuchen Zhang (https://github.com/yczhang-nv) - David Gardner (https://github.com/dagardner-nv) - https://github.com/liamy-nv - Matthew Penn (https://github.com/mpenn) - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah) - Ayush Thakur (https://github.com/ayulockin) - Soumili Nandi (https://github.com/soumilinandi) - Eric Evans II (https://github.com/ericevans-nv) - https://github.com/hsin-c - Zac Wang (https://github.com/zac-wang-nv) - Hritik Raj (https://github.com/Hritik003) - Victor Yudin (https://github.com/VictorYudin) - Dhruv Nandakumar (https://github.com/dnandakumar-nv) - Michael Demoret (https://github.com/mdemoret-nv) Approvers: - Michael Demoret (https://github.com/mdemoret-nv) URL: NVIDIA#238
scheckerNV
pushed a commit
to scheckerNV/aiq-factory-reset
that referenced
this pull request
Aug 22, 2025
- Add support for AWS Bedrock LLM provider. The provider can be accessed by type `aws_bedrock`, and it is compatible with both `langchain` and `llamaindex` - Added unit tests for existing combinations of existing LLM providers and LLM frameworks. The unit tests are skipped by default, since each of them needs credential to interact with LLM models, but it should be updated and checked each time we add/modify the LLM provider/client to make sure they are still working - Fixed OpenAI + `llamaindex` LLM client that is not working Closes [AIQ-1213](https://jirasw.nvidia.com/browse/AIQ-1213) ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AIQToolkit/blob/develop/docs/source/advanced/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - Yuchen Zhang (https://github.com/yczhang-nv) - David Gardner (https://github.com/dagardner-nv) - https://github.com/liamy-nv - Matthew Penn (https://github.com/mpenn) - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah) - Ayush Thakur (https://github.com/ayulockin) - Soumili Nandi (https://github.com/soumilinandi) - Eric Evans II (https://github.com/ericevans-nv) - https://github.com/hsin-c - Zac Wang (https://github.com/zac-wang-nv) - Hritik Raj (https://github.com/Hritik003) - Victor Yudin (https://github.com/VictorYudin) - Dhruv Nandakumar (https://github.com/dnandakumar-nv) - Michael Demoret (https://github.com/mdemoret-nv) Approvers: - Michael Demoret (https://github.com/mdemoret-nv) URL: NVIDIA#238
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
aws_bedrock, and it is compatible with bothlangchainandllamaindexllamaindexLLM client that is not workingCloses AIQ-1213
By Submitting this PR I confirm: