KEMBAR78
Add support for environment variable interpolation in config files by mpenn · Pull Request #157 · NVIDIA/NeMo-Agent-Toolkit · GitHub
Skip to content

Conversation

@mpenn
Copy link
Contributor

@mpenn mpenn commented Apr 29, 2025

Description

Closes: #156

This PR adds support for environment variable interpolation in YAML configuration files using the format ${VAR:-default_value}. This allows developers to:

  1. Reference environment variables in configuration
  2. Provide default values if the environment variable is not set
  3. Use empty strings as default values if needed

To illustrate this concept, an example from the llms section of the configuration file is provided below.

llms:
  nim_llm:
    _type: nim
    base_url: ${NIM_BASE_URL:-"http://default.com"}  # Optional with default value
    api_key: ${NIM_API_KEY}  # Will use empty string if `NIM_API_KEY` not set
    model_name: ${MODEL_NAME:-}  # Will use empty string if `MODEL_NAME` not set
    temperature: 0.0

The environment variable interpolation process follow the rules enumerated below.

  • ${VAR} - Uses the value of environment variable VAR, or empty string if not set
  • ${VAR:-default} - Uses the value of environment variable VAR, or default if not set
  • ${VAR:-} - Uses the value of environment variable VAR, or empty string if not set

By Submitting this PR I confirm:

  • I am familiar with the Contributing Guidelines.
  • We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license.
    • Any contribution which contains commits that are not Signed-Off will not be accepted.
  • When the PR is ready for review, new or existing tests cover these changes.
  • When the PR is ready for review, the documentation is up to date with these changes.

@mpenn mpenn requested a review from a team as a code owner April 29, 2025 04:51
@mpenn mpenn added the feature request New feature or request label Apr 29, 2025
@mpenn mpenn force-pushed the mpenn_env-var-interpolation branch 2 times, most recently from 75b1762 to 72bdc81 Compare April 29, 2025 05:05
@mpenn mpenn added the non-breaking Non-breaking change label Apr 29, 2025
@mpenn mpenn self-assigned this Apr 29, 2025
Copy link
Collaborator

@mdemoret-nv mdemoret-nv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good functionality, but I fear we are missing edge cases

mpenn and others added 11 commits April 29, 2025 12:33
…g files and unit tests.

Signed-off-by: mpenn <mpenn@nvidia.com>
Signed-off-by: mpenn <mpenn@nvidia.com>
Signed-off-by: mpenn <mpenn@nvidia.com>
Signed-off-by: mpenn <mpenn@nvidia.com>
## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AgentIQ/blob/develop/docs/source/advanced/contributing.md).
- We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license.
  - Any contribution which contains commits that are not Signed-Off will not be accepted.
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - David Gardner (https://github.com/dagardner-nv)

Approvers:
  - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah)

URL: NVIDIA#154
Signed-off-by: mpenn <mpenn@nvidia.com>
## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AgentIQ/blob/develop/docs/source/advanced/contributing.md).
- We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license.
  - Any contribution which contains commits that are not Signed-Off will not be accepted.
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - David Gardner (https://github.com/dagardner-nv)

Approvers:
  - Anuradha Karuppiah (https://github.com/AnuradhaKaruppiah)

URL: NVIDIA#155
Signed-off-by: mpenn <mpenn@nvidia.com>
- Downgrades Python requirement from `3.12` to `3.11` across all packages
- Adds compatibility layer for `typing.override` decorator `type_utils.py`
- Replaces all `@typing.override` instances with the compatible version

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AgentIQ/blob/develop/docs/source/advanced/contributing.md).
- We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license.
  - Any contribution which contains commits that are not Signed-Off will not be accepted.
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - Yuchen Zhang (https://github.com/yczhang-nv)

Approvers:
  - Michael Demoret (https://github.com/mdemoret-nv)

URL: NVIDIA#148
Signed-off-by: mpenn <mpenn@nvidia.com>
Signed-off-by: mpenn <mpenn@nvidia.com>
@mpenn mpenn force-pushed the mpenn_env-var-interpolation branch from 26a43e3 to 02f20c2 Compare April 29, 2025 16:34
@mpenn mpenn requested a review from mdemoret-nv April 29, 2025 19:55
Copy link
Collaborator

@mdemoret-nv mdemoret-nv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good

…ols.py and test_yaml_tools.py to streamline codebase.

Signed-off-by: Matthew Penn <mpenn@nvidia.com>
@mpenn
Copy link
Contributor Author

mpenn commented Apr 30, 2025

/merge

@rapids-bot rapids-bot bot merged commit f46d207 into NVIDIA:develop Apr 30, 2025
10 checks passed
yczhang-nv pushed a commit to yczhang-nv/NeMo-Agent-Toolkit that referenced this pull request May 8, 2025
…VIDIA#157)

Closes: NVIDIA#156

This PR adds support for environment variable interpolation in YAML configuration files using the format `${VAR:-default_value}`. This allows developers to:

1. Reference environment variables in configuration
2. Provide default values if the environment variable is not set
3. Use empty strings as default values if needed

To illustrate this concept, an example from the `llms` section of the configuration file is provided below.

```yaml
llms:
  nim_llm:
    _type: nim
    base_url: ${NIM_BASE_URL:-"http://default.com"}  # Optional with default value
    api_key: ${NIM_API_KEY}  # Will use empty string if `NIM_API_KEY` not set
    model_name: ${MODEL_NAME:-}  # Will use empty string if `MODEL_NAME` not set
    temperature: 0.0
```

The environment variable interpolation process follow the rules enumerated below.

- `${VAR}` - Uses the value of environment variable `VAR`, or empty string if not set
- `${VAR:-default}` - Uses the value of environment variable `VAR`, or `default` if not set
- `${VAR:-}` - Uses the value of environment variable `VAR`, or empty string if not set

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AgentIQ/blob/develop/docs/source/advanced/contributing.md).
- We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license.
  - Any contribution which contains commits that are not Signed-Off will not be accepted.
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - Matthew Penn (https://github.com/mpenn)
  - David Gardner (https://github.com/dagardner-nv)
  - Yuchen Zhang (https://github.com/yczhang-nv)

Approvers:
  - Michael Demoret (https://github.com/mdemoret-nv)

URL: NVIDIA#157
Signed-off-by: Yuchen Zhang <134643420+yczhang-nv@users.noreply.github.com>
ericevans-nv pushed a commit to ericevans-nv/agent-iq that referenced this pull request Jun 3, 2025
…VIDIA#157)

Closes: NVIDIA#156

This PR adds support for environment variable interpolation in YAML configuration files using the format `${VAR:-default_value}`. This allows developers to:

1. Reference environment variables in configuration
2. Provide default values if the environment variable is not set
3. Use empty strings as default values if needed

To illustrate this concept, an example from the `llms` section of the configuration file is provided below.

```yaml
llms:
  nim_llm:
    _type: nim
    base_url: ${NIM_BASE_URL:-"http://default.com"}  # Optional with default value
    api_key: ${NIM_API_KEY}  # Will use empty string if `NIM_API_KEY` not set
    model_name: ${MODEL_NAME:-}  # Will use empty string if `MODEL_NAME` not set
    temperature: 0.0
```

The environment variable interpolation process follow the rules enumerated below.

- `${VAR}` - Uses the value of environment variable `VAR`, or empty string if not set
- `${VAR:-default}` - Uses the value of environment variable `VAR`, or `default` if not set
- `${VAR:-}` - Uses the value of environment variable `VAR`, or empty string if not set

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AgentIQ/blob/develop/docs/source/advanced/contributing.md).
- We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license.
  - Any contribution which contains commits that are not Signed-Off will not be accepted.
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - Matthew Penn (https://github.com/mpenn)
  - David Gardner (https://github.com/dagardner-nv)
  - Yuchen Zhang (https://github.com/yczhang-nv)

Approvers:
  - Michael Demoret (https://github.com/mdemoret-nv)

URL: NVIDIA#157
Signed-off-by: Eric Evans <194135482+ericevans-nv@users.noreply.github.com>
ericevans-nv pushed a commit to ericevans-nv/agent-iq that referenced this pull request Jun 3, 2025
…VIDIA#157)

Closes: NVIDIA#156

This PR adds support for environment variable interpolation in YAML configuration files using the format `${VAR:-default_value}`. This allows developers to:

1. Reference environment variables in configuration
2. Provide default values if the environment variable is not set
3. Use empty strings as default values if needed

To illustrate this concept, an example from the `llms` section of the configuration file is provided below.

```yaml
llms:
  nim_llm:
    _type: nim
    base_url: ${NIM_BASE_URL:-"http://default.com"}  # Optional with default value
    api_key: ${NIM_API_KEY}  # Will use empty string if `NIM_API_KEY` not set
    model_name: ${MODEL_NAME:-}  # Will use empty string if `MODEL_NAME` not set
    temperature: 0.0
```

The environment variable interpolation process follow the rules enumerated below.

- `${VAR}` - Uses the value of environment variable `VAR`, or empty string if not set
- `${VAR:-default}` - Uses the value of environment variable `VAR`, or `default` if not set
- `${VAR:-}` - Uses the value of environment variable `VAR`, or empty string if not set

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AgentIQ/blob/develop/docs/source/advanced/contributing.md).
- We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license.
  - Any contribution which contains commits that are not Signed-Off will not be accepted.
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - Matthew Penn (https://github.com/mpenn)
  - David Gardner (https://github.com/dagardner-nv)
  - Yuchen Zhang (https://github.com/yczhang-nv)

Approvers:
  - Michael Demoret (https://github.com/mdemoret-nv)

URL: NVIDIA#157
Signed-off-by: Eric Evans <194135482+ericevans-nv@users.noreply.github.com>
AnuradhaKaruppiah pushed a commit to AnuradhaKaruppiah/oss-agentiq that referenced this pull request Aug 4, 2025
…VIDIA#157)

Closes: NVIDIA#156

This PR adds support for environment variable interpolation in YAML configuration files using the format `${VAR:-default_value}`. This allows developers to:

1. Reference environment variables in configuration
2. Provide default values if the environment variable is not set
3. Use empty strings as default values if needed

To illustrate this concept, an example from the `llms` section of the configuration file is provided below.

```yaml
llms:
  nim_llm:
    _type: nim
    base_url: ${NIM_BASE_URL:-"http://default.com"}  # Optional with default value
    api_key: ${NIM_API_KEY}  # Will use empty string if `NIM_API_KEY` not set
    model_name: ${MODEL_NAME:-}  # Will use empty string if `MODEL_NAME` not set
    temperature: 0.0
```

The environment variable interpolation process follow the rules enumerated below.

- `${VAR}` - Uses the value of environment variable `VAR`, or empty string if not set
- `${VAR:-default}` - Uses the value of environment variable `VAR`, or `default` if not set
- `${VAR:-}` - Uses the value of environment variable `VAR`, or empty string if not set

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AgentIQ/blob/develop/docs/source/advanced/contributing.md).
- We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license.
  - Any contribution which contains commits that are not Signed-Off will not be accepted.
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - Matthew Penn (https://github.com/mpenn)
  - David Gardner (https://github.com/dagardner-nv)
  - Yuchen Zhang (https://github.com/yczhang-nv)

Approvers:
  - Michael Demoret (https://github.com/mdemoret-nv)

URL: NVIDIA#157
scheckerNV pushed a commit to scheckerNV/aiq-factory-reset that referenced this pull request Aug 22, 2025
…VIDIA#157)

Closes: NVIDIA#156

This PR adds support for environment variable interpolation in YAML configuration files using the format `${VAR:-default_value}`. This allows developers to:

1. Reference environment variables in configuration
2. Provide default values if the environment variable is not set
3. Use empty strings as default values if needed

To illustrate this concept, an example from the `llms` section of the configuration file is provided below.

```yaml
llms:
  nim_llm:
    _type: nim
    base_url: ${NIM_BASE_URL:-"http://default.com"}  # Optional with default value
    api_key: ${NIM_API_KEY}  # Will use empty string if `NIM_API_KEY` not set
    model_name: ${MODEL_NAME:-}  # Will use empty string if `MODEL_NAME` not set
    temperature: 0.0
```

The environment variable interpolation process follow the rules enumerated below.

- `${VAR}` - Uses the value of environment variable `VAR`, or empty string if not set
- `${VAR:-default}` - Uses the value of environment variable `VAR`, or `default` if not set
- `${VAR:-}` - Uses the value of environment variable `VAR`, or empty string if not set

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/AgentIQ/blob/develop/docs/source/advanced/contributing.md).
- We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license.
  - Any contribution which contains commits that are not Signed-Off will not be accepted.
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - Matthew Penn (https://github.com/mpenn)
  - David Gardner (https://github.com/dagardner-nv)
  - Yuchen Zhang (https://github.com/yczhang-nv)

Approvers:
  - Michael Demoret (https://github.com/mdemoret-nv)

URL: NVIDIA#157
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature request New feature or request non-breaking Non-breaking change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEA]: Support Environment Variable Interpolation in Config Files

4 participants