Add S3 request signing #15925

charliermarsh · 2025-09-18T03:00:16Z

Summary

This PR enables users to mark a URL as an S3 endpoint, at which point uv will sign requests to that URL by detecting credentials from the standard AWS environment variables, configuration files, etc.

Signing is handled by the reqsign crate, which we can also use in the future to sign requests for other providers.

charliermarsh · 2025-09-18T03:01:16Z

Cargo.lock

+ "num-traits",
+ "wasm-bindgen",
+ "windows-link 0.2.0",
+]


I'd like to add Jiff support upstream if we can, possibly before merging this \cc @BurntSushi

From a very quick glance, it's probably non-trivial for them to support both. Depending.

Anywho, I opened an issue: apache/opendal-reqsign#631

crates/uv-auth/src/credentials.rs

crates/uv/src/commands/project/sync.rs

crates/uv-auth/src/providers.rs

zanieb · 2025-09-18T12:35:47Z

Pretty easy!

charliermarsh · 2025-09-22T19:04:14Z

I think this is ready for a real review.

zanieb · 2025-09-22T22:28:44Z

crates/uv-auth/src/credentials.rs

+                    .method(request.method().clone())
+                    .uri(uri)
+                    .body(())
+                    .unwrap();


Should we note why this unwrap is safe?

This MR contains the following updates: | Package | Update | Change | |---|---|---| | [astral-sh/uv](https://github.com/astral-sh/uv) | patch | `0.8.17` -> `0.8.22` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>astral-sh/uv (astral-sh/uv)</summary> ### [`v0.8.22`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0822) [Compare Source](astral-sh/uv@0.8.21...0.8.22) Released on 2025-09-23. ##### Python - Upgrade Pyodide to 0.28.3 ([#15999](astral-sh/uv#15999)) ##### Security - Upgrade `astral-tokio-tar` to 0.5.5 which [hardens tar archive extraction](GHSA-3wgq-wrwc-vqmv) ([#16004](astral-sh/uv#16004)) ### [`v0.8.21`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0821) [Compare Source](astral-sh/uv@0.8.20...0.8.21) Released on 2025-09-23. ##### Enhancements - Refresh lockfile when `--refresh` is provided ([#15994](astral-sh/uv#15994)) ##### Preview features Add support for S3 request signing ([#15925](astral-sh/uv#15925)) ### [`v0.8.20`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0820) [Compare Source](astral-sh/uv@0.8.19...0.8.20) Released on 2025-09-22. ##### Enhancements - Add `--force` flag for `uv cache clean` ([#15992](astral-sh/uv#15992)) - Improve resolution errors with proxied packages ([#15200](astral-sh/uv#15200)) ##### Preview features - Allow upgrading pre-release versions of the same minor Python version ([#15959](astral-sh/uv#15959)) ##### Bug fixes - Hide `freethreaded+debug` Python downloads in `uv python list` ([#15985](astral-sh/uv#15985)) - Retain the cache lock and temporary caches during `uv run` and `uvx` ([#15990](astral-sh/uv#15990)) ##### Documentation - Add `package` level conflicts to the conflicting dependencies docs ([#15963](astral-sh/uv#15963)) - Document pyodide support ([#15962](astral-sh/uv#15962)) - Document support for free-threaded and debug Python versions ([#15961](astral-sh/uv#15961)) - Expand the contribution docs on issue selection ([#15966](astral-sh/uv#15966)) - Tweak title for viewing version in project guide ([#15964](astral-sh/uv#15964)) ### [`v0.8.19`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0819) [Compare Source](astral-sh/uv@0.8.18...0.8.19) Released on 2025-09-19. ##### Python - Add CPython 3.14.0rc3 - Upgrade OpenSSL to 3.5.3 See the [python-build-standalone release notes](https://github.com/astral-sh/python-build-standalone/releases/tag/20250918) for more details. ##### Bug fixes - Make `uv cache clean` parallel process safe ([#15888](astral-sh/uv#15888)) - Fix implied `platform_machine` marker for `win_arm64` platform tag ([#15921](astral-sh/uv#15921)) ### [`v0.8.18`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0818) [Compare Source](astral-sh/uv@0.8.17...0.8.18) Released on 2025-09-17. ##### Enhancements - Add PyG packages to torch backend ([#15911](astral-sh/uv#15911)) - Add handling for unnamed conda environments in base environment detection ([#15681](astral-sh/uv#15681)) - Allow selection of debug build interpreters ([#11520](astral-sh/uv#11520)) - Improve `uv init` defaults for native build backend cache keys ([#15705](astral-sh/uv#15705)) - Error when `pyproject.toml` target does not exist for dependency groups ([#15831](astral-sh/uv#15831)) - Infer check URL from publish URL when known ([#15886](astral-sh/uv#15886)) - Support Gitlab CI/CD as a trusted publisher ([#15583](astral-sh/uv#15583)) - Add GraalPy 25.0.0 with support for Python 3.12 ([#15900](astral-sh/uv#15900)) - Add `--no-clear` to `uv venv` to disable removal prompts ([#15795](astral-sh/uv#15795)) - Add conflict detection between `--only-group` and `--extra` flags ([#15788](astral-sh/uv#15788)) - Allow `[project]` to be missing from a `pyproject.toml` ([#14113](astral-sh/uv#14113)) - Always treat conda environments named `base` and `root` as base environments ([#15682](astral-sh/uv#15682)) - Improve log message when direct build for `uv_build` is skipped ([#15898](astral-sh/uv#15898)) - Log when the cache is disabled ([#15828](astral-sh/uv#15828)) - Show pyx organization name after authenticating ([#15823](astral-sh/uv#15823)) - Use `_CONDA_ROOT` to detect Conda base environments ([#15680](astral-sh/uv#15680)) - Include blake2b hash in `uv publish` upload form ([#15794](astral-sh/uv#15794)) - Fix misleading debug message when removing environments in `uv sync` ([#15881](astral-sh/uv#15881)) ##### Deprecations - Deprecate `tool.uv.dev-dependencies` ([#15469](astral-sh/uv#15469)) - Revert "feat(ci): build loongarch64 binaries in CI ([#15387](astral-sh/uv#15387))" ([#15820](astral-sh/uv#15820)) ##### Preview features - Propagate preview flag to client for `native-auth` feature ([#15872](astral-sh/uv#15872)) - Store native credentials for realms with the https scheme stripped ([#15879](astral-sh/uv#15879)) - Use the root index URL when retrieving credentials from the native store ([#15873](astral-sh/uv#15873)) ##### Bug fixes - Fix `uv sync --no-sources` not switching from editable to registry installations ([#15234](astral-sh/uv#15234)) - Avoid display of an empty string when a path is the working directory ([#15897](astral-sh/uv#15897)) - Allow cached environment reuse with `@latest` ([#15827](astral-sh/uv#15827)) - Allow escaping spaces in --env-file handling ([#15815](astral-sh/uv#15815)) - Avoid ANSI codes in debug! messages ([#15843](astral-sh/uv#15843)) - Improve BSD tag construction ([#15829](astral-sh/uv#15829)) - Include SHA when listing lockfile changes ([#15817](astral-sh/uv#15817)) - Invert the logic for determining if a path is a base conda environment ([#15679](astral-sh/uv#15679)) - Load credentials for explicit members when lowering ([#15844](astral-sh/uv#15844)) - Re-add `triton` as a torch backend package ([#15910](astral-sh/uv#15910)) - Respect `UV_INSECURE_NO_ZIP_VALIDATION=1` in duplicate header errors ([#15912](astral-sh/uv#15912)) ##### Documentation - Add GitHub Actions to PyPI trusted publishing example ([#15753](astral-sh/uv#15753)) - Add Coiled integration documentation ([#14430](astral-sh/uv#14430)) - Add verbose output to the getting help section ([#15915](astral-sh/uv#15915)) - Document `NO_PROXY` support ([#15816](astral-sh/uv#15816)) - Document cache-keys for native build backends ([#15811](astral-sh/uv#15811)) - Add documentation for dependency group `requires-python` ([#14282](astral-sh/uv#14282)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).

## Summary This PR enables users to mark a URL as an S3 endpoint, at which point uv will sign requests to that URL by detecting credentials from the standard AWS environment variables, configuration files, etc. Signing is handled by the [reqsign](https://docs.rs/reqsign/latest/reqsign/) crate, which we can also use in the future to sign requests for other providers.

@BurntSushi

This follows up #15925. cc @BurntSushi @charliermarsh Signed-off-by: tison <wander4096@gmail.com>

charliermarsh requested a review from zanieb September 18, 2025 03:00

charliermarsh commented Sep 18, 2025

View reviewed changes

crates/uv-auth/src/credentials.rs Show resolved Hide resolved

charliermarsh had a problem deploying to uv-test-registries September 18, 2025 03:03 — with GitHub Actions Error

charliermarsh commented Sep 18, 2025

View reviewed changes

crates/uv/src/commands/project/sync.rs Show resolved Hide resolved

charliermarsh force-pushed the charlie/s3 branch 2 times, most recently from 8a22c31 to 416b2d7 Compare September 18, 2025 03:04

charliermarsh temporarily deployed to uv-test-registries September 18, 2025 03:11 — with GitHub Actions Inactive

zanieb reviewed Sep 18, 2025

View reviewed changes

crates/uv-auth/src/providers.rs Outdated Show resolved Hide resolved

zanieb reviewed Sep 18, 2025

View reviewed changes

crates/uv-auth/src/providers.rs Outdated Show resolved Hide resolved

BurntSushi mentioned this pull request Sep 18, 2025

Add support for jiff instead of chrono apache/opendal-reqsign#631

Closed

charliermarsh force-pushed the charlie/s3 branch 2 times, most recently from d0a28fb to 8283afc Compare September 19, 2025 02:01

charliermarsh temporarily deployed to uv-test-registries September 19, 2025 02:08 — with GitHub Actions Inactive

charliermarsh temporarily deployed to uv-test-registries September 19, 2025 02:39 — with GitHub Actions Inactive

charliermarsh force-pushed the charlie/s3 branch from 9bb4c4f to 08c7928 Compare September 20, 2025 15:29

charliermarsh temporarily deployed to uv-test-registries September 20, 2025 15:32 — with GitHub Actions Inactive

charliermarsh force-pushed the charlie/s3 branch from 08c7928 to 560b4f6 Compare September 22, 2025 00:25

charliermarsh temporarily deployed to uv-test-registries September 22, 2025 00:29 — with GitHub Actions Inactive

charliermarsh force-pushed the charlie/s3 branch 2 times, most recently from 0a6a7f2 to 714366e Compare September 22, 2025 18:42

charliermarsh temporarily deployed to uv-test-registries September 22, 2025 18:49 — with GitHub Actions Inactive

charliermarsh requested a review from zanieb September 22, 2025 18:49

charliermarsh marked this pull request as ready for review September 22, 2025 18:49

charliermarsh added enhancement New feature or improvement to existing functionality preview Experimental behavior labels Sep 22, 2025

zanieb reviewed Sep 22, 2025

View reviewed changes

zanieb approved these changes Sep 22, 2025

View reviewed changes

charliermarsh added 5 commits September 22, 2025 19:48

Add reqsign

876a0be

Use latest

364c362

Redo struct hierarchy

4512a28

Add is_subdomain_of

5a12a8d

Add preview flag

6ac0ea4

charliermarsh force-pushed the charlie/s3 branch from 714366e to 6ac0ea4 Compare September 22, 2025 23:48

charliermarsh enabled auto-merge (squash) September 22, 2025 23:49

charliermarsh temporarily deployed to uv-test-registries September 22, 2025 23:50 — with GitHub Actions Inactive

charliermarsh merged commit 7f7fac8 into main Sep 22, 2025
159 checks passed

charliermarsh deleted the charlie/s3 branch September 22, 2025 23:59

BrewTestBot mentioned this pull request Sep 23, 2025

uv 0.8.21 Homebrew/homebrew-core#245395

Merged

tisonkun mentioned this pull request Oct 8, 2025

Upgrade reqsign to 0.18.0 to remove chrono deps #16172

Merged

charliermarsh pushed a commit that referenced this pull request Oct 8, 2025

Upgrade reqsign to 0.18.0 to remove chrono deps (#16172)

aadf103

This follows up #15925. cc @BurntSushi @charliermarsh Signed-off-by: tison <wander4096@gmail.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add S3 request signing #15925

Add S3 request signing #15925

Uh oh!

charliermarsh commented Sep 18, 2025 •

edited

Loading

Uh oh!

charliermarsh Sep 18, 2025

Uh oh!

BurntSushi Sep 18, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

zanieb commented Sep 18, 2025

Uh oh!

charliermarsh commented Sep 22, 2025

Uh oh!

zanieb Sep 22, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Add S3 request signing #15925

Add S3 request signing #15925

Uh oh!

Conversation

charliermarsh commented Sep 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

charliermarsh Sep 18, 2025

Choose a reason for hiding this comment

Uh oh!

BurntSushi Sep 18, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

zanieb commented Sep 18, 2025

Uh oh!

charliermarsh commented Sep 22, 2025

Uh oh!

zanieb Sep 22, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

charliermarsh commented Sep 18, 2025 •

edited

Loading