Add X-Error-Message header for error context #15979

doddi · 2025-09-22T13:20:46Z

Summary

HTTP1.1 RFC 9112 - HTTP/1.1 section 4 defines the response status code to optionally include a text description (human readable) of the reason for the status code.

RFC9113 - HTTP/2 is the HTTP2 protocol standard and the response status only considers the status code and not the reason phrase, as such important information can be lost in helping the client determine a route cause of a failure.

Purpose

Allow error header in a HEAD response

Similar to huggingface have a X-Error-Code and a X-Error-Message header.

X-Error-Code: defines a grouping of the error type
X-Error-Message: detailed message enhancing the reason for the failure

For this PR I opted to only consider the X-Error-Message header to avoid making the solution overly complicated for its need.

This will allow the following:

Enhance the information displayed to the user when an error status is received. Both HEAD and GET requests will be able to provide context.
This solution will allow error information to be transmitted over any http version.

Test Plan

Created a server that additionally add a X-Error-Message header response when an error occurs. Observe in the cli that the message is displayed verbatim.

Note: There are 2 entries for the error in the screenshot above because this test was over http1.1 which additionally sends the same message in the 'traditional' errorPhrase section.

zanieb · 2025-09-22T13:42:17Z

crates/uv-resolver/src/pubgrub/report.rs

+                if let Some(message) = custom_message {
+                    write!(f, "{}{} {}", "hint".bold().cyan(), ":".bold(), message,)
+                } else {
+                    write!(
+                        f,
+                        "{}{} An index URL ({}) could not be queried due to a lack of valid authentication credentials ({}).",
+                        "hint".bold().cyan(),
+                        ":".bold(),
+                        index.without_credentials().cyan(),
+                        "401 Unauthorized".red(),
+                    )
+                }


This is a bit drive-by, as I haven't had time to look at the rest of the pull request in depth, but... it seems wrong to drop the rest of our messaging here. Can you explain the motivation for that? I'd expect the custom messaging to just be an addendum.

I can update to include the existing messaging (not drop) 👍

konstin · 2025-09-25T09:05:45Z

I see there's an example in the screenshot, but can you say a bit more about which server uses this header and how, so we get a complete picture of the user experience?

konstin · 2025-09-25T09:06:18Z

crates/uv-client/src/cached_client.rs

+    ) -> ErrorKind {
+        if matches!(
+            response.status(),
+            http::StatusCode::FORBIDDEN | http::StatusCode::UNAUTHORIZED


Why only for those status codes?

Yeah, if I understand correctly we should probably check for this on 400..=599, i.e. all client- and server- originated error responses?

konstin · 2025-09-25T09:08:28Z

crates/uv-client/src/cached_client.rs

+            response.status(),
+            http::StatusCode::FORBIDDEN | http::StatusCode::UNAUTHORIZED
+        ) {
+            if let Some(custom_message) = crate::error::extract_custom_error_message(response) {


nit: inline this function

konstin · 2025-09-25T09:09:35Z

crates/uv-client/src/error.rs

    fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
-        if self.is_likely_offline() {
+        if let Some(ref custom_msg) = self.custom_message {
+            write!(f, "{custom_msg}")


We need to show that this is something coming from the server and not uv or the os, e.g. Server says:

konstin · 2025-09-25T09:14:10Z

crates/uv-distribution-types/src/status_code_strategy.rs

                    IndexStatusCodeDecision::Ignore
                } else {
-                    Self::Default.handle_status_code(status_code, index_url, capabilities)
+                    Self::Default.handle_status_code_with_message(


Do we need to recurse here or can we do an early return for Self::IgnoreErrorCodes and status_codes.contains(&status_code) instead?

konstin · 2025-09-25T09:19:48Z

crates/uv-client/src/cached_client.rs

        }
    }

+    fn maybe_generate_error_message(


I would make this a WrappedReqwestError method - it seems useful to have this universally available.

konstin · 2025-09-25T09:20:48Z

crates/uv-client/src/error.rs

 }
+
+#[cfg(test)]
+mod tests {


Can you add an integration test? There are some references with MockServer e.g. in pip_install.rs

woodruffw · 2025-09-25T15:36:17Z

I see there's an example in the screenshot, but can you say a bit more about which server uses this header and how, so we get a complete picture of the user experience?

I agree with this -- a quick search online shows that there's a lot of variance in third-party headers used for error messaging (X-Error-Message, X-Error, X-Err-Msg, etc.), and I think accepting one non-standard header would establish a precedent for accepting other non-standard headers (which presents a long term quality/maintenance risk IMO, since we can't assert that these headers are plain text and not JSON or some other serialized form).

@doddi Out of curiosity, have you engaged with the Python packaging standards processes for this use case? I think there would be an immense amount of value in having Python package indices align on error response behavior/encoding in index API (PEP 503/691) responses, and would make the integration story a lot simpler and more uniform here.

doddi · 2025-09-26T13:18:18Z

I see there's an example in the screenshot, but can you say a bit more about which server uses this header and how, so we get a complete picture of the user experience?

I agree with this -- a quick search online shows that there's a lot of variance in third-party headers used for error messaging (X-Error-Message, X-Error, X-Err-Msg, etc.), and I think accepting one non-standard header would establish a precedent for accepting other non-standard headers (which presents a long term quality/maintenance risk IMO, since we can't assert that these headers are plain text and not JSON or some other serialized form).

@doddi Out of curiosity, have you engaged with the Python packaging standards processes for this use case? I think there would be an immense amount of value in having Python package indices align on error response behavior/encoding in index API (PEP 503/691) responses, and would make the integration story a lot simpler and more uniform here.

@woodruffw I have not engaged with the Python packaging standards process, I was not aware of such a thing :). Would you recommend this as a first step? if so could you suggest what would be the best approach?

As for the screenshot this is from Sonatype Nexus Repository Manager, we are currently working through all our supported formats and determining how we can support such error types when communicating using http2. For our use case we generate a 403 response when we determine that a download of a component should be blocked due to violating a policy, such as being malicious or perhaps licensing. The response usually includes a url link to a report on why the download was blocked. Moving to http2 (no reasonPhrase option) the end user is simply presented with a 403 and no context as to why a download 'failed'.

woodruffw · 2025-09-26T15:19:51Z

@woodruffw I have not engaged with the Python packaging standards process, I was not aware of such a thing :). Would you recommend this as a first step? if so could you suggest what would be the best approach?

Yeah, I think this would be a good starting point -- changes to standard interfaces (like the index) are IMO a lot easier to justify if they can be substantiated through the standards process 🙂

In terms of starting: I think a discussion thread on the packaging forum is probably right starting point -- the community there is familiar with the rationale behind the original PEP 503/691 specifications, and will likely have useful feedback on whether making explicit accommodations for error states makes sense.

However, to take a step back, there might be an existing solution that works for you:

For our use case we generate a 403 response when we determine that a download of a component should be blocked due to violating a policy, such as being malicious or perhaps licensing. The response usually includes a url link to a report on why the download was blocked. Moving to http2 (no reasonPhrase option) the end user is simply presented with a 403 and no context as to why a download 'failed'.

Hmm, have you looked at implementing PEP 792 for this? That's a pre-existing standard mechanism that allows the index to communicate to clients whether a project has been quarantined (among other states).

The main restriction with PEP 792 is that it operates at the project level, not the per-file level: you set the status marker in the index response itself. I'm not sure if that makes sense for your use case, but if it does you'll get compatibility with uv and other installers for "free" without having to make any kind of changes or standards efforts 🙂

doddi · 2025-09-26T16:21:04Z

@woodruffw I have not engaged with the Python packaging standards process, I was not aware of such a thing :). Would you recommend this as a first step? if so could you suggest what would be the best approach?

Yeah, I think this would be a good starting point -- changes to standard interfaces (like the index) are IMO a lot easier to justify if they can be substantiated through the standards process 🙂

In terms of starting: I think a discussion thread on the packaging forum is probably right starting point -- the community there is familiar with the rationale behind the original PEP 503/691 specifications, and will likely have useful feedback on whether making explicit accommodations for error states makes sense.

However, to take a step back, there might be an existing solution that works for you:

For our use case we generate a 403 response when we determine that a download of a component should be blocked due to violating a policy, such as being malicious or perhaps licensing. The response usually includes a url link to a report on why the download was blocked. Moving to http2 (no reasonPhrase option) the end user is simply presented with a 403 and no context as to why a download 'failed'.

Hmm, have you looked at implementing PEP 792 for this? That's a pre-existing standard mechanism that allows the index to communicate to clients whether a project has been quarantined (among other states).

The main restriction with PEP 792 is that it operates at the project level, not the per-file level: you set the status marker in the index response itself. I'm not sure if that makes sense for your use case, but if it does you'll get compatibility with uv and other installers for "free" without having to make any kind of changes or standards efforts 🙂

That is really useful information re PEP792 but sadly we need to communicate policy violations on a per version basis. Thanks again for the information, I will reach out to the PEP standards 👍

doddi · 2025-09-30T11:00:59Z

I have opened up a thread for discussion https://discuss.python.org/t/block-download-of-components-when-violating-policy/104021. Probably best I close this PR for now and see what traction I get over there 👍

## Summary HTTP1.1 [RFC 9112 - HTTP/1.1](https://www.rfc-editor.org/rfc/rfc9112.html#name-status-line) section 4 defines the response status code to optionally include a text description (human readable) of the reason for the status code. [RFC9113 - HTTP/2](https://www.rfc-editor.org/rfc/rfc9113) is the HTTP2 protocol standard and the response status only considers the [status code](https://www.rfc-editor.org/rfc/rfc9113#name-response-pseudo-header-fiel) and not the reason phrase, and as such important information can be lost in helping the client determine a route cause of a failure. As per discussion on this [PR](#15979) the current feeling is that implementing the RFC9457 standard might be the preferred route. This PR makes those changes to aid the discussion which has also been moved to the [PEP board](https://discuss.python.org/t/block-download-of-components-when-violating-policy/104021/1) ## Test Plan Pulling components that violate our policy over HTTP2 and without any RFC9457 implementation the following message is presented to the user: <img width="1482" height="104" alt="image" src="https://github.com/user-attachments/assets/0afcd0d8-ca67-4f94-a6c2-131e3b6d8dcc" /> With the RFC9457 standard implemented, below you can see the advantage in the extra context as to why the component has been blocked: <img width="2171" height="127" alt="image" src="https://github.com/user-attachments/assets/25bb5465-955d-4a76-9f30-5477fc2c866f" /> --------- Co-authored-by: konstin <konstin@mailbox.org>

doddi added 2 commits September 22, 2025 14:14

Add X-Error-Message header for error context

42ec21a

clippy and fmt

57223c2

zanieb reviewed Sep 22, 2025

View reviewed changes

doddi added 2 commits September 22, 2025 14:57

Do not drop existing messaging

f686147

more clippy

7526b95

konstin reviewed Sep 25, 2025

View reviewed changes

konstin marked this pull request as draft September 30, 2025 12:44

doddi mentioned this pull request Oct 9, 2025

Implement RFC9457 compliant messaging #16199

Merged

Add X-Error-Message header for error context #15979

Are you sure you want to change the base?

Add X-Error-Message header for error context #15979

Conversation

doddi commented Sep 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Purpose

Test Plan

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

konstin commented Sep 25, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

woodruffw commented Sep 25, 2025

Uh oh!

doddi commented Sep 26, 2025

Uh oh!

woodruffw commented Sep 26, 2025

Uh oh!

doddi commented Sep 26, 2025

Uh oh!

doddi commented Sep 30, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

doddi commented Sep 22, 2025 •

edited

Loading