KEMBAR78
Download new API files once per Homebrew instance by Rylan12 · Pull Request #20619 · Homebrew/brew · GitHub
Skip to content

Conversation

Rylan12
Copy link
Member

@Rylan12 Rylan12 commented Sep 1, 2025

For consistency, we don’t want to download new API files mid-run, even if new data is available on the server. This PR modifies the API fetcher logic to only attempt to fetch the data once per execution.

To clarify why mtime and --time-cond are insufficient here: these checks simply ensure that enough time has passed in between fetches, and that a new fetch will actually yield a new file. However, when building from source, it's possible that an installation will take long enough that new data is available and enough time has passed that Homebrew::API considers the file to be stale. This means that future calls to API data will try to download the data once again, which is what this PR is attempting to prevent.

Copy link
Member

@MikeMcQuaid MikeMcQuaid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Good to merge as-is but would be nice to cleanup as follow-up.

@Rylan12 Rylan12 force-pushed the single-api-fetch-per-run branch 2 times, most recently from 74a2487 to 85d76d1 Compare September 1, 2025 20:20
@Rylan12
Copy link
Member Author

Rylan12 commented Sep 1, 2025

I pushed a new, much simpler approach. Now, use environment variables to indicate when a download has already occurred. This is preferable because it will apply to any sub-processes as well (e.g. any brew install that we trigger within Homebrew, build.rb, postinstall.rb, test.rb)

Also, to clarify why mtime and --time-cond are insufficient here: these checks simply ensure that enough time has passed in between fetches, and that a new fetch will actually yield a new file. However, when building from source, it's possible that an installation will take long enough that new data is available and enough time has passed that Homebrew::API considers the file to be stale. This means that future calls to API data will try to download the data once again, which is what this PR is attempting to prevent. I've updated the description to make this clearer.

Note: I haven't tested this thoroughly yet on my machine, but I will before merging

@Rylan12 Rylan12 force-pushed the single-api-fetch-per-run branch 2 times, most recently from 9e2ec79 to c31be49 Compare September 1, 2025 21:33
@Rylan12 Rylan12 marked this pull request as draft September 2, 2025 08:39
@Rylan12 Rylan12 force-pushed the single-api-fetch-per-run branch from c31be49 to 262d9cc Compare September 3, 2025 19:28
@Rylan12 Rylan12 force-pushed the single-api-fetch-per-run branch from 262d9cc to 00f9601 Compare September 3, 2025 19:30
@Rylan12 Rylan12 marked this pull request as ready for review September 3, 2025 20:08
Copy link
Member

@MikeMcQuaid MikeMcQuaid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this approach makes sense, appreciate the patience here, nice work @Rylan12!

@MikeMcQuaid MikeMcQuaid added this pull request to the merge queue Sep 4, 2025
Merged via the queue into main with commit c73538a Sep 4, 2025
36 checks passed
@MikeMcQuaid MikeMcQuaid deleted the single-api-fetch-per-run branch September 4, 2025 07:59
@Rylan12
Copy link
Member Author

Rylan12 commented Sep 4, 2025

Thanks for the help and iteration here @MikeMcQuaid!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants