-
-
Notifications
You must be signed in to change notification settings - Fork 1.8k
Description
Describe the problem as clearly as you can
The desired behavior is:
Given I am on a linux musl system,
And I depend on a gem that with native (precompiled) gems,
And that gem ships a-linux-muslvariant,
When Ibundle lock,
The PLATFORMS section of myGemfile.lockshould include my-linux-muslplatform,
And I should bundle the-linux-muslvariant of the gem's native package.
However, in some scenarios, the correct *-linux-musl platform is omitted by bundler, leading to either choosing a less-appropriate platform gem (scenario 1 below), or a non-resolvable lock file (scenario 3 below).
Please note that this issue is probably a blocker for general consumption of musl-specific native gems.
/cc @segiddins @deivid-rodriguez
Did you try upgrading rubygems & bundler?
Yes, this behavior is present with Rubygems 3.5.5 and Bundler 2.5.5.
Post steps to reproduce the problem
Here are three scenarios to explore this problem space. I'm running these commands using the ruby:3.3-alpine docker image, and deleting Gemfile.lock at the beginning of each scenario.
To summarize what you're about to see:
- when bundling a gem with
linuxandlinux-muslvariants, bundler omits the correct platform from the lock file - when bundling a gem with
linux-gnuandlinux-muslvariants, everything works as expected - but when combining scenario 2 with a second gem that publishes a
linuxvariant, the problem re-emerges
Here we go.
1. a gem that publishes *-linux and *-linux-musl native gem variants
Let's use rcee_precompiled v0.5.0 (see https://rubygems.org/gems/rcee_precompiled/versions), which publishes
- rcee_precompiled-0.5.0-x86_64-linux.gem
- rcee_precompiled-0.5.0-x86_64-linux-musl.gem
Gemfile:
# Gemfile
source "https://rubygems.org"
gem "rcee_precompiled", "0.5.0"run bundle lock (or bundle install), and the lock file contains:
GEM
remote: https://rubygems.org/
specs:
rcee_precompiled (0.5.0-aarch64-linux)
rcee_precompiled (0.5.0-aarch64-linux-musl)
rcee_precompiled (0.5.0-arm-linux)
rcee_precompiled (0.5.0-arm-linux-musl)
rcee_precompiled (0.5.0-arm64-darwin)
rcee_precompiled (0.5.0-x86-linux)
rcee_precompiled (0.5.0-x86-linux-musl)
rcee_precompiled (0.5.0-x86_64-darwin)
rcee_precompiled (0.5.0-x86_64-linux)
PLATFORMS
aarch64-linux
aarch64-linux-musl
arm-linux
arm-linux-musl
arm64-darwin
x86-linux
x86-linux-musl
x86_64-darwin
x86_64-linux
DEPENDENCIES
rcee_precompiled (= 0.5.0)
BUNDLED WITH
2.5.5
Note the absence of x86_64-linux-musl in the PLATFORMS section.
If I run bundle show --paths I can see that it's incorrectly chosen the *-linux flavor:
# bundle show --paths
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.5
/usr/local/bundle/gems/rcee_precompiled-0.5.0-x86_64-linux
I can fix this by specifically adding the correct platform to the lock file:
bundle lock --add-platform x86_64-linux-musl
bundle install
then the Gemfile.lock contains x86_64-linux-musl:
PLATFORMS
aarch64-linux
aarch64-linux-musl
arm-linux
arm-linux-musl
arm64-darwin
x86-linux
x86-linux-musl
x86_64-darwin
x86_64-linux
x86_64-linux-musl
and I can see it's now choosing the correct musl native gem variant:
# bundle show --paths
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.5
/usr/local/bundle/gems/rcee_precompiled-0.5.0-x86_64-linux-musl
2. a gem that publishes *-linux-gnu and *-linux-musl native gem variants
Now let's use rcee_precompiled v0.5.1 which switched from *-linux to *-linux-gnu, and publishes
- rcee_precompiled-0.5.0-x86_64-linux-gnu.gem
- rcee_precompiled-0.5.0-x86_64-linux-musl.gem
Gemfile:
# Gemfile
source "https://rubygems.org"
gem "rcee_precompiled", "0.5.1"run bundle lock (or bundle install), and the lock file contains (in part):
PLATFORMS
aarch64-linux-gnu
aarch64-linux-musl
arm-linux-gnu
arm-linux-musl
arm64-darwin
x86-linux-gnu
x86-linux-musl
x86_64-darwin
x86_64-linux-gnu
x86_64-linux-musl
Yay! This works. I see x86_64-linux-musl in the PLATFORMS section and bundle show --paths tells me it's choosing the right variant.
Cool cool cool.
3. but now add another gem that publishes a *-linux platform native gem
Let's add nokogiri:
# Gemfile
source "https://rubygems.org"
gem "rcee_precompiled", "0.5.1"
gem "nokogiri"run bundle lock (or bundle install), and the lock file contains (in part):
PLATFORMS
aarch64-linux
aarch64-linux-gnu
aarch64-linux-musl
arm-linux
arm-linux-gnu
arm-linux-musl
arm64-darwin
x86-linux
x86-linux-gnu
x86-linux-musl
x86_64-darwin
x86_64-linux
x86_64-linux-gnu
Note that x86_64-linux-musl is now missing again! And if I attempt to bundle install I get an error:
Could not find gems matching 'rcee_precompiled (= 0.5.1)' valid for all resolution platforms (aarch64-linux,
aarch64-linux-gnu, aarch64-linux-musl, arm-linux, arm-linux-gnu, arm-linux-musl, arm64-darwin, x86-linux,
x86-linux-gnu, x86-linux-musl, x86_64-darwin, x86_64-linux, x86_64-linux-gnu) in rubygems repository
https://rubygems.org/ or installed locally.
However if I go through the process of:
bundle lock --add-platform x86_64-linux-musl
bundle install
then bundler will include x86_64-linux-musl in the PLATFORMS section of the lock file:
PLATFORMS
aarch64-linux
aarch64-linux-gnu
aarch64-linux-musl
arm-linux
arm-linux-gnu
arm-linux-musl
arm64-darwin
x86-linux
x86-linux-gnu
x86-linux-musl
x86_64-darwin
x86_64-linux
x86_64-linux-gnu
x86_64-linux-musl
and will choose the correct variant again:
# bundle show --paths
/usr/local/lib/ruby/gems/3.3.0/gems/bundler-2.5.5
/usr/local/bundle/gems/nokogiri-1.16.0-x86_64-linux
/usr/local/lib/ruby/gems/3.3.0/gems/racc-1.7.3
/usr/local/bundle/gems/rcee_precompiled-0.5.1-x86_64-linux-musl
What were you expecting to happen?
In scenarios 1 and 3, I expect
bundle lockto addx86_64-linux-muslto the PLATFORMS section of the lock filebundle installto resolve to thex86_64-linux-muslnative gem variant forrcee_precompiled
What actually happened?
Scenarios 1 and 3 seem to confuse bundler, which omits the correct local platform, x86_64-linux-musl, from the lock file.
If not included with the output of your command, run bundle env and paste the output below
At the end of scenario 3, this is what bundle env emits:
Environment
Bundler 2.5.5
Platforms ruby, x86_64-linux-musl
Ruby 3.3.0p0 (2023-12-25 revision 5124f9ac7513eb590c37717337c430cb93caa151) [x86_64-linux-musl]
Full Path /usr/local/bin/ruby
Config Dir /usr/local/etc
RubyGems 3.5.5
Gem Home /usr/local/bundle
Gem Path /root/.local/share/gem/ruby/3.3.0:/usr/local/lib/ruby/gems/3.3.0:/usr/local/bundle
User Home /root
User Path /root/.local/share/gem/ruby/3.3.0
Bin Dir /usr/local/bundle/bin
Tools
Git not installed
RVM not installed
rbenv not installed
chruby not installed
Bundler Build Metadata
Built At 2024-01-18
Git SHA 2efa8cec93
Released Version true
Bundler settings
app_config
Set via BUNDLE_APP_CONFIG: "/usr/local/bundle"
silence_root_warning
Set via BUNDLE_SILENCE_ROOT_WARNING: true
Gemfile
Gemfile
source "https://rubygems.org"
gem "rcee_precompiled", "0.5.1"
gem "nokogiri"Gemfile.lock
GEM
remote: https://rubygems.org/
specs:
nokogiri (1.16.0-aarch64-linux)
racc (~> 1.4)
nokogiri (1.16.0-arm-linux)
racc (~> 1.4)
nokogiri (1.16.0-arm64-darwin)
racc (~> 1.4)
nokogiri (1.16.0-x86-linux)
racc (~> 1.4)
nokogiri (1.16.0-x86_64-darwin)
racc (~> 1.4)
nokogiri (1.16.0-x86_64-linux)
racc (~> 1.4)
racc (1.7.3)
rcee_precompiled (0.5.1-aarch64-linux-gnu)
rcee_precompiled (0.5.1-aarch64-linux-musl)
rcee_precompiled (0.5.1-arm-linux-gnu)
rcee_precompiled (0.5.1-arm-linux-musl)
rcee_precompiled (0.5.1-arm64-darwin)
rcee_precompiled (0.5.1-x86-linux-gnu)
rcee_precompiled (0.5.1-x86-linux-musl)
rcee_precompiled (0.5.1-x86_64-darwin)
rcee_precompiled (0.5.1-x86_64-linux-gnu)
rcee_precompiled (0.5.1-x86_64-linux-musl)
PLATFORMS
aarch64-linux
aarch64-linux-gnu
aarch64-linux-musl
arm-linux
arm-linux-gnu
arm-linux-musl
arm64-darwin
x86-linux
x86-linux-gnu
x86-linux-musl
x86_64-darwin
x86_64-linux
x86_64-linux-gnu
x86_64-linux-musl
DEPENDENCIES
nokogiri
rcee_precompiled (= 0.5.1)
BUNDLED WITH
2.5.5