docker: Use miniforge, install from pip #134274
Conversation
Switch installation of the pytorch package to be installed from our download.pytorch.org sources which are better maintained. As well, switching over the miniconda installation to a miniforge installation in order to ensure backwards compat for users expecting to have the conda package manager installed. Signed-off-by: Eli Uriegas <eliuriegas@meta.com>
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/134274
Note: Links to docs will display an error until the docs builds have been completed. ✅ You can merge normally! (1 Unrelated Failure)As of commit d3cbabd with merge base ff61f55 ( FLAKY - The following job failed but was likely due to flakiness present on trunk:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
| *) MINICONDA_ARCH=x86_64 ;; \ | ||
| esac && \ | ||
| curl -fsSL -v -o ~/miniconda.sh -O "https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-${MINICONDA_ARCH}.sh" | ||
| curl -fsSL -v -o ~/miniconda.sh -O "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-${MINICONDA_ARCH}.sh" |
There was a problem hiding this comment.
This should probably actually be pinned to a specific version instead of latest
Signed-off-by: Eli Uriegas <eliuriegas@meta.com>
Dockerfile
Outdated
| RUN case ${TARGETPLATFORM} in \ | ||
| "linux/arm64") pip install --extra-index-url https://download.pytorch.org/whl/cpu/ torch torchvision torchaudio ;; \ | ||
| *) /opt/conda/bin/conda install -c "${INSTALL_CHANNEL}" -c "${CUDA_CHANNEL}" -y "python=${PYTHON_VERSION}" pytorch torchvision torchaudio "pytorch-cuda=$(echo $CUDA_VERSION | cut -d'.' -f 1-2)" ;; \ | ||
| *) pip install --extra-index-url https://download.pytorch.org/whl/${CUDA_VERSION#.}/ torch torchvision torchaudio ;; |
There was a problem hiding this comment.
We need INSTALL_CHANNEL logic here to switch between:
https://download.pytorch.org/whl/nightly when INSTALL_CHANNEL==pytorch-nightly
https://download.pytorch.org/whl/test when INSTALL_CHANNEL==pytorch-test
https://download.pytorch.org/whl/ when INSTALL_CHANNEL==pytorch
I believe this is parameter passed from the yml workflow here: https://github.com/pytorch/pytorch/blob/main/.github/workflows/docker-release.yml#L122
There was a problem hiding this comment.
Do you just want to add another switch to rename those to whl/nightly/, whl/test/ and whl/ prefixes?
There was a problem hiding this comment.
Yes I think we can change the variable INSTALL_CHANNEL in https://github.com/pytorch/pytorch/blob/main/.github/workflows/docker-release.yml#L122 to reflect pypi.
And hence here in Docker address will be: https://download.pytorch.org/${INSTALL_CHANNEL}/${CUDA_VERSION#.}/
Signed-off-by: Eli Uriegas <eliuriegas@meta.com>
atalman
force-pushed
the
seemethere/switch_docker_sources
branch
from
f3e0299 to
3daf641
Compare
|
@pytorchmergebot merge -f "lint and docker builds are green" |
Merge startedYour change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
|
@pytorchbot cherry-pick --onto release/2.4 -c critical |
Switch installation of the pytorch package to be installed from our download.pytorch.org sources which are better maintained. As well, switching over the miniconda installation to a miniforge installation in order to ensure backwards compat for users expecting to have the conda package manager installed. Pull Request resolved: #134274 Approved by: https://github.com/malfet, https://github.com/atalman Co-authored-by: atalman <atalman@fb.com> (cherry picked from commit b2eb0e8)
Cherry picking #134274The cherry pick PR is at #134497 and it is recommended to link a critical cherry pick PR with an issue. The following tracker issues are updated: Details for Dev Infra teamRaised by workflow job |
docker: Use miniforge, install from pip (#134274) Switch installation of the pytorch package to be installed from our download.pytorch.org sources which are better maintained. As well, switching over the miniconda installation to a miniforge installation in order to ensure backwards compat for users expecting to have the conda package manager installed. Pull Request resolved: #134274 Approved by: https://github.com/malfet, https://github.com/atalman Co-authored-by: atalman <atalman@fb.com> (cherry picked from commit b2eb0e8) Co-authored-by: Eli Uriegas <eliuriegas@meta.com>
5 participants
Switch installation of the pytorch package to be installed from our download.pytorch.org sources which are better maintained.
As well, switching over the miniconda installation to a miniforge installation in order to ensure backwards compat for users expecting to have the conda package manager installed.