[CI] Move Windows build/tests to Python-3.10 #162862

malfet · 2025-09-12T23:24:50Z

Stack from ghstack (oldest at bottom):

What supposed to be a very simple change end up being quite involved, as current Windows CI framework is quite inflexible, i.e. it takes a lots of argument, but later on ignores them, namely:

PYTHON_VERSION used to be a no-op that is simply ignored by the scripts
With this change, setup-win action will create an environment called py_tmp with specific python version + intel-openmp (that is hard runtime requirement, but for some reason not packaged into the wheel nor marked as such)
Copied test type dependencies from https://github.com/pytorch/test-infra/blob/be01a40157c36cd5a48391fdf44a7bc3ebd4c7e3/aws/ami/windows/scripts/Installers/Install-Pip-Dependencies.ps1#L16 into win-test.sh, but made some adjustments to be compatible with 3.10 runtime (scipy version update) and just make rerun-tests compatible with the rest of the deps

I think in the long run, one needs to update https://github.com/pytorch/test-infra/blob/4432e2cacd8a5bb7a46158e71d08c937c502e35a/aws/ami/windows/scripts/Installers/Install-Miniconda3.ps1 that currently pins Miniconda python to 3.9, but also figure out how CI can still create a new environment without having to download all the dependencies all the time

[ghstack-poisoned]

pytorch-bot · 2025-09-12T23:24:53Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/162862

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 3 Pending

As of commit 89e2564 with merge base cfc539f ():

NEW FAILURE - The following job has failed:

trunk / linux-jammy-rocm-py3.10 / test (distributed, 1, 1, linux.rocm.gpu.gfx942.4) (gh)
Process completed with exit code 134.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

ghstack-source-id: 4fa1f3d Pull Request resolved: #162862

huydhn

OK as a workaround I guess, but maybe we need to remember to revert this one the AMI is updated

malfet · 2025-09-13T00:17:39Z

OK as a workaround I guess, but maybe we need to remember to revert this one the AMI is updated

I'll wait for CI to be green, and then push a change with the TODO comment...

[ghstack-poisoned]

ghstack-source-id: 6c7b185 Pull Request resolved: #162862

malfet · 2025-09-13T01:04:04Z

@pytorchbot merge -f "Let's test in prod"

[ghstack-poisoned]

malfet · 2025-09-14T15:16:38Z

CI is green, but all those settings are lie, not sure why it happening, but Python version are still 3.9...

The right way is to update the AMI, as https://github.com/pytorch/test-infra/blob/4432e2cacd8a5bb7a46158e71d08c937c502e35a/aws/ami/windows/scripts/Installers/Install-Miniconda3.ps1 pins Miniconda python to 3.9 [ghstack-poisoned]

ghstack-source-id: 3f1f804 Pull Request resolved: #162862

The right way is to update the AMI, as https://github.com/pytorch/test-infra/blob/4432e2cacd8a5bb7a46158e71d08c937c502e35a/aws/ami/windows/scripts/Installers/Install-Miniconda3.ps1 pins Miniconda python to 3.9 [ghstack-poisoned]

…162877) Related to #162862 Pull Request resolved: #162877 Approved by: https://github.com/malfet

[ghstack-poisoned]

ghstack-source-id: b563ba6 Pull Request resolved: #162862

[ghstack-poisoned]

ghstack-source-id: 7dff944 Pull Request resolved: #162862

This reverts commit 3016616. Reverted pytorch#162310 on behalf of https://github.com/malfet due to Breaks some windows tests ([comment](pytorch#162862 (comment)))

This reverts commit 2dcd153. Reverted pytorch#162862 on behalf of https://github.com/malfet due to Breaks some windows tests ([comment](pytorch#162862 (comment)))

What supposed to be a very simple change end up being quite involved, as current Windows CI framework is quite inflexible, i.e. it takes a lots of argument, but later on ignores them, namely: - `PYTHON_VERSION` used to be a no-op that is simply ignored by the scripts - With this change, `setup-win` action will create an environment called `py_tmp` with specific python version + intel-openmp (that is hard runtime requirement, but for some reason not packaged into the wheel nor marked as such) - Copied test type dependencies from https://github.com/pytorch/test-infra/blob/be01a40157c36cd5a48391fdf44a7bc3ebd4c7e3/aws/ami/windows/scripts/Installers/Install-Pip-Dependencies.ps1#L16 into `win-test.sh`, but made some adjustments to be compatible with 3.10 runtime (scipy version update) and just make rerun-tests compatible with the rest of the deps I think in the long run, one needs to update https://github.com/pytorch/test-infra/blob/4432e2cacd8a5bb7a46158e71d08c937c502e35a/aws/ami/windows/scripts/Installers/Install-Miniconda3.ps1 that currently pins Miniconda python to 3.9, but also figure out how CI can still create a new environment without having to download all the dependencies all the time Pull Request resolved: pytorch#162862 Approved by: https://github.com/wdvr, https://github.com/huydhn ghstack dependencies: pytorch#163339, pytorch#163341

…ytorch#162877) Related to pytorch#162862 Pull Request resolved: pytorch#162877 Approved by: https://github.com/malfet

What supposed to be a very simple change end up being quite involved, as current Windows CI framework is quite inflexible, i.e. it takes a lots of argument, but later on ignores them, namely: - `PYTHON_VERSION` used to be a no-op that is simply ignored by the scripts - With this change, `setup-win` action will create an environment called `py_tmp` with specific python version + intel-openmp (that is hard runtime requirement, but for some reason not packaged into the wheel nor marked as such) - Introduced `CONDA_ROOT_DIR` env variable in `activate_miniconda3.bat` to avoid `%CONDA_PARENT_DIR%\Miniconda3` invocations throughout the codebase - Copied test type dependencies from https://github.com/pytorch/test-infra/blob/be01a40157c36cd5a48391fdf44a7bc3ebd4c7e3/aws/ami/windows/scripts/Installers/Install-Pip-Dependencies.ps1#L16 into `win-test.sh`, but made some adjustments to be compatible with 3.10 runtime (scipy version update) and just make rerun-tests compatible with the rest of the deps I think in the long run, one needs to update https://github.com/pytorch/test-infra/blob/4432e2cacd8a5bb7a46158e71d08c937c502e35a/aws/ami/windows/scripts/Installers/Install-Miniconda3.ps1 that currently pins Miniconda python to 3.9, but also figure out how CI can still create a new environment without having to download all the dependencies all the time Pull Request resolved: pytorch#162862 Approved by: https://github.com/wdvr, https://github.com/huydhn

Pull Request resolved: pytorch#162310 Approved by: https://github.com/atalman, https://github.com/Skylion007, https://github.com/ZainRizvi ghstack dependencies: pytorch#162862

This reverts commit 3016616. Reverted pytorch#162310 on behalf of https://github.com/malfet due to Breaks some windows tests ([comment](pytorch#162862 (comment)))

This reverts commit 2dcd153. Reverted pytorch#162862 on behalf of https://github.com/malfet due to Breaks some windows tests ([comment](pytorch#162862 (comment)))

What supposed to be a very simple change end up being quite involved, as current Windows CI framework is quite inflexible, i.e. it takes a lots of argument, but later on ignores them, namely: - `PYTHON_VERSION` used to be a no-op that is simply ignored by the scripts - With this change, `setup-win` action will create an environment called `py_tmp` with specific python version + intel-openmp (that is hard runtime requirement, but for some reason not packaged into the wheel nor marked as such) - Copied test type dependencies from https://github.com/pytorch/test-infra/blob/be01a40157c36cd5a48391fdf44a7bc3ebd4c7e3/aws/ami/windows/scripts/Installers/Install-Pip-Dependencies.ps1#L16 into `win-test.sh`, but made some adjustments to be compatible with 3.10 runtime (scipy version update) and just make rerun-tests compatible with the rest of the deps I think in the long run, one needs to update https://github.com/pytorch/test-infra/blob/4432e2cacd8a5bb7a46158e71d08c937c502e35a/aws/ami/windows/scripts/Installers/Install-Miniconda3.ps1 that currently pins Miniconda python to 3.9, but also figure out how CI can still create a new environment without having to download all the dependencies all the time Pull Request resolved: pytorch#162862 Approved by: https://github.com/wdvr, https://github.com/huydhn ghstack dependencies: pytorch#163339, pytorch#163341

Camyll · 2025-09-22T20:16:30Z

@pytorchbot cherry-pick --onto release/2.9 --c critical --fixes #161167

pytorchbot · 2025-09-22T20:21:38Z

Cherry picking #162862

Command git -C /home/runner/work/pytorch/pytorch cherry-pick -x 0098e5636d3afa7c75aef8c447a5c402ea9ed524 returned non-zero exit code 1

Auto-merging .ci/pytorch/win-test-helpers/build_pytorch.bat
CONFLICT (content): Merge conflict in .ci/pytorch/win-test-helpers/build_pytorch.bat
Auto-merging .ci/pytorch/win-test-helpers/installation-helpers/activate_miniconda3.bat
CONFLICT (content): Merge conflict in .ci/pytorch/win-test-helpers/installation-helpers/activate_miniconda3.bat
Auto-merging .ci/pytorch/win-test-helpers/setup_pytorch_env.bat
CONFLICT (content): Merge conflict in .ci/pytorch/win-test-helpers/setup_pytorch_env.bat
Auto-merging .github/workflows/_win-build.yml
Auto-merging .github/workflows/_win-test.yml
error: could not apply 0098e5636d3... [CI] Move Windows build/tests to Python-3.10 (#162862)
hint: After resolving the conflicts, mark them with
hint: "git add/rm <pathspec>", then run
hint: "git cherry-pick --continue".
hint: You can instead skip this commit with "git cherry-pick --skip".
hint: To abort and get back to the state before "git cherry-pick",
hint: run "git cherry-pick --abort".
hint: Disable this message with "git config set advice.mergeConflict false"

Details for Dev Infra team

Raised by workflow job

What supposed to be a very simple change end up being quite involved, as current Windows CI framework is quite inflexible, i.e. it takes a lots of argument, but later on ignores them, namely: - `PYTHON_VERSION` used to be a no-op that is simply ignored by the scripts - With this change, `setup-win` action will create an environment called `py_tmp` with specific python version + intel-openmp (that is hard runtime requirement, but for some reason not packaged into the wheel nor marked as such) - Copied test type dependencies from https://github.com/pytorch/test-infra/blob/be01a40157c36cd5a48391fdf44a7bc3ebd4c7e3/aws/ami/windows/scripts/Installers/Install-Pip-Dependencies.ps1#L16 into `win-test.sh`, but made some adjustments to be compatible with 3.10 runtime (scipy version update) and just make rerun-tests compatible with the rest of the deps I think in the long run, one needs to update https://github.com/pytorch/test-infra/blob/4432e2cacd8a5bb7a46158e71d08c937c502e35a/aws/ami/windows/scripts/Installers/Install-Miniconda3.ps1 that currently pins Miniconda python to 3.9, but also figure out how CI can still create a new environment without having to download all the dependencies all the time Pull Request resolved: #162862 Approved by: https://github.com/wdvr, https://github.com/huydhn ghstack dependencies: #163339, #163341

Pull Request resolved: #162310 Approved by: https://github.com/atalman, https://github.com/Skylion007, https://github.com/ZainRizvi ghstack dependencies: #162862

Pull Request resolved: #162310 Approved by: https://github.com/atalman, https://github.com/Skylion007, https://github.com/ZainRizvi ghstack dependencies: #162862 Co-authored-by: Nikita Shulga <nikita.shulga@gmail.com>

…63800) [CI] Move Windows build/tests to Python-3.10 (#162862) What supposed to be a very simple change end up being quite involved, as current Windows CI framework is quite inflexible, i.e. it takes a lots of argument, but later on ignores them, namely: - `PYTHON_VERSION` used to be a no-op that is simply ignored by the scripts - With this change, `setup-win` action will create an environment called `py_tmp` with specific python version + intel-openmp (that is hard runtime requirement, but for some reason not packaged into the wheel nor marked as such) - Copied test type dependencies from https://github.com/pytorch/test-infra/blob/be01a40157c36cd5a48391fdf44a7bc3ebd4c7e3/aws/ami/windows/scripts/Installers/Install-Pip-Dependencies.ps1#L16 into `win-test.sh`, but made some adjustments to be compatible with 3.10 runtime (scipy version update) and just make rerun-tests compatible with the rest of the deps I think in the long run, one needs to update https://github.com/pytorch/test-infra/blob/4432e2cacd8a5bb7a46158e71d08c937c502e35a/aws/ami/windows/scripts/Installers/Install-Miniconda3.ps1 that currently pins Miniconda python to 3.9, but also figure out how CI can still create a new environment without having to download all the dependencies all the time Pull Request resolved: #162862 Approved by: https://github.com/wdvr, https://github.com/huydhn ghstack dependencies: #163339, #163341 Co-authored-by: Nikita Shulga <nikita.shulga@gmail.com>

…ytorch#162877) Related to pytorch#162862 Pull Request resolved: pytorch#162877 Approved by: https://github.com/malfet

What supposed to be a very simple change end up being quite involved, as current Windows CI framework is quite inflexible, i.e. it takes a lots of argument, but later on ignores them, namely: - `PYTHON_VERSION` used to be a no-op that is simply ignored by the scripts - With this change, `setup-win` action will create an environment called `py_tmp` with specific python version + intel-openmp (that is hard runtime requirement, but for some reason not packaged into the wheel nor marked as such) - Introduced `CONDA_ROOT_DIR` env variable in `activate_miniconda3.bat` to avoid `%CONDA_PARENT_DIR%\Miniconda3` invocations throughout the codebase - Copied test type dependencies from https://github.com/pytorch/test-infra/blob/be01a40157c36cd5a48391fdf44a7bc3ebd4c7e3/aws/ami/windows/scripts/Installers/Install-Pip-Dependencies.ps1#L16 into `win-test.sh`, but made some adjustments to be compatible with 3.10 runtime (scipy version update) and just make rerun-tests compatible with the rest of the deps I think in the long run, one needs to update https://github.com/pytorch/test-infra/blob/4432e2cacd8a5bb7a46158e71d08c937c502e35a/aws/ami/windows/scripts/Installers/Install-Miniconda3.ps1 that currently pins Miniconda python to 3.9, but also figure out how CI can still create a new environment without having to download all the dependencies all the time Pull Request resolved: pytorch#162862 Approved by: https://github.com/wdvr, https://github.com/huydhn

Pull Request resolved: pytorch#162310 Approved by: https://github.com/atalman, https://github.com/Skylion007, https://github.com/ZainRizvi ghstack dependencies: pytorch#162862

This reverts commit 3016616. Reverted pytorch#162310 on behalf of https://github.com/malfet due to Breaks some windows tests ([comment](pytorch#162862 (comment)))

This reverts commit 2dcd153. Reverted pytorch#162862 on behalf of https://github.com/malfet due to Breaks some windows tests ([comment](pytorch#162862 (comment)))

What supposed to be a very simple change end up being quite involved, as current Windows CI framework is quite inflexible, i.e. it takes a lots of argument, but later on ignores them, namely: - `PYTHON_VERSION` used to be a no-op that is simply ignored by the scripts - With this change, `setup-win` action will create an environment called `py_tmp` with specific python version + intel-openmp (that is hard runtime requirement, but for some reason not packaged into the wheel nor marked as such) - Copied test type dependencies from https://github.com/pytorch/test-infra/blob/be01a40157c36cd5a48391fdf44a7bc3ebd4c7e3/aws/ami/windows/scripts/Installers/Install-Pip-Dependencies.ps1#L16 into `win-test.sh`, but made some adjustments to be compatible with 3.10 runtime (scipy version update) and just make rerun-tests compatible with the rest of the deps I think in the long run, one needs to update https://github.com/pytorch/test-infra/blob/4432e2cacd8a5bb7a46158e71d08c937c502e35a/aws/ami/windows/scripts/Installers/Install-Miniconda3.ps1 that currently pins Miniconda python to 3.9, but also figure out how CI can still create a new environment without having to download all the dependencies all the time Pull Request resolved: pytorch#162862 Approved by: https://github.com/wdvr, https://github.com/huydhn ghstack dependencies: pytorch#163339, pytorch#163341

…162877) Related to #162862 Pull Request resolved: #162877 Approved by: https://github.com/malfet (cherry picked from commit c9e57d7)

…164182) [CI] Move libtorch-cpu-shared-with-deps-release-build to python 3.10 (#162877) Related to #162862 Pull Request resolved: #162877 Approved by: https://github.com/malfet (cherry picked from commit c9e57d7) Co-authored-by: atalman <atalman@fb.com>

Update

1977560

[ghstack-poisoned]

malfet requested a review from a team as a code owner September 12, 2025 23:24

pytorch-bot bot added the topic: not user facing topic category label Sep 12, 2025

malfet added a commit that referenced this pull request Sep 12, 2025

[CI] Move Windows build/tests to Python-3.10

647f3e3

ghstack-source-id: 4fa1f3d Pull Request resolved: #162862

malfet added the ciflow/trunk Trigger trunk jobs on your pull request label Sep 12, 2025

wdvr approved these changes Sep 13, 2025

View reviewed changes

huydhn approved these changes Sep 13, 2025

View reviewed changes

Update

adf5fc4

[ghstack-poisoned]

malfet added a commit that referenced this pull request Sep 13, 2025

[CI] Move Windows build/tests to Python-3.10

025d6f7

ghstack-source-id: 6c7b185 Pull Request resolved: #162862

malfet mentioned this pull request Sep 13, 2025

[BE] Update Python min version to 3.10 #162310

Closed

Update on "[CI] Move Windows build/tests to Python-3.10"

714ab3a

[ghstack-poisoned]

atalman mentioned this pull request Sep 13, 2025

[CI] Move libtorch-cpu-shared-with-deps-release-build to python 3.10 #162877

Closed

atalman added this to the 2.9.0 milestone Sep 13, 2025

malfet added 5 commits September 14, 2025 08:28

Update on "[CI] Move Windows build/tests to Python-3.10"

41fa36e

The right way is to update the AMI, as https://github.com/pytorch/test-infra/blob/4432e2cacd8a5bb7a46158e71d08c937c502e35a/aws/ami/windows/scripts/Installers/Install-Miniconda3.ps1 pins Miniconda python to 3.9 [ghstack-poisoned]

Update on "[CI] Move Windows build/tests to Python-3.10"

f900377

The right way is to update the AMI, as https://github.com/pytorch/test-infra/blob/4432e2cacd8a5bb7a46158e71d08c937c502e35a/aws/ami/windows/scripts/Installers/Install-Miniconda3.ps1 pins Miniconda python to 3.9 [ghstack-poisoned]

Update on "[CI] Move Windows build/tests to Python-3.10"

cc9c6de

The right way is to update the AMI, as https://github.com/pytorch/test-infra/blob/4432e2cacd8a5bb7a46158e71d08c937c502e35a/aws/ami/windows/scripts/Installers/Install-Miniconda3.ps1 pins Miniconda python to 3.9 [ghstack-poisoned]

Update on "[CI] Move Windows build/tests to Python-3.10"

dd162ea

The right way is to update the AMI, as https://github.com/pytorch/test-infra/blob/4432e2cacd8a5bb7a46158e71d08c937c502e35a/aws/ami/windows/scripts/Installers/Install-Miniconda3.ps1 pins Miniconda python to 3.9 [ghstack-poisoned]

Update on "[CI] Move Windows build/tests to Python-3.10"

3596add

The right way is to update the AMI, as https://github.com/pytorch/test-infra/blob/4432e2cacd8a5bb7a46158e71d08c937c502e35a/aws/ami/windows/scripts/Installers/Install-Miniconda3.ps1 pins Miniconda python to 3.9 [ghstack-poisoned]

malfet added a commit that referenced this pull request Sep 14, 2025

[CI] Move Windows build/tests to Python-3.10

e6ddb01

ghstack-source-id: 3f1f804 Pull Request resolved: #162862

Update on "[CI] Move Windows build/tests to Python-3.10"

8a570ce

The right way is to update the AMI, as https://github.com/pytorch/test-infra/blob/4432e2cacd8a5bb7a46158e71d08c937c502e35a/aws/ami/windows/scripts/Installers/Install-Miniconda3.ps1 pins Miniconda python to 3.9 [ghstack-poisoned]

pytorchmergebot pushed a commit that referenced this pull request Sep 15, 2025

[CI] Move libtorch-cpu-shared-with-deps-release-build to python 3.10 (#…

c9e57d7

…162877) Related to #162862 Pull Request resolved: #162877 Approved by: https://github.com/malfet

Update

cc3ff38

[ghstack-poisoned]

malfet added a commit that referenced this pull request Sep 15, 2025

[CI] Move Windows build/tests to Python-3.10

ef1a691

ghstack-source-id: b563ba6 Pull Request resolved: #162862

Update

dc5b124

[ghstack-poisoned]

malfet added a commit that referenced this pull request Sep 15, 2025

[CI] Move Windows build/tests to Python-3.10

9637be2

ghstack-source-id: 7dff944 Pull Request resolved: #162862

malfet added the no-runner-experiments Bypass Meta/LF runner determinator label Sep 15, 2025

Camyll mentioned this pull request Sep 24, 2025

[cherrypick] [CI] Move Windows build/tests to Python-3.10 #162862 #163800

Merged

Camyll pushed a commit that referenced this pull request Sep 24, 2025

[BE] Update Python min version to 3.10 (#162310)

cc64790

Pull Request resolved: #162310 Approved by: https://github.com/atalman, https://github.com/Skylion007, https://github.com/ZainRizvi ghstack dependencies: #162862

Camyll mentioned this pull request Sep 24, 2025

[BE] Update Python min version to 3.10 (#162310) #163802

Merged

This was referenced Sep 24, 2025

Move ROCM trunk wheel builds to 3.10 #163804

Merged

[BE] Introduce CONDA_ROOT_DIR #163805

Merged

Camyll mentioned this pull request Sep 24, 2025

[v.2.9.0] Release Tracker #162497

Closed

pytorchbot mentioned this pull request Sep 29, 2025

[CI] Move libtorch-cpu-shared-with-deps-release-build to python 3.10 #164182

Merged

github-actions bot deleted the gh/malfet/519/head branch October 23, 2025 02:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[CI] Move Windows build/tests to Python-3.10 #162862

[CI] Move Windows build/tests to Python-3.10 #162862

Uh oh!

malfet commented Sep 12, 2025 •

edited

Loading

Uh oh!

pytorch-bot bot commented Sep 12, 2025 •

edited

Loading

Uh oh!

huydhn left a comment

Uh oh!

malfet commented Sep 13, 2025

Uh oh!

malfet commented Sep 13, 2025

Uh oh!

malfet commented Sep 14, 2025

Uh oh!

Camyll commented Sep 22, 2025

Uh oh!

pytorchbot commented Sep 22, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

[CI] Move Windows build/tests to Python-3.10 #162862

[CI] Move Windows build/tests to Python-3.10 #162862

Uh oh!

Conversation

malfet commented Sep 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Sep 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/162862

❌ 1 New Failure, 3 Pending

Uh oh!

huydhn left a comment

Choose a reason for hiding this comment

Uh oh!

malfet commented Sep 13, 2025

Uh oh!

malfet commented Sep 13, 2025

Uh oh!

malfet commented Sep 14, 2025

Uh oh!

Camyll commented Sep 22, 2025

Uh oh!

pytorchbot commented Sep 22, 2025

Cherry picking #162862

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

malfet commented Sep 12, 2025 •

edited

Loading

pytorch-bot bot commented Sep 12, 2025 •

edited

Loading