-
Notifications
You must be signed in to change notification settings - Fork 1.6k
Stop using ctest #5169
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Stop using ctest #5169
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
CaseyCarter
commented
Dec 6, 2024
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as resolved.
This comment was marked as resolved.
69e92c1 to
bfd25fb
Compare
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as resolved.
This comment was marked as resolved.
CaseyCarter
commented
Dec 7, 2024
It's pointless to use ctest to run one of two different LIT commands when we can simply run LIT directly. This will make it easier to add future tests: we can simply add more targets, we won't need to jump through hoops to tell ctest not to run the new test all the time. This change will ease adding a new EDG-only test target for the IntelliSense team.
Detailed changes:
* In `tests/CMakeLists.txt`:
* Replace `add_test` targets with `add_custom_target`s. The `stl` and `stlasan` tests are now `test` and `extra-asan-tests` targets, respectively.
* Replace `list(APPEND unset_variable_name a b c d)` with the equivalent `set(unset_variable_name a b c d)` which makes it clear that we expect `unset_variable_name` to be undefined.
* Merge `${Python_EXECUTABLE}` into `STL_LIT_COMMAND`, and extract `${STL_LIT_TEST_DIRS}`. The makes it easy to reuse `${STL_LIT_COMMAND}` and `${STL_LIT_TEST_DIRS}` with target-specific LIT options.
* Note that `USES_TERMINAL` in the `add_custom_target`s makes ninja run those targets serially, maintaining the observable effects of the removed `set_tests_properties(stl stlasan PROPERTIES RUN_SERIAL ON)`.
* Remove `enable_testing` from the top-level `CMakeLists.txt`; it does nothing now that we have no `add_test`s. We still use `BUILD_TESTING` to control including the test targets in the generated build system, but I'm not certain there's a good reason to do so. Maybe to enable folks without python to build but not test the STL?
* In `run-tests.yml`, replace `ctestOptions` parameter with a `testTargets` parameter, a list of targets to build via `cmake --build meow --target woof quack`. This needs to be set in `asan-pipeline.yml` and plumbed through `build-and-test.yml` with a default of `test` to run the vanilla STL tests.
* Remove/Replace the `ctest` discussion in `README.md`.
* Drive-by: Report that the `psutil` python module is unavailable at most once per `stl-lit` run. Seeing three repetitions in the CI logs bothered me.
b31e90a to
bfd4cb5
Compare
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as resolved.
This comment was marked as resolved.
StephanTLavavej
requested changes
Dec 8, 2024
|
This comment was marked as resolved.
This comment was marked as resolved.
StephanTLavavej
approved these changes
Dec 9, 2024
|
I'm mirroring this to the MSVC-internal repo - please notify me if any further changes are pushed. |
🛑 😻 💚 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
It's pointless to use ctest to run one of two different LIT commands when we can simply run LIT directly. This will make it easier to add future tests: we can simply add more targets, we won't need to jump through hoops to tell ctest not to run the new test all the time. This change will ease adding a new EDG-only test target for the IntelliSense team.
Detailed changes:
tests/CMakeLists.txt:add_testtargets withadd_custom_targets. Thestlandstlasantests are nowtestandextra-asan-teststargets, respectively.list(APPEND unset_variable_name a b c d)with the equivalentset(unset_variable_name a b c d)which makes it clear that we expectunset_variable_nameto be undefined.${Python_EXECUTABLE}intoSTL_LIT_COMMAND, and extract${STL_LIT_TEST_DIRS}. The makes it easy to reuse${STL_LIT_COMMAND}and${STL_LIT_TEST_DIRS}with target-specific LIT options.USES_TERMINALin theadd_custom_targets makes ninja run those targets serially, maintaining the observable effects of the removedset_tests_properties(stl stlasan PROPERTIES RUN_SERIAL ON).enable_testingfrom the top-levelCMakeLists.txt; it does nothing now that we have noadd_tests. We still useBUILD_TESTINGto control including the test targets in the generated build system, but I'm not certain there's a good reason to do so. Maybe to enable folks without python to build but not test the STL?run-tests.yml, replacectestOptionsparameter with atestTargetsparameter, a list of targets to build viacmake --build meow --target woof quack. This needs to be set inasan-pipeline.ymland plumbed throughbuild-and-test.ymlwith a default oftestto run the vanilla STL tests.ctestdiscussion inREADME.md.psutilpython module is unavailable at most once perstl-litrun. Seeing three repetitions in the CI logs bothered me.