4beb9fc5d3
Previously, `spack test` automatically passed all of its arguments to `pytest -k` if no options were provided, and to `pytest` if they were. `spack test -l` also provided a list of test filenames, but they didn't really let you completely narrow down which tests you wanted to run. Instead of trying to do our own weird thing, this passes `spack test` args directly to `pytest`, and omits the implicit `-k`. This means we can now run, e.g.: ```console $ spack test spec_syntax.py::TestSpecSyntax::test_ambiguous ``` This wasn't possible before, because we'd pass the fully qualified name to `pytest -k` and get an error. Because `pytest` doesn't have the greatest ability to list tests, I've tweaked the `-l`/`--list`, `-L`/`--list-long`, and `-N`/`--list-names` options to `spack test` so that they help you understand the names better. you can combine these options with `-k` or other arguments to do pretty powerful searches. This one makes it easy to get a list of names so you can run tests in different orders (something I find useful for debugging `pytest` issues): ```console $ spack test --list-names -k "spec and concretize" cmd/env.py::test_concretize_user_specs_together concretize.py::TestConcretize::test_conflicts_in_spec concretize.py::TestConcretize::test_find_spec_children concretize.py::TestConcretize::test_find_spec_none concretize.py::TestConcretize::test_find_spec_parents concretize.py::TestConcretize::test_find_spec_self concretize.py::TestConcretize::test_find_spec_sibling concretize.py::TestConcretize::test_no_matching_compiler_specs concretize.py::TestConcretize::test_simultaneous_concretization_of_specs spec_dag.py::TestSpecDag::test_concretize_deptypes spec_dag.py::TestSpecDag::test_copy_concretized ``` You can combine any list option with keywords: ```console $ spack test --list -k microarchitecture llnl/util/cpu.py modules/lmod.py ``` ```console $ spack test --list-long -k microarchitecture llnl/util/cpu.py:: test_generic_microarchitecture modules/lmod.py::TestLmod:: test_only_generic_microarchitectures_in_root ``` Or just list specific files: ```console $ spack test --list-long cmd/test.py cmd/test.py:: test_list test_list_names_with_pytest_arg test_list_long test_list_with_keywords test_list_long_with_pytest_arg test_list_with_pytest_arg test_list_names ``` Hopefully this stuff will help with debugging test issues. - [x] make `spack test` send args directly to `pytest` instead of trying to do fancy things. - [x] rework `--list`, `--list-long`, and add `--list-names` to make searching for tests easier. - [x] make it possible to mix Spack's list args with `pytest` args (they're just fancy parsing around `pytest --collect-only`) - [x] add docs - [x] add tests - [x] update spack completion |
||
---|---|---|
.. | ||
spack |