Commit graph

13667 commits

Author SHA1 Message Date
Todd Gamblin
4beb9fc5d3 tests: improved spack test command line options
Previously, `spack test` automatically passed all of its arguments to
`pytest -k` if no options were provided, and to `pytest` if they were.
`spack test -l` also provided a list of test filenames, but they didn't
really let you completely narrow down which tests you wanted to run.

Instead of trying to do our own weird thing, this passes `spack test`
args directly to `pytest`, and omits the implicit `-k`.  This means we
can now run, e.g.:

```console
$ spack test spec_syntax.py::TestSpecSyntax::test_ambiguous
```

This wasn't possible before, because we'd pass the fully qualified name
to `pytest -k` and get an error.

Because `pytest` doesn't have the greatest ability to list tests, I've
tweaked the `-l`/`--list`, `-L`/`--list-long`, and `-N`/`--list-names`
options to `spack test` so that they help you understand the names
better.  you can combine these options with `-k` or other arguments to do
pretty powerful searches.

This one makes it easy to get a list of names so you can run tests in
different orders (something I find useful for debugging `pytest` issues):

```console
$ spack test --list-names -k "spec and concretize"
cmd/env.py::test_concretize_user_specs_together
concretize.py::TestConcretize::test_conflicts_in_spec
concretize.py::TestConcretize::test_find_spec_children
concretize.py::TestConcretize::test_find_spec_none
concretize.py::TestConcretize::test_find_spec_parents
concretize.py::TestConcretize::test_find_spec_self
concretize.py::TestConcretize::test_find_spec_sibling
concretize.py::TestConcretize::test_no_matching_compiler_specs
concretize.py::TestConcretize::test_simultaneous_concretization_of_specs
spec_dag.py::TestSpecDag::test_concretize_deptypes
spec_dag.py::TestSpecDag::test_copy_concretized
```

You can combine any list option with keywords:

```console
$ spack test --list -k microarchitecture
llnl/util/cpu.py  modules/lmod.py
```

```console
$ spack test --list-long -k microarchitecture
llnl/util/cpu.py::
    test_generic_microarchitecture

modules/lmod.py::TestLmod::
    test_only_generic_microarchitectures_in_root
```

Or just list specific files:

```console
$ spack test --list-long cmd/test.py
cmd/test.py::
    test_list                       test_list_names_with_pytest_arg
    test_list_long                  test_list_with_keywords
    test_list_long_with_pytest_arg  test_list_with_pytest_arg
    test_list_names
```

Hopefully this stuff will help with debugging test issues.

- [x] make `spack test` send args directly to `pytest` instead of trying
  to do fancy things.
- [x] rework `--list`, `--list-long`, and add `--list-names` to make
  searching for tests easier.
- [x] make it possible to mix Spack's list args with `pytest` args
  (they're just fancy parsing around `pytest --collect-only`)
- [x] add docs
- [x] add tests
- [x] update spack completion
2020-01-01 21:37:02 -08:00
Sinan
de73121ebd new package: py-numcodecs (#14330)
* new package: py-numcodecs

* make msgpack variant
2020-01-01 23:09:29 -06:00
Adam J. Stewart
7a97dc3770
py-numpy: add 1.17.5 (#14341) 2020-01-01 15:05:23 -06:00
Todd Gamblin
9192f046d2 tests: rename checks in github actions
I usually want to look at the Travis CI output, but I currently have to
scroll down to see it. This renames checks to be a bit shorter and more
consistent with Travis's naming, and also so that actions appear lower
than travis and codecov in the list of checks.
2019-12-31 17:59:59 -08:00
Glenn Johnson
1ac0c51dad Modify create clue list so R packages are detected (#12277)
R packages can contain configure scripts so R needs to be before
autotools in the clue list.
2019-12-31 16:44:31 -06:00
Todd Gamblin
ffc91bd86e tests: move mock config.yaml files to common directory
Test configuration files (except modules.yaml) were in the root level of
test/data, but should really just be in their own directory.  The absence
of modules.yaml was also breaking module tests if we got module
preferences after tests started, as the mock modules.yaml was not in the
test directory.
2019-12-31 13:48:01 -08:00
Todd Gamblin
3017584c48 config: remove all module-scope calls to spack.config.get()
This avoids parsing modules.yaml on startup.
2019-12-31 13:48:01 -08:00
Todd Gamblin
9cc013cc0f modules: make the module hook more robust
The module hook would previously fail if there were no enabled module types.

- Instead of looking for a `KeyError`, default to empty list when the
  config variable is not present.

- Convert lambdas to real functions for clarity.
2019-12-31 13:48:01 -08:00
Todd Gamblin
58cb4e5241 hooks: remove pre_run hook to improve startup time.
- Remove legacy yaml_version_check() hook
- Remove the pre_run hook from `hook/__init__.py` and `main.py`

We want to discourage the use of pre-run hooks because they have to run
at startup.  To keep Spack fast, we should do things like this lazily
instead of in hooks that require spidering directories full of modules.
2019-12-31 13:48:01 -08:00
Sinan
5ddbd2fa6c new package: py-msgpack (#14329)
* new package: py-msgpack

* remove python dependency

* remove cython dependency
2019-12-31 15:14:40 -06:00
Adam J. Stewart
5b979aee81
Ignore coverage drop due to lack of macOS tests (#14333)
* Ignore coverage drop due to lack of macOS tests

* Fix codecov tree structure
2019-12-31 15:13:12 -06:00
Jonathan R. Madsen
026534dadb Updated versions and more variants (#14310)
* Updated versions and more variants

- Added 'develop' and '3.0.0' versions
- Added 'tau', 'upcxx', 'gotcha', and 'likwid'

* Added conflict handling for +cupti~cuda

* Removed extra cmake args line
2019-12-31 13:33:41 -06:00
darmac
b70a5245f2 add new package : libfastcommon (#14303)
* add new package : libfastcommon

* update libfastcommon install()
2019-12-31 11:46:07 -06:00
darmac
e92f6c70a6 add new package : moosefs (#14305)
* add new package : moosefs

* remove no-need depens : zlib
2019-12-31 11:45:45 -06:00
darmac
9ff6079986 add new package : opentsdb (#14317)
* add new package : opentsdb

* remove redundant url_for_version()
2019-12-31 11:43:27 -06:00
Glenn Johnson
9a0fd8db9a New package - zziplib (#14318)
This PR adds the zziplib package.
2019-12-31 11:42:59 -06:00
Sinan
62cd7e4490 new package: py-asciitree (#14327)
* new package: py-asciitree

* fix typo
2019-12-31 11:39:43 -06:00
Mark Olesen
3bf7998bb5 Add OpenFOAM 1912 (#14322) 2019-12-31 11:19:05 -06:00
Todd Gamblin
4af6303086
copyright: update copyright dates for 2020 (#14328) 2019-12-30 22:36:56 -08:00
Todd Gamblin
98ad6e39b5
bugfix: add required fixture for CDash authentication test (#14325) 2019-12-30 16:55:05 -08:00
Zack Galbreath
cc96758fdc Add support for authenticated CDash uploads (#14200) 2019-12-30 15:54:56 -08:00
Todd Gamblin
65ef6d5dcb refactor: rename mock_config fixture to mock_low_high_config
This avoids confusion with mock_configuration.
2019-12-30 13:01:31 -08:00
Todd Gamblin
b2e9696052 argparse: lazily construct common arguments
Continuing to shave small bits of time off startup --
`spack.cmd.common.arguments` constructs many `Args` objects at module
scope, which has to be done for all commands that import it.  Instead of
doing this at load time, do it lazily.

- [x] construct Args objects lazily

- [x] remove the module-scoped argparse fixture

- [x] make the mock config scope set dirty to False by default (like the
  regular scope)

This *seems* to reduce load time slightly
2019-12-30 13:01:31 -08:00
Todd Gamblin
e7dc8a2bea tests: refactor tests to avoid persistent global state
Previously, fixtures like `config`, `database`, and `store` were
module-scoped, but frequently used as test function arguments.  These
fixtures swap out global on setup and restore them on teardown.  As
function arguments, they would do the right set-up, but they'd leave the
global changes in place for the whole module the function lived in.  This
meant that if you use `config` once, other functions in the same module
would inadvertently inherit the mock Spack configuration, as it would
only be torn down once all tests in the module were complete.

In general, we should module- or session-scope the *STATE* required for
these global objects (as it's expensive to create0, but we shouldn't
module-or session scope the activation/use of them, or things can get
really confusing.

- [x] Make generic context managers for global-modifying fixtures.

- [x] Make session- and module-scoped fixtures that ONLY build filesystem
  state and create objects, but do not swap out any variables.

- [x] Make seeparate function-scoped fixtures that *use* the session
  scoped fixtures and actually swap out (and back in) the global
  variables like `config`, `database`, and `store`.

These changes make it so that global changes are *only* ever alive for a
singlee test function, and we don't get weird dependencies because a
global fixture hasn't been destroyed.
2019-12-30 13:01:31 -08:00
Todd Gamblin
e839432472 tests: make env tests that use configs non-destructive
Environment tests pushed config scopes but didn't properly remove them.

- [x] use `with env:` context manager instead of `env.prepare_config_scopes()`
2019-12-30 13:01:31 -08:00
Todd Gamblin
8e8235043d package_prefs: move class-level cache to PackagePref instance
`PackagePrefs` has had a class-level cache of data from `packages.yaml` for
a long time, but it complicates testing and leads to subtle errors,
especially now that we frequently manipulate custom config scopes and
environments.

Moving the cache to instance-level doesn't slow down concretization or
the test suite, and it just caches for the life of a `PackagePrefs`
instance (i.e., for a single cocncretization) so we don't need to worry
about global state anymore.

- [x] Remove class-level caches from `PackagePrefs`
- [x] Add a cached _spec_order object on each `PackagePrefs` instance
- [x] Remove all calls to `PackagePrefs.clear_caches()`
2019-12-30 13:01:31 -08:00
Adam J. Stewart
0699f8ac9d
Add py-numpy 1.16.6 (#14313) 2019-12-29 20:04:04 -06:00
Glenn Johnson
25f217f5f2 Add support for graphite2 support to harfbuzz (#14300)
This PR adds graphite2 support to harfbuzz. This PR depends on PR #14299, which added the graphite2 package to spack.
2019-12-29 12:40:09 -06:00
Todd Gamblin
4d6462247e
externals: avoid importing jinja2 on startup (#14308)
Jinja2 costs a tenth to a few tenths of a second to import, so we should avoid importing it on startup.

- [x] only import jinja2 within functions
2019-12-28 14:43:23 -08:00
Todd Gamblin
2dafeaf819
bugfix: colify_table should not revert to 1 column for non-tty (#14307)
Commands like `spack blame` were printig poorly when redirected to files,
as colify reverts to a single column when redirected.  This works for
list data but not tables.

- [x] Force a table by always passing `tty=True` from `colify_table()`
2019-12-28 11:26:31 -08:00
darmac
855f9afa6e add new package : jafka (#14304) 2019-12-28 12:20:55 -06:00
Glenn Johnson
bd345e16b8 New package - TECkit (#14301)
This PR adds the teckit package to spack.
2019-12-28 12:20:20 -06:00
Glenn Johnson
12692424a7 New package - graphite2 (#14299)
* New package - graphite2

This PR adds the Graphite2 package to Spack.

* Replace smart quotes with standard quotes
2019-12-28 12:19:21 -06:00
Johannes Blaschke
c0d5c360d5 setup-env-test: fix pipe redirect (#14306) 2019-12-27 22:37:47 -08:00
Dr. Christian Tacke
8ee75e19bd Improve info variant header (#14275)
In "spack info" the Variants header currently has two blank
lines under it. That's too much. It looks like the actual
content belongs to something else.

Instead underline the headers to make things more obvious.
2019-12-27 15:21:15 -08:00
Glenn Johnson
2166a91ec5 New package - libpaper (#14302)
This PR adds the paper library to spack.
2019-12-27 15:18:16 -08:00
Gregory Lee
6c957b6b62 added build dependences required after cairo package change (#14297)
* added build dependences required after cairo package change

* fixed pango build dependencies
2019-12-27 12:13:50 -06:00
Gregory Lee
08898a2ad7 patch cairo build files for build without fontconfig fixes #14112 (#14296)
* patch cairo build files for build without fontconfig

* added build dependencies for cairo
2019-12-26 16:20:16 -06:00
asmaahassan90
3315700da9 Add Avizo Recipe (#12792)
* Add Avizo Recipe

* make changes as per review

* fix home url and linting

* Fix url

* fix identation

* change checksum to sha256 instead of md5

* fix installation

* fix lint

* fix identation

* make it compatible with python 2.6

* enhancing recipe and fixing avizo licensing

changes as per suggestions from reviewer; fix licensing

* fix identation

* use new setup_run_environment function
2019-12-26 14:43:51 -06:00
t-karatsu
c7f1e2835b aspa: Fix common errors. (#14295)
* aspa: Fix common errors.

* defined new member 'Vec_value_type' in 'self' namespace.

* fix narrowing error.

* fix patch about typecast.
2019-12-26 14:42:59 -06:00
Massimiliano Culpo
2aa8132afd Migrate build tests from Travis to Github Actions (#13967)
This PR moves build smoke tests from TravisCI and migrates them to Github Actions. The result is that build tests are performed in parallel with unit tests and they don't hog additional resources on Travis. The workflow will not run if a PR only changes packages in the built-in repository, but will always run on pushes to develop or master.

* Removed build tests from Travis and passed them to Github Actions
* Store ~/.ccache in Github Actions cache
* Add filters on paths and make sure this workflow don't run
* Use paths-ignore and exclude only files in the built-in repo
* Added a badge to README.md
2019-12-25 00:06:48 -08:00
Todd Gamblin
61b4ad1837
tests: finish removing pyqver from the repository (#14294)
Remove a few remaining mentions of the pyqver package, which was removed in #14289.
2019-12-24 17:37:03 -08:00
Massimiliano Culpo
d333e14721 tests: check min required python version with vermin (#14289)
This commit removes the `python_version.py` unit test module
and the vendored dependencies `pyqver2.py` and `pyqver3.py`.
It substitutes them with an equivalent check done using
`vermin` that is run as a separate workflow via Github Actions.

This allows us to delete 2 vendored dependencies that are unmaintained
and substitutes them with a maintained tool.

Also, updates the list of vendored dependencies.
2019-12-24 09:28:33 -08:00
t-karatsu
1e2c9d960c a64fx: fix typo in GCC flags (#14286) 2019-12-24 17:45:03 +01:00
Matthias Diener
6a084a8289 intel-pin: add version 3.11 (#14292) 2019-12-24 10:06:51 -06:00
Matthias Diener
37c1d2d004 cmake: add version 3.16.2 (#14291) 2019-12-24 10:05:54 -06:00
Matthias Diener
ee8bc0df98 scons: add version 3.1.2 (#14290) 2019-12-24 10:05:18 -06:00
健美猞猁
748c552992 Bump nektar to 5.0.0. (#14288) 2019-12-24 10:01:16 -06:00
Todd Gamblin
7652d1a4c1
Merge branch 'releases/v0.13' into develop 2019-12-24 01:04:41 -08:00
Todd Gamblin
55d5b435c8
update CHANGELOG.md for 0.13.3 2019-12-23 23:48:11 -08:00