Commit graph

14511 commits

Author SHA1 Message Date
Todd Gamblin
3017584c48 config: remove all module-scope calls to spack.config.get()
This avoids parsing modules.yaml on startup.
2019-12-31 13:48:01 -08:00
Todd Gamblin
9cc013cc0f modules: make the module hook more robust
The module hook would previously fail if there were no enabled module types.

- Instead of looking for a `KeyError`, default to empty list when the
  config variable is not present.

- Convert lambdas to real functions for clarity.
2019-12-31 13:48:01 -08:00
Todd Gamblin
58cb4e5241 hooks: remove pre_run hook to improve startup time.
- Remove legacy yaml_version_check() hook
- Remove the pre_run hook from `hook/__init__.py` and `main.py`

We want to discourage the use of pre-run hooks because they have to run
at startup.  To keep Spack fast, we should do things like this lazily
instead of in hooks that require spidering directories full of modules.
2019-12-31 13:48:01 -08:00
Sinan
5ddbd2fa6c new package: py-msgpack (#14329)
* new package: py-msgpack

* remove python dependency

* remove cython dependency
2019-12-31 15:14:40 -06:00
Adam J. Stewart
5b979aee81
Ignore coverage drop due to lack of macOS tests (#14333)
* Ignore coverage drop due to lack of macOS tests

* Fix codecov tree structure
2019-12-31 15:13:12 -06:00
Jonathan R. Madsen
026534dadb Updated versions and more variants (#14310)
* Updated versions and more variants

- Added 'develop' and '3.0.0' versions
- Added 'tau', 'upcxx', 'gotcha', and 'likwid'

* Added conflict handling for +cupti~cuda

* Removed extra cmake args line
2019-12-31 13:33:41 -06:00
darmac
b70a5245f2 add new package : libfastcommon (#14303)
* add new package : libfastcommon

* update libfastcommon install()
2019-12-31 11:46:07 -06:00
darmac
e92f6c70a6 add new package : moosefs (#14305)
* add new package : moosefs

* remove no-need depens : zlib
2019-12-31 11:45:45 -06:00
darmac
9ff6079986 add new package : opentsdb (#14317)
* add new package : opentsdb

* remove redundant url_for_version()
2019-12-31 11:43:27 -06:00
Glenn Johnson
9a0fd8db9a New package - zziplib (#14318)
This PR adds the zziplib package.
2019-12-31 11:42:59 -06:00
Sinan
62cd7e4490 new package: py-asciitree (#14327)
* new package: py-asciitree

* fix typo
2019-12-31 11:39:43 -06:00
Mark Olesen
3bf7998bb5 Add OpenFOAM 1912 (#14322) 2019-12-31 11:19:05 -06:00
Todd Gamblin
4af6303086
copyright: update copyright dates for 2020 (#14328) 2019-12-30 22:36:56 -08:00
Todd Gamblin
98ad6e39b5
bugfix: add required fixture for CDash authentication test (#14325) 2019-12-30 16:55:05 -08:00
Zack Galbreath
cc96758fdc Add support for authenticated CDash uploads (#14200) 2019-12-30 15:54:56 -08:00
Todd Gamblin
65ef6d5dcb refactor: rename mock_config fixture to mock_low_high_config
This avoids confusion with mock_configuration.
2019-12-30 13:01:31 -08:00
Todd Gamblin
b2e9696052 argparse: lazily construct common arguments
Continuing to shave small bits of time off startup --
`spack.cmd.common.arguments` constructs many `Args` objects at module
scope, which has to be done for all commands that import it.  Instead of
doing this at load time, do it lazily.

- [x] construct Args objects lazily

- [x] remove the module-scoped argparse fixture

- [x] make the mock config scope set dirty to False by default (like the
  regular scope)

This *seems* to reduce load time slightly
2019-12-30 13:01:31 -08:00
Todd Gamblin
e7dc8a2bea tests: refactor tests to avoid persistent global state
Previously, fixtures like `config`, `database`, and `store` were
module-scoped, but frequently used as test function arguments.  These
fixtures swap out global on setup and restore them on teardown.  As
function arguments, they would do the right set-up, but they'd leave the
global changes in place for the whole module the function lived in.  This
meant that if you use `config` once, other functions in the same module
would inadvertently inherit the mock Spack configuration, as it would
only be torn down once all tests in the module were complete.

In general, we should module- or session-scope the *STATE* required for
these global objects (as it's expensive to create0, but we shouldn't
module-or session scope the activation/use of them, or things can get
really confusing.

- [x] Make generic context managers for global-modifying fixtures.

- [x] Make session- and module-scoped fixtures that ONLY build filesystem
  state and create objects, but do not swap out any variables.

- [x] Make seeparate function-scoped fixtures that *use* the session
  scoped fixtures and actually swap out (and back in) the global
  variables like `config`, `database`, and `store`.

These changes make it so that global changes are *only* ever alive for a
singlee test function, and we don't get weird dependencies because a
global fixture hasn't been destroyed.
2019-12-30 13:01:31 -08:00
Todd Gamblin
e839432472 tests: make env tests that use configs non-destructive
Environment tests pushed config scopes but didn't properly remove them.

- [x] use `with env:` context manager instead of `env.prepare_config_scopes()`
2019-12-30 13:01:31 -08:00
Todd Gamblin
8e8235043d package_prefs: move class-level cache to PackagePref instance
`PackagePrefs` has had a class-level cache of data from `packages.yaml` for
a long time, but it complicates testing and leads to subtle errors,
especially now that we frequently manipulate custom config scopes and
environments.

Moving the cache to instance-level doesn't slow down concretization or
the test suite, and it just caches for the life of a `PackagePrefs`
instance (i.e., for a single cocncretization) so we don't need to worry
about global state anymore.

- [x] Remove class-level caches from `PackagePrefs`
- [x] Add a cached _spec_order object on each `PackagePrefs` instance
- [x] Remove all calls to `PackagePrefs.clear_caches()`
2019-12-30 13:01:31 -08:00
Adam J. Stewart
0699f8ac9d
Add py-numpy 1.16.6 (#14313) 2019-12-29 20:04:04 -06:00
Glenn Johnson
25f217f5f2 Add support for graphite2 support to harfbuzz (#14300)
This PR adds graphite2 support to harfbuzz. This PR depends on PR #14299, which added the graphite2 package to spack.
2019-12-29 12:40:09 -06:00
Todd Gamblin
4d6462247e
externals: avoid importing jinja2 on startup (#14308)
Jinja2 costs a tenth to a few tenths of a second to import, so we should avoid importing it on startup.

- [x] only import jinja2 within functions
2019-12-28 14:43:23 -08:00
Todd Gamblin
2dafeaf819
bugfix: colify_table should not revert to 1 column for non-tty (#14307)
Commands like `spack blame` were printig poorly when redirected to files,
as colify reverts to a single column when redirected.  This works for
list data but not tables.

- [x] Force a table by always passing `tty=True` from `colify_table()`
2019-12-28 11:26:31 -08:00
darmac
855f9afa6e add new package : jafka (#14304) 2019-12-28 12:20:55 -06:00
Glenn Johnson
bd345e16b8 New package - TECkit (#14301)
This PR adds the teckit package to spack.
2019-12-28 12:20:20 -06:00
Glenn Johnson
12692424a7 New package - graphite2 (#14299)
* New package - graphite2

This PR adds the Graphite2 package to Spack.

* Replace smart quotes with standard quotes
2019-12-28 12:19:21 -06:00
Johannes Blaschke
c0d5c360d5 setup-env-test: fix pipe redirect (#14306) 2019-12-27 22:37:47 -08:00
Dr. Christian Tacke
8ee75e19bd Improve info variant header (#14275)
In "spack info" the Variants header currently has two blank
lines under it. That's too much. It looks like the actual
content belongs to something else.

Instead underline the headers to make things more obvious.
2019-12-27 15:21:15 -08:00
Glenn Johnson
2166a91ec5 New package - libpaper (#14302)
This PR adds the paper library to spack.
2019-12-27 15:18:16 -08:00
Gregory Lee
6c957b6b62 added build dependences required after cairo package change (#14297)
* added build dependences required after cairo package change

* fixed pango build dependencies
2019-12-27 12:13:50 -06:00
Gregory Lee
08898a2ad7 patch cairo build files for build without fontconfig fixes #14112 (#14296)
* patch cairo build files for build without fontconfig

* added build dependencies for cairo
2019-12-26 16:20:16 -06:00
asmaahassan90
3315700da9 Add Avizo Recipe (#12792)
* Add Avizo Recipe

* make changes as per review

* fix home url and linting

* Fix url

* fix identation

* change checksum to sha256 instead of md5

* fix installation

* fix lint

* fix identation

* make it compatible with python 2.6

* enhancing recipe and fixing avizo licensing

changes as per suggestions from reviewer; fix licensing

* fix identation

* use new setup_run_environment function
2019-12-26 14:43:51 -06:00
t-karatsu
c7f1e2835b aspa: Fix common errors. (#14295)
* aspa: Fix common errors.

* defined new member 'Vec_value_type' in 'self' namespace.

* fix narrowing error.

* fix patch about typecast.
2019-12-26 14:42:59 -06:00
Massimiliano Culpo
2aa8132afd Migrate build tests from Travis to Github Actions (#13967)
This PR moves build smoke tests from TravisCI and migrates them to Github Actions. The result is that build tests are performed in parallel with unit tests and they don't hog additional resources on Travis. The workflow will not run if a PR only changes packages in the built-in repository, but will always run on pushes to develop or master.

* Removed build tests from Travis and passed them to Github Actions
* Store ~/.ccache in Github Actions cache
* Add filters on paths and make sure this workflow don't run
* Use paths-ignore and exclude only files in the built-in repo
* Added a badge to README.md
2019-12-25 00:06:48 -08:00
Todd Gamblin
61b4ad1837
tests: finish removing pyqver from the repository (#14294)
Remove a few remaining mentions of the pyqver package, which was removed in #14289.
2019-12-24 17:37:03 -08:00
Massimiliano Culpo
d333e14721 tests: check min required python version with vermin (#14289)
This commit removes the `python_version.py` unit test module
and the vendored dependencies `pyqver2.py` and `pyqver3.py`.
It substitutes them with an equivalent check done using
`vermin` that is run as a separate workflow via Github Actions.

This allows us to delete 2 vendored dependencies that are unmaintained
and substitutes them with a maintained tool.

Also, updates the list of vendored dependencies.
2019-12-24 09:28:33 -08:00
t-karatsu
1e2c9d960c a64fx: fix typo in GCC flags (#14286) 2019-12-24 17:45:03 +01:00
Matthias Diener
6a084a8289 intel-pin: add version 3.11 (#14292) 2019-12-24 10:06:51 -06:00
Matthias Diener
37c1d2d004 cmake: add version 3.16.2 (#14291) 2019-12-24 10:05:54 -06:00
Matthias Diener
ee8bc0df98 scons: add version 3.1.2 (#14290) 2019-12-24 10:05:18 -06:00
健美猞猁
748c552992 Bump nektar to 5.0.0. (#14288) 2019-12-24 10:01:16 -06:00
Todd Gamblin
7652d1a4c1
Merge branch 'releases/v0.13' into develop 2019-12-24 01:04:41 -08:00
Todd Gamblin
55d5b435c8
update CHANGELOG.md for 0.13.3 2019-12-23 23:48:11 -08:00
Todd Gamblin
231e237764
version bump: 0.13.3 2019-12-23 23:48:11 -08:00
Massimiliano Culpo
8616a26406
Travis exits at the first failing test, pin codecov at v4.5.4 (#14179)
Before this commit we used to run the entire unit test suite
in the presence of a failure. Since we currently rely a lot
on the state of the filesystem etc. the end report was most
of the time showing spurious failures that were a consequence
of the first failing test.

This PR makes unit tests exit at the first failing test

Also, pin codecov at v4.5.4 (last one supporting Python 2.6)
2019-12-23 23:48:03 -08:00
Todd Gamblin
e22d3250dd
performance: dont' read spec.yaml files twice in view regeneration
`ViewDescriptor.regenerate()` calls `get_all_specs()`, which reads
`spec.yaml` files, which is slow.  It's fine to do this once, but
`view.remove_specs()` *also* calls it immediately afterwards.

- [x] Pass the result of `get_all_specs()` as an optional parameter to
  `view.remove_specs()` to avoid reading `spec.yaml` files twice.
2019-12-23 23:18:47 -08:00
Todd Gamblin
e3939b0c72
performance: don't recompute hashes when regenerating environments
`ViewDescriptor.regenerate()` was copying specs and stripping build
dependencies, which clears `_hash` and other cached fields on concrete
specs, which causes a bunch of YAML hashes to be recomputed.

- [x] Preserve the `_hash` and `_normal` fields on stripped specs, as
  these will be unchanged.
2019-12-23 23:18:46 -08:00
Todd Gamblin
f013687397
performance: reduce system calls required for remove_dead_links
`os.path.exists()` will report False if the target of a symlink doesn't
exist, so we can avoid a costly call to realpath here.
2019-12-23 23:18:46 -08:00
Todd Gamblin
79ddf6cf0d
performance: only regenerate env views once in spack install
`spack install` previously concretized, writes the entire environment
out, regenerated views, then wrote and regenerated views
again. Regenerating views is slow, so ensure that we only do that once.

- [x] add an option to env.write() to skip view regeneration

- [x] add a note on whether regenerate_views() shouldn't just be a
  separate operation -- not clear if we want to keep it as part of write
  to ensure consistency, or take it out to avoid performance issues.
2019-12-23 23:18:45 -08:00