Commit graph

3088 commits

Author SHA1 Message Date
Denis Davydov
39b23d277b environment: ignore invalid files names in var/spack/environments (#10198) 2018-12-25 22:22:21 -08:00
Todd Gamblin
35dd929651
bugfix: handle unicode properly in spack.util.executable (#10186)
- When returning string output, use text_type and decode utf-8 in Python
  2 instead of using `str`

- This properly handles unicode, whereas before we would pass bad strings
  to colify in `spack blame` when reading git output

- add a test that round-trips some unicode through an Executable object
2018-12-23 23:59:58 -08:00
Elsa Gonsiorowski, PhD
74a5d92f07 Remove /nfs/tmp2 from default build_stage locations (#10170)
* Remove /nfs/tmp2 from default configuration
* /nfs/tmp2 is going away from LC... and doesn’t exist for the rest of the world.
* update documentation to remove /nfs/tmp2 as well
2018-12-21 02:03:54 -08:00
Zack Galbreath
b90f619037 Performance improvements for CDash reporter
* Record build output as an array of lines rather than concatenating to a
  single large string.
* Use string.find to avoid running re.search on every line of output.
2018-12-20 09:23:08 -08:00
Zack Galbreath
6355ee208c Include accurate build time in CDash report 2018-12-20 09:23:08 -08:00
Zack Galbreath
8d0872083c Report current git commit of Spack to CDash
When using the CDash reporter, upload a Update.xml file that
indicates the hash of Spack's current git commit.
2018-12-20 09:23:08 -08:00
Scott Wittenburg
7217b4a4b9 Make sure to urlencode cdash submit parameters 2018-12-20 09:23:08 -08:00
Zack Galbreath
aed9a532c6 Get buildId from CDash at submit time
Pass extra data when submitting to CDash.  If CDash responds with a buildId,
construct and display helpful URL that links to the uploaded report.
2018-12-20 09:23:08 -08:00
Zack Galbreath
738d2bd77a Allow more customization for CDash reporter
Add new command line arguments to `spack install` that allow users
to set the build name, site name, and track in their CDash report.
2018-12-20 09:23:08 -08:00
Zack Galbreath
0bae6626a4 Avoid UnboundLocalError
Make sure cdash_phase is defined before referring to it.
2018-12-20 09:23:08 -08:00
Todd Gamblin
207c37759c env: all commands that disambiguate specs should be env-aware
- some commands were missed in the rollout of spack environments

- this makes all commands that need to disambiguate specs restrict the
  disambiguation to installed packages in the active environment, as
  users would expect
2018-12-19 17:49:51 -08:00
Michael Kuhn
67ab73d381 fetch_strategy: improve GitFetchStrategy (#10153)
Currently, only the Git repsository's URL shows up in the `spack info`
output, which makes it hard to distinguish different versions:
```
Safe versions:
    develop    [git] https://github.com/LLNL/H5Z-ZFP.git
    0.8.0      [git] https://github.com/LLNL/H5Z-ZFP.git
    0.7.0      [git] https://github.com/LLNL/H5Z-ZFP.git
```

This change adds additional information as shown when cloning a Git
repository:
```
Safe versions:
    develop    [git] https://github.com/LLNL/H5Z-ZFP.git on branch master
    0.8.0      [git] https://github.com/LLNL/H5Z-ZFP.git at commit af165c4
    0.7.0      [git] https://github.com/LLNL/H5Z-ZFP.git at commit 58ac811
```
2018-12-19 14:33:58 -08:00
Adam J. Stewart
c1a99bf8ec
Add additional info to MesonPackage docs (#10133)
* Add additional info to MesonPackage docs

* No Pygments lexer for Meson code, default to none
2018-12-18 21:51:42 -06:00
Adam J. Stewart
2d0ddd5d11
Add help messages to spack gpg subcommands (#10117) 2018-12-17 13:02:51 -06:00
Adam J. Stewart
d5d64bdf3b
spack buildcache: --allow_root -> --allow-root (#10115) 2018-12-17 13:02:28 -06:00
Peter Scheibel
0217a651c8 mirrors: patches are now properly added to mirrors (#8993)
* This fixes a number of bugs:

  * Patches were not properly downloaded and added to mirrors.

  * Mirror create didn't respect `list_url` in packages

  * Update the `spack mirror` command to add all packages in the
    concretized DAG (where originally it only added the package specified
    by the user). This is required in order to collect patches that are specified
    by dependents. Example:
      * if X->Y and X requires a patch on Y called Pxy, then Pxy will only
        be discovered if you create a mirror with X.

  * replace confusing --one-version-per-spec option for `spack mirror create`
    with --versions-per-spec; support retrieving multiple versions for 
    concrete specs

* Implementation details:

  * `spack mirror create` now uses regular staging logic to download files 
    into a mirror, instead of reimplementing it in `add_single_spec`.

  * use a separate resource caching object to keep track of new
    resources and already-existing resources; also accepts storing
    resources retrieved from a cache (unlike the local cache)

  * mirror cache object now stores resources that are considered
    non-cachable, like (e.g. the tip of a branch);

  * the 'create' function of the mirror module no longer traverses
    dependencies since this was already handled by the 'mirror' command; 

  * Change handling of `--no-checksum`:

    * now that 'mirror create' uses stages, the mirror tests disable
      checksums when creating the mirror

    * remove `no_checksum` argument from library functions - this is now
      handled at the Spack-command-level (like for 'spack install')
2018-12-16 10:15:22 -08:00
Adam J. Stewart
1424830378 spack pkg add: add help message (#10116) 2018-12-15 11:44:37 +01:00
Greg Becker
d2d0ab06b7 Fix spack package inheritance for module variables (#10097)
* we weren't properly setting module variables for the root package in a DAG -- just for transitive dependencies.
2018-12-14 08:37:22 -08:00
Adam J. Stewart
ad046402b6 Make downloads from list_url deterministic (#10047) 2018-12-10 14:36:55 +01:00
Gregory Becker
b072c9b457 multimethod: slight refactoring, documentation for code review 2018-12-06 15:48:23 -08:00
Gregory Becker
2621af41d1 fix MRO for multimethod.__call__ using iterative algorithm.
Add tests MRO for inherited multimethods with multiple inheritance
Add tests for inherited and overridden multimethods
2018-12-06 15:48:23 -08:00
Todd Gamblin
43d94d4a30 tests: fully parametrize multimethod test
- all multimethod tests are now run for both `multimethod` and
  `multimethod-inheritor`

- do this with a parameterized fixture (pkg_name) that runs the same
  tests on both
2018-12-06 15:48:23 -08:00
Todd Gamblin
61b859193d multimethod uses Spec() instead of parse_anonymous_spec()
- simplify logic in multimethod
- remove the requirement of multimethod invocations to walk up the stack.
2018-12-06 15:48:23 -08:00
Todd Gamblin
88cb11758b spec: refactor and clean up Spec initialization
- Since early Spack versions, the SpecParser has (weirdly) been
  responsible for initializing Spec fields.

- This refactors initialization to take place in Spec.__init__, as it
  probably should have originally.

- This makes the code easier to read, the parser easier to understand,
  and removes the use of __new__ in the parser to initialize the Spec.

- This also makes it possible to make a completely empty Spec with
  `Spec()` -- this is an abstract Spec that will match anything.
2018-12-06 15:48:23 -08:00
Gregory Becker
7bb7b94666 Added test for multimethod inheritance 2018-12-06 15:48:23 -08:00
Peter Scheibel
34f64f1f3f improved constraint conflict error message (#9975) 2018-12-04 14:19:57 +01:00
Massimiliano Culpo
e998a399da Injected flags have a consistent order (#9960)
Fixes #9908

Spack was assembling flags in a manner that could in different
orderings for repeated concretizations of the same spec and config
2018-11-28 10:23:38 -08:00
Scott Wittenburg
33b112a60d Expose option to save all dependencies when writing spec.yaml (#9965) 2018-11-27 17:13:39 -08:00
Denis Davydov
abd7b185f7 intel-mkl: fix usage of openmp_libs and tbb_libs (#9863)
* intel-mkl: fix usage of openmp_libs and tbb_libs
* intel-mkl: remove trailing whitespace from openmp lib
2018-11-16 10:25:15 -08:00
Adam J. Stewart
b771b4015a Fix bugs in Modules Tutorial (#9821)
Found a couple bugs while trying to copy and paste the exact
tutorial content.
2018-11-13 09:31:09 -06:00
Adam J. Stewart
7bca5b9c61 Fix typos in Build Systems Tutorial (#9822) 2018-11-12 18:32:13 -06:00
Todd Gamblin
2efaf1b6f7 docs: update tutorial slide link for SC18 2018-11-12 09:44:47 -06:00
Todd Gamblin
d079a5c73e version bump: v0.12.0 2018-11-12 06:04:17 -06:00
Todd Gamblin
f92349ff8c docs: add environments tutorial
- tutorial goes through three sections:
  - installing and uninstalling environments
  - dealing with many specs
  - spack.yaml and spack.lock and workflows
2018-11-12 05:57:08 -06:00
Adam J. Stewart
a1676ab012 Updates to Configuration Tutorial for SC18 (#9811)
* Updates to Configuration Tutorial for SC18

* Suggested rewording
2018-11-11 22:50:59 -08:00
Adam J. Stewart
71561f9d2b Consistent capitalization of Build Caches docs for SC18 (#9810) 2018-11-11 21:13:01 -08:00
Adam J. Stewart
49c66db2fa Minor changes to Build Settings docs (#9808) 2018-11-11 21:12:37 -08:00
Adam J. Stewart
c227e1f02e Minor changes to Basic Settings docs for SC18 (#9809)
Grammar/spelling issues
2018-11-11 23:10:05 -06:00
Greg Becker
dfd0e96508 Update basic usage and modules tutorial (#9807)
* "spack install" now uses cache by default, update examples accordingly
* Replace some example packages with others
* Packing tutorial reference to "spack env" replaced with "spack build-env"
* Command line prompts in examples are shortened
* Example output (including paths) are updated to be more relevant to training environment
2018-11-11 22:58:50 -06:00
Peter Scheibel
2a5d5fda26
Advanced packaging tutorial: reorganize for binary caches (#9804)
Update all examples that need an MPI provider to build with MPICH; reorganize so that fixing MPICH (as part of environment section) comes first in the tutorial (most examples in the tutorial use an MPI provider).
2018-11-11 21:31:51 -06:00
Todd Gamblin
a8e8d80750 env: uninstall just removes specs that are still needed by other envs
- previously, uninstall would complain if a spec was needed by an
  environment.

- Now, we analyze dependents and dependent environments and simply remove
  (not uninstall) specs that are needed by environments
2018-11-11 18:32:24 -06:00
Todd Gamblin
87aec4134d bugfix: preserve abstract specs when installing an environment
- `spack install` was setting the root to be the concrete spec
- abstract spec is now preserved
2018-11-11 18:32:24 -06:00
Todd Gamblin
0f2bfd7492 env: fix display of concretized specs in spack find
- fix highlighting of roots in concretized specs in `spack find`
- tighten up the `spack find` output in environments
2018-11-11 18:32:24 -06:00
Todd Gamblin
8d92fd6640 env: make spack config edit and spack config get environment-aware
- with no arguments, these commands will now edit or dump the
  environment's `spack.yaml` file.

- users may not know where named environments live

- this makes it convenient for users to get to the spack.yaml
  configuration file for their named environment.
2018-11-11 18:32:24 -06:00
Todd Gamblin
25f8abb963 bugfix: make defaults consistent --use-cache and --no-cache (#9803) 2018-11-11 17:18:21 -06:00
Mario Melara
7d98c73e40 Update buildsystem tut (#9795)
* Update Makefile to use property methods ("build_targets"/"install_targets")
  to demonstrate their usage
* Fix highlighting
* Change cbench example to ESMF:
  CBench package file was changed and no longer uses the example shown in
  the old docs
2018-11-11 14:21:54 -06:00
Adam J. Stewart
d366e642e4 Updates to Configuration Files docs for SC18 (#9801)
Scopes added with -C are now referred to as "custom scopes"
rather than "command line scopes". "command line scope" now refers
to specific config options that are set on the command line (like
"--insecure")
2018-11-11 10:57:46 -08:00
Todd Gamblin
6141ae49fd
install: add --use-cache back so that existing tooling does not break (#9797)
- default is still to use the cache, but we've added back the
  `--use-cache` argument so that scripts that used it are still correct.

- `--no-cache` is stil present and is mutually exclusive with `--use-cache`
2018-11-09 16:41:48 -08:00
Nichols A. Romero
236b34f3df Introduce fftw-api virtual package for Intel-MKL and FFTW (#9618)
* Introduce FFTW2 and FFT3 providers for Intel-MKL and FFTW Spack packages.

* make fftw default package for fftw-api virtual package

* virtual package test assertion now provides location of default virtual packages.

* Change name of virtual package to fftw-api and used versioned interface.
2018-11-09 08:16:11 -08:00
Greg Becker
988d37757f buildcache: update spack install to use build cache by default (#9772) 2018-11-09 00:59:28 -08:00