Commit graph

10303 commits

Author SHA1 Message Date
Todd Gamblin
a1818f971f env: environments can be named or created in directories
- `spack env create <name>` works as before

- `spack env create <path>` now works as well -- environments can be
  created in their own directories outside of Spack.

- `spack install` will look for a `spack.yaml` file in the current
  directory, and will install the entire project from the environment

- The Environment class has been refactored so that it does not depend on
  the internal Spack environment root; it just takes a path and operates
  on an environment in that path (so internal and external envs are
  handled the same)

- The named environment interface has been hoisted to the
  spack.environment module level.

- env.yaml is now spack.yaml in all places.  It was easier to go with one
  name for these files than to try to handle logic for both env.yaml and
  spack.yaml.
2018-11-09 00:31:24 -08:00
Todd Gamblin
9fb37dfd76 env: spack install SPEC installs into currently active environment.
- install will now add (if necessary), concretize, and install a single
  spec into the active environment.
2018-11-09 00:31:24 -08:00
Todd Gamblin
3fd9fc8994 env: spack env install automatically concretizes specs 2018-11-09 00:31:24 -08:00
Todd Gamblin
d14f7b82bb env: add spack env activate/deactivate and shell support
- `spack env activate foo`: sets SPACK_ENV to the current active env name

- `spack env deactivate`: unsets SPACK_ENV, deactivates the environment

- added support to setup_env.sh and setup_env.csh

- other env commands work properly with SPACK_ENV, as with an environment
  arguments.

- command-line --env arguments take precedence over the active
  environment, if given.
2018-11-09 00:31:24 -08:00
Todd Gamblin
15c5c36eaf env: bugfix: spack env list won't fail if var/spack/environments doesn't exist 2018-11-09 00:31:24 -08:00
Todd Gamblin
ce230fa3f4 env: rework environments
- env.yaml is now meaningful; it contains authoritative user specs

  - concretize diffs user specs in env.yaml and env.json to allow user to
    add/remove by simply updating env.yaml

  - comments are preserved when env.yaml is updated by add/unadd

  - env.yaml can contain configuration and include external configuration
    either from merged files or from config scopes

  - there is only one file format to remember (env.yaml, no separate init
    format)

- env.json is now env.lock, and it stores the *last* user specs to be
  concretized, along with full provenance.
  - internal structure was modified slightly for readability
  - env.lock contains a _meta section with metadata, in case needed

- added more tests for environments

- env commands follow Spack conventions; no more `spack env foo install`
2018-11-09 00:31:24 -08:00
Todd Gamblin
6af5dfbbc2 config: allow env.yaml to contain configuration in a single file
- add `SingleFileScope` to configuration, which allows us to pull config
  sections from a single file.

- update `env.yaml` and tests to ensure that the env.yaml schema works
  when pulling configurtion from the env file.
2018-11-09 00:31:24 -08:00
Todd Gamblin
9ee2623486 env: rename spack env list to spack env status 2018-11-09 00:31:24 -08:00
Todd Gamblin
c19000038b schemas: rework schemas so that they can be included from other files
- Each schema now has a top-level `properties` and `schema` attribute.

- The `properties` is a fragment that can be included in other
  jsonschemas, via Python, not via '$ref'

- Th `schema` is a complete `jsonschema` with `title` and `$schema`
  properties.
2018-11-09 00:31:24 -08:00
Todd Gamblin
83323f4e71 bugfix: Avoid KeyError in compilers.yaml version check 2018-11-09 00:31:24 -08:00
Todd Gamblin
84140c6cd3 env: add -e as global spack argument, make spack -e <env> spec work
- add -E/--exact-env instead of --use-env-repo
- simplify env handling in `spack find`
2018-11-09 00:31:24 -08:00
Todd Gamblin
180d804615 env: add --env argument to spack location 2018-11-09 00:31:24 -08:00
Todd Gamblin
47e60d5ef8 env: add --env argument to spack find
- add a common argument for `-e/--env`
- modify the database to support queries on subsets of hashes
- allow `spack find` to be filtered by hashes in an environment
2018-11-09 00:31:24 -08:00
Todd Gamblin
ea7648ff84 bugfix: identical specs with different DAG hashes don't shadow each other
- logic used in `spack find` was hiding duplicate installations if their
  hashes were different

- short hash doesn't work in this scenario, since specs are structurally
  identical

- ConstraintAction always works on a DB query, so use the DAG hash to
  ensure uniqueness
2018-11-09 00:31:24 -08:00
Todd Gamblin
3e94c4d573 env: move main Environment class and logic to spack.environment
- `spack.environment` is now the home for most of the infrastructure
   around Spack environments

- refactor `cmd/env.py` to use everything from spack.environment

- refactor the cmd/env test to use pytest and fixtures
2018-11-09 00:31:24 -08:00
Todd Gamblin
0e60fcccfb utils: merge spack.environment into spack.util.environment
- `spack.util.environment` is the new home for routines that modify
  environment variables.

- This is to make room for `spack.environment` to contain new routines
  for dealing with spack environments
2018-11-09 00:31:24 -08:00
Todd Gamblin
cd075b04d2 env: refactor imports in cmd/env.py and tests 2018-11-09 00:31:24 -08:00
Todd Gamblin
b738b7b05b env: refactor subparsers in env.py
- Instead of one method with all parsers, each subcommand gets two
  functions: `setup_<cmd>_parser()` and `environment_<cmd>()`

- the `setup_parser()` and `env()` functions now generate the parser
  based on these and a list of subcommands.

- it is now easier to associate the arguments with the subcommand.
2018-11-09 00:31:24 -08:00
Peter Josef Scheibel
31cb2041c3 env: add spack env command, along with env.yaml schema and tests
Co-authored-by: Elizabeth Fischer <rpf2116@columbia.edu>
2018-11-09 00:31:24 -08:00
Peter Josef Scheibel
4b2f51d063 env: preserve command_line as the scope of highest precedence
Co-authored-by: Elizabeth Fischer <rpf2116@columbia.edu>
2018-11-09 00:31:24 -08:00
Elizabeth Fischer
d1cce990cd env: refactor common arguments 2018-11-09 00:31:24 -08:00
Elizabeth Fischer
037457adc9 specs: save/restore concrete & patches when exporting/importing Specs 2018-11-09 00:31:24 -08:00
Peter Josef Scheibel
7f224b616d specs: _concrete should never be cleared when copying Specs 2018-11-09 00:31:24 -08:00
Peter Josef Scheibel
acdb391931 specs: add convenience function for reading spec dependencies from node dict 2018-11-09 00:31:24 -08:00
Peter Josef Scheibel
4daa164fbf specs: allow writing full spec (including build deps) to dict 2018-11-09 00:31:24 -08:00
Peter Josef Scheibel
e6c6ab64b8 env: rename 'spack env' command to 'spack build-env' 2018-11-09 00:31:24 -08:00
Peter Josef Scheibel
bb8da72f0c env: renamed 'environment' test 2018-11-09 00:31:24 -08:00
Justin Stanley
d1b17e8207 canu: use built-in makefile install (#9726) 2018-11-08 10:28:10 +01:00
Denis Davydov
8f4d6caf32 muparser: minor fixes to 2.2.5 and add 2.2.6 with CMake (#9724)
* muparser: minor fixes to 2.2.5 and add 2.2.6 with CMake
* dealii: make muparser optional
2018-11-08 10:05:52 +01:00
Michael Kuhn
da193a07a3 ncl: Add 6.5.0 (#9736) 2018-11-08 10:03:29 +01:00
Levi Baber
55b6105af7
r-delayedarray: new dep in latest vers (#9731) 2018-11-07 10:27:08 -06:00
Valentin Clement (バレンタイン クレメン)
768111c365 Use the raw compiler path for the preprocessing pass in CLAW driver (#9748) 2018-11-07 15:24:24 +01:00
Ricardo Silva
eed5bdad68 sra-toolkit: improvements and new version (#9465)
* files under `bin` were copied 3 times instead of symlink'ed
* sha256 checksums
* new release 2.9.2 of July 24, 2018
2018-11-07 10:52:43 +01:00
Peter Scheibel
7d97e8b067 New repo for advanced packaging tutorial (#9711)
* modified tutorial packages

* update hint in hdf5 tutorial file (typo for suggested argument)

* add repo.yaml to tutorial repository

* update tutorial docs to refer user to tutorial package repository

* flake edits

* recommend site scope vs. defaults

* you don't specify the repo's name when adding a repo, just the path
2018-11-06 19:57:32 -08:00
Peter Scheibel
058cf81312 Binary caching: remove symlinks, copy files instead (#9747)
* omit symlinks and create file copies when making a binary cache of a package

* unrelated flake edits involving regexes that recent flake is now angry about
2018-11-06 19:56:03 -08:00
Zack Galbreath
30eda8b837 cdash: report clean results to CDash server (#9564)
* Record stdout for packages without errors

  Previously our reporter only stored stdout if something went wrong
  while installing a package.  This prevented us from properly reporting
  on steps where everything went as expected.

* More robustly report all phases to CDash

  Previously if a phase generated no output it would not be reported to CDash.
  For example, consider the following output:

    ==> Executing phase: 'configure'
    ==> Executing phase: 'build'

  This would not generate a report for the configure phase. Now it does.

* Add test case for CDash reporting clean builds

* Fix default directory for CDash reports

  The default 'cdash_report' directory name was getting overwritten
  by 'junit-report'.

* Upload the build phase first to CDash

  Older versions of CDash expect Build.xml to be the first file uploaded
  for any given build.

* Define cdash_phase before referring to it
2018-11-06 16:09:47 -08:00
Massimiliano Culpo
a474d83d58 bugfix: work around ruamel.yaml vendoring issues (#9725)
- Delete references to ruamel.yaml at Spack start-up, if they are present

- ruamel.yaml generates a .pth file when installed via pip that has the
  effect of always preferring the version of this package installed at
  site scope (effectively preventing us from vendoring it).

- This mechanism triggers when implicitly importing the 'site' module
  when the python interpreter is started. In this PR we explicitly delete
  references to 'ruamel.yaml' and 'ruamel' in sys.modules, if any, after
  we set 'sys.path' to search from the directory where we store vendored
  packages. This ensures that the imports after those statements will be
  done from our vendored version.

- See #9206 for further details
2018-11-06 16:06:18 -08:00
Massimiliano Culpo
05779d911f Adapted the code of the non-daemonic pool to recent python versions
fixes #9739

The non-daemonic pool relies heavily on implementation details of the
multiprocessing package. In this commit we provide an implementation
that fits recent python versions.
2018-11-06 16:02:37 -08:00
Levi Baber
4ba3c81bc8
snpeff: add wrapper for SnpSift.jar (#9674) 2018-11-06 08:36:47 -06:00
sknigh
d5327d8d4b Add OpenMPI 3.1.3 and a bug patch (#9680) 2018-11-06 09:12:48 +01:00
Gregory Lee
e6eb2b960e py-backports-functools-lru-cache: add to 'backports' namespace (#9720) 2018-11-05 16:01:03 -08:00
Justin Stanley
d7c9128875 new package: homer (starting at 4.9.1) (#9669) 2018-11-02 18:18:05 -07:00
Eric
b471a3626e freetype: enable-freetype-config build option for 2.9.1 (#9653)
This builds the 'freetype-config' binary which can be used to get
configuration information about the freetype install, used by some
dependents.
2018-11-02 18:06:40 -07:00
Tim Law
a33d123e2e Add version 2.0.2 of BookLeaf_Cpp and version 3.0.2 of Typhon. (#9662) 2018-11-02 13:30:52 -07:00
Nick Forrington
9f5865a68d compilers: add arm compiler detection to Spack
- added arm.py with support for detecting `armclang` and `armflang`

Co-authored-by: Srinath Vadlamani <srinath.vadlamani@arm.com>
2018-11-02 12:53:50 -07:00
Dmitriy
36ddcc2e97 henson: new package (#9657)
* henson: new package

* henson: change github path to henson-insitu

* henson: make mpi-wrappers=off by default

* henson: remove unsued variable and spaces to make linter happy

* henson: rename version master to develop
2018-11-01 18:15:12 +01:00
Thomas Stitt
6a3552274f xeus: new package (#9682)
* xeus: new package
cppzmq: add version 4.3.0
zeromq: make libsodium optional, on by default

* xeus: add patch so it builds, add new version
nlohmann-json: add more versions
cryptopp: add more versions

* xeus: flake8

* xeus: fix license
2018-10-31 20:23:44 -07:00
Tim Fuller
fc25ba1b22 Parse the ${NAMESPACE} format string in a spec's format method. (#9686)
This allows installing software on a namespace basis by including ${NAMESPACE} in `install_path_scheme`. e.g.,

```
cat ~/.spack/config.yaml
config:
  install_path_scheme:
      "${ARCHITECTURE}/${NAMESPACE}/${COMPILERNAME}-${COMPILERVER}/${PACKAGE}-${VERSION}-${HASH}"
```
2018-10-31 20:07:07 -07:00
Satish Balay
fb849a4b7f static_to_shared_library: separate options from option values (#9690)
The 'static_to_shared_library' function takes a compiler Executable,
which is intended to be invoked with a list of arguments; the
arguments must be separated from their values in the list, given
the way that 'Executable.__call__' invokes the underlying executable.
'static_to_shared_library' was not doing this, which this commit fixes.
2018-10-31 12:49:04 -07:00
Abhinav Bhatele
939ce40032 ecp-proxy-apps: disable building CANDLE Benchmarks by default (#9691)
add as a variant
2018-10-31 12:50:37 -06:00