Commit graph

10429 commits

Author SHA1 Message Date
Adam J. Stewart
d366e642e4 Updates to Configuration Files docs for SC18 (#9801)
Scopes added with -C are now referred to as "custom scopes"
rather than "command line scopes". "command line scope" now refers
to specific config options that are set on the command line (like
"--insecure")
2018-11-11 10:57:46 -08:00
Todd Gamblin
6141ae49fd
install: add --use-cache back so that existing tooling does not break (#9797)
- default is still to use the cache, but we've added back the
  `--use-cache` argument so that scripts that used it are still correct.

- `--no-cache` is stil present and is mutually exclusive with `--use-cache`
2018-11-09 16:41:48 -08:00
Jim Galarowicz
5dc0b65719 Update Dyninst package file for TBB dependency (#9728)
* Update Dyninst package file for TBB dependency

Dyninst master and 10.0 release
versions require the tbb package.  This mod
updates the tbb versions adding the one that
dyninst uses and adds the required spack package
changes to dyninst/package.py.

* Put new version of tbb in proper location, per reviewer comment.

* Only depend on tbb if building the development version.

* Changes for the release of Dyninst 10.0.0
2018-11-09 13:35:11 -08:00
Todd Gamblin
88a3393462
travis: allow Python 2.6 to fail until we figure out why Travis hangs (#9776) 2018-11-09 09:29:13 -08:00
Nichols A. Romero
236b34f3df Introduce fftw-api virtual package for Intel-MKL and FFTW (#9618)
* Introduce FFTW2 and FFT3 providers for Intel-MKL and FFTW Spack packages.

* make fftw default package for fftw-api virtual package

* virtual package test assertion now provides location of default virtual packages.

* Change name of virtual package to fftw-api and used versioned interface.
2018-11-09 08:16:11 -08:00
Greg Becker
988d37757f buildcache: update spack install to use build cache by default (#9772) 2018-11-09 00:59:28 -08:00
Todd Gamblin
423d3e75ab env: clean up command access to the active environment
- all commands (except `spack find`, through `ConstraintAction`) now go
  through get_env() to get the active environment

- ev.active was hard to read -- and the name wasn't descriptive.
  - rename it to _active_environment to be more descriptive and to strongly
    indicate that spack.environment manages it
2018-11-09 00:31:24 -08:00
Todd Gamblin
562482d9cc env: only add concrete attribute to abstract specs
- to aovid changing spec hashes drastically, only add this attribute to
  differentiated abstract specs.

- othherwise assume that read-in specs are concrete
2018-11-09 00:31:24 -08:00
Todd Gamblin
13164b114a env: make environment search more consistent; simplify code
- spack.yaml files in the current directory were picked up inconsistently
  -- make this a sure thing by moving that logic into find_environment()
  and moving find_environment() to main()

- simplify arguments to Spack command:
  - remove short args for infrequently used commands (--pdb/-D, -P, -s)
  - `spack -D` now forces an env with a directory
2018-11-09 00:31:24 -08:00
Todd Gamblin
3f68d8c53a externals: bugfix in ruamel for ordereddict in Python 2.6
- args weren't being delegated properly from CommentedMap to OrderedDict
2018-11-09 00:31:24 -08:00
Todd Gamblin
62f8ea1a75 env: rename EnvError to SpackEnvironmentError 2018-11-09 00:31:24 -08:00
Todd Gamblin
13aca774e3 bugfix: preserve patch ordering when specs are copied
- The `Spec` class maintains a special `_patches_in_order_of_appearance`
  attribute on patch variants, but it is was preserved when specs are
  copied.

- This caused issues for some builds

- Add special logic to `Spec` to preserve this variant on copy

- TODO: in the long term we should get rid of the special variant and
  make it the responsibility of one of the variant classes.
2018-11-09 00:31:24 -08:00
Todd Gamblin
a41bce2148 fix bad regular expressions and docstrings with '\' 2018-11-09 00:31:24 -08:00
Todd Gamblin
1137b183e3 commands: rework command categories and arguments
- split 'environment' section into 'environments' and 'modules'
- move location to 'query packages' section
- move cd to developer section

- --env-dir no longer has a short optino (was -E)
- -E now means "run without an environment" (no longer same as --env-dir)
- -D now means "run with this directory environment"
- remove short options for may infrequently used top-level commands
2018-11-09 00:31:24 -08:00
Todd Gamblin
efad7ac81b env: consolidate most of spack env status into spack find
- `spack env status` used to show install status; consolidate that into
  `spack find`.

- `spack env status` will still print out whether there is an active
  environment
2018-11-09 00:31:24 -08:00
Todd Gamblin
26a55ff749 env: move env uninstall into spack uninstall
- uninstall now:
  - restricts its spec search to the current environment
  - removes uninstalled specs from the current environment
  - reports envs that still need specs you're trying to uninstall

- removed spack env uninstall command
- updated tests
2018-11-09 00:31:24 -08:00
Todd Gamblin
7136274f4b env: move spack env stage into spack stage command 2018-11-09 00:31:24 -08:00
Todd Gamblin
e62506571f env: spack env destroy is now spack env remove 2018-11-09 00:31:24 -08:00
Todd Gamblin
8b549f664c env: move add, remove, and concretize to top-level commands 2018-11-09 00:31:24 -08:00
Todd Gamblin
e63b45b293 env: moved all spack env install functionality into spack install
- moved get_env from cmd/env.py to environment.py

- spack install will now install into the active environment when no
  arguments are provided.  It looks:
  1. at the command line
  2. for a local spack.yaml file
  3. for any currently activated environment
2018-11-09 00:31:24 -08:00
Todd Gamblin
08e4720ed9 env: remove all -e arguments on subcommands
- add and remove now require an active environment
- update tests to use with <ENV> instead of -e
2018-11-09 00:31:24 -08:00
Todd Gamblin
d483e6e17b env: currently activated environment cannot be destroyed 2018-11-09 00:31:24 -08:00
Todd Gamblin
40af955b94 env: prevent any active environments from interfering with tests
- ensure that `SPACK_ENV` is unset before tests
- ensure that `spack.environment.active` is deactivated if set
2018-11-09 00:31:24 -08:00
Todd Gamblin
c27b78ee36 env: remove upgrade() and relocate() for now
- these won't be in the first release of environments
- they'll be added back in later
2018-11-09 00:31:24 -08:00
Todd Gamblin
36623a27fd env: add test to ensure config precedence is high-to-low 2018-11-09 00:31:24 -08:00
Todd Gamblin
66aa3426ac env: make install_status output more concise 2018-11-09 00:31:24 -08:00
Todd Gamblin
a1818f971f env: environments can be named or created in directories
- `spack env create <name>` works as before

- `spack env create <path>` now works as well -- environments can be
  created in their own directories outside of Spack.

- `spack install` will look for a `spack.yaml` file in the current
  directory, and will install the entire project from the environment

- The Environment class has been refactored so that it does not depend on
  the internal Spack environment root; it just takes a path and operates
  on an environment in that path (so internal and external envs are
  handled the same)

- The named environment interface has been hoisted to the
  spack.environment module level.

- env.yaml is now spack.yaml in all places.  It was easier to go with one
  name for these files than to try to handle logic for both env.yaml and
  spack.yaml.
2018-11-09 00:31:24 -08:00
Todd Gamblin
9fb37dfd76 env: spack install SPEC installs into currently active environment.
- install will now add (if necessary), concretize, and install a single
  spec into the active environment.
2018-11-09 00:31:24 -08:00
Todd Gamblin
3fd9fc8994 env: spack env install automatically concretizes specs 2018-11-09 00:31:24 -08:00
Todd Gamblin
d14f7b82bb env: add spack env activate/deactivate and shell support
- `spack env activate foo`: sets SPACK_ENV to the current active env name

- `spack env deactivate`: unsets SPACK_ENV, deactivates the environment

- added support to setup_env.sh and setup_env.csh

- other env commands work properly with SPACK_ENV, as with an environment
  arguments.

- command-line --env arguments take precedence over the active
  environment, if given.
2018-11-09 00:31:24 -08:00
Todd Gamblin
15c5c36eaf env: bugfix: spack env list won't fail if var/spack/environments doesn't exist 2018-11-09 00:31:24 -08:00
Todd Gamblin
ce230fa3f4 env: rework environments
- env.yaml is now meaningful; it contains authoritative user specs

  - concretize diffs user specs in env.yaml and env.json to allow user to
    add/remove by simply updating env.yaml

  - comments are preserved when env.yaml is updated by add/unadd

  - env.yaml can contain configuration and include external configuration
    either from merged files or from config scopes

  - there is only one file format to remember (env.yaml, no separate init
    format)

- env.json is now env.lock, and it stores the *last* user specs to be
  concretized, along with full provenance.
  - internal structure was modified slightly for readability
  - env.lock contains a _meta section with metadata, in case needed

- added more tests for environments

- env commands follow Spack conventions; no more `spack env foo install`
2018-11-09 00:31:24 -08:00
Todd Gamblin
6af5dfbbc2 config: allow env.yaml to contain configuration in a single file
- add `SingleFileScope` to configuration, which allows us to pull config
  sections from a single file.

- update `env.yaml` and tests to ensure that the env.yaml schema works
  when pulling configurtion from the env file.
2018-11-09 00:31:24 -08:00
Todd Gamblin
9ee2623486 env: rename spack env list to spack env status 2018-11-09 00:31:24 -08:00
Todd Gamblin
c19000038b schemas: rework schemas so that they can be included from other files
- Each schema now has a top-level `properties` and `schema` attribute.

- The `properties` is a fragment that can be included in other
  jsonschemas, via Python, not via '$ref'

- Th `schema` is a complete `jsonschema` with `title` and `$schema`
  properties.
2018-11-09 00:31:24 -08:00
Todd Gamblin
83323f4e71 bugfix: Avoid KeyError in compilers.yaml version check 2018-11-09 00:31:24 -08:00
Todd Gamblin
84140c6cd3 env: add -e as global spack argument, make spack -e <env> spec work
- add -E/--exact-env instead of --use-env-repo
- simplify env handling in `spack find`
2018-11-09 00:31:24 -08:00
Todd Gamblin
180d804615 env: add --env argument to spack location 2018-11-09 00:31:24 -08:00
Todd Gamblin
47e60d5ef8 env: add --env argument to spack find
- add a common argument for `-e/--env`
- modify the database to support queries on subsets of hashes
- allow `spack find` to be filtered by hashes in an environment
2018-11-09 00:31:24 -08:00
Todd Gamblin
ea7648ff84 bugfix: identical specs with different DAG hashes don't shadow each other
- logic used in `spack find` was hiding duplicate installations if their
  hashes were different

- short hash doesn't work in this scenario, since specs are structurally
  identical

- ConstraintAction always works on a DB query, so use the DAG hash to
  ensure uniqueness
2018-11-09 00:31:24 -08:00
Todd Gamblin
3e94c4d573 env: move main Environment class and logic to spack.environment
- `spack.environment` is now the home for most of the infrastructure
   around Spack environments

- refactor `cmd/env.py` to use everything from spack.environment

- refactor the cmd/env test to use pytest and fixtures
2018-11-09 00:31:24 -08:00
Todd Gamblin
0e60fcccfb utils: merge spack.environment into spack.util.environment
- `spack.util.environment` is the new home for routines that modify
  environment variables.

- This is to make room for `spack.environment` to contain new routines
  for dealing with spack environments
2018-11-09 00:31:24 -08:00
Todd Gamblin
cd075b04d2 env: refactor imports in cmd/env.py and tests 2018-11-09 00:31:24 -08:00
Todd Gamblin
b738b7b05b env: refactor subparsers in env.py
- Instead of one method with all parsers, each subcommand gets two
  functions: `setup_<cmd>_parser()` and `environment_<cmd>()`

- the `setup_parser()` and `env()` functions now generate the parser
  based on these and a list of subcommands.

- it is now easier to associate the arguments with the subcommand.
2018-11-09 00:31:24 -08:00
Peter Josef Scheibel
31cb2041c3 env: add spack env command, along with env.yaml schema and tests
Co-authored-by: Elizabeth Fischer <rpf2116@columbia.edu>
2018-11-09 00:31:24 -08:00
Peter Josef Scheibel
4b2f51d063 env: preserve command_line as the scope of highest precedence
Co-authored-by: Elizabeth Fischer <rpf2116@columbia.edu>
2018-11-09 00:31:24 -08:00
Elizabeth Fischer
d1cce990cd env: refactor common arguments 2018-11-09 00:31:24 -08:00
Elizabeth Fischer
037457adc9 specs: save/restore concrete & patches when exporting/importing Specs 2018-11-09 00:31:24 -08:00
Peter Josef Scheibel
7f224b616d specs: _concrete should never be cleared when copying Specs 2018-11-09 00:31:24 -08:00
Peter Josef Scheibel
acdb391931 specs: add convenience function for reading spec dependencies from node dict 2018-11-09 00:31:24 -08:00