Commit graph

3047 commits

Author SHA1 Message Date
Todd Gamblin
87aec4134d bugfix: preserve abstract specs when installing an environment
- `spack install` was setting the root to be the concrete spec
- abstract spec is now preserved
2018-11-11 18:32:24 -06:00
Todd Gamblin
0f2bfd7492 env: fix display of concretized specs in spack find
- fix highlighting of roots in concretized specs in `spack find`
- tighten up the `spack find` output in environments
2018-11-11 18:32:24 -06:00
Todd Gamblin
8d92fd6640 env: make spack config edit and spack config get environment-aware
- with no arguments, these commands will now edit or dump the
  environment's `spack.yaml` file.

- users may not know where named environments live

- this makes it convenient for users to get to the spack.yaml
  configuration file for their named environment.
2018-11-11 18:32:24 -06:00
Todd Gamblin
25f8abb963 bugfix: make defaults consistent --use-cache and --no-cache (#9803) 2018-11-11 17:18:21 -06:00
Mario Melara
7d98c73e40 Update buildsystem tut (#9795)
* Update Makefile to use property methods ("build_targets"/"install_targets")
  to demonstrate their usage
* Fix highlighting
* Change cbench example to ESMF:
  CBench package file was changed and no longer uses the example shown in
  the old docs
2018-11-11 14:21:54 -06:00
Adam J. Stewart
d366e642e4 Updates to Configuration Files docs for SC18 (#9801)
Scopes added with -C are now referred to as "custom scopes"
rather than "command line scopes". "command line scope" now refers
to specific config options that are set on the command line (like
"--insecure")
2018-11-11 10:57:46 -08:00
Todd Gamblin
6141ae49fd
install: add --use-cache back so that existing tooling does not break (#9797)
- default is still to use the cache, but we've added back the
  `--use-cache` argument so that scripts that used it are still correct.

- `--no-cache` is stil present and is mutually exclusive with `--use-cache`
2018-11-09 16:41:48 -08:00
Nichols A. Romero
236b34f3df Introduce fftw-api virtual package for Intel-MKL and FFTW (#9618)
* Introduce FFTW2 and FFT3 providers for Intel-MKL and FFTW Spack packages.

* make fftw default package for fftw-api virtual package

* virtual package test assertion now provides location of default virtual packages.

* Change name of virtual package to fftw-api and used versioned interface.
2018-11-09 08:16:11 -08:00
Greg Becker
988d37757f buildcache: update spack install to use build cache by default (#9772) 2018-11-09 00:59:28 -08:00
Todd Gamblin
423d3e75ab env: clean up command access to the active environment
- all commands (except `spack find`, through `ConstraintAction`) now go
  through get_env() to get the active environment

- ev.active was hard to read -- and the name wasn't descriptive.
  - rename it to _active_environment to be more descriptive and to strongly
    indicate that spack.environment manages it
2018-11-09 00:31:24 -08:00
Todd Gamblin
562482d9cc env: only add concrete attribute to abstract specs
- to aovid changing spec hashes drastically, only add this attribute to
  differentiated abstract specs.

- othherwise assume that read-in specs are concrete
2018-11-09 00:31:24 -08:00
Todd Gamblin
13164b114a env: make environment search more consistent; simplify code
- spack.yaml files in the current directory were picked up inconsistently
  -- make this a sure thing by moving that logic into find_environment()
  and moving find_environment() to main()

- simplify arguments to Spack command:
  - remove short args for infrequently used commands (--pdb/-D, -P, -s)
  - `spack -D` now forces an env with a directory
2018-11-09 00:31:24 -08:00
Todd Gamblin
3f68d8c53a externals: bugfix in ruamel for ordereddict in Python 2.6
- args weren't being delegated properly from CommentedMap to OrderedDict
2018-11-09 00:31:24 -08:00
Todd Gamblin
62f8ea1a75 env: rename EnvError to SpackEnvironmentError 2018-11-09 00:31:24 -08:00
Todd Gamblin
13aca774e3 bugfix: preserve patch ordering when specs are copied
- The `Spec` class maintains a special `_patches_in_order_of_appearance`
  attribute on patch variants, but it is was preserved when specs are
  copied.

- This caused issues for some builds

- Add special logic to `Spec` to preserve this variant on copy

- TODO: in the long term we should get rid of the special variant and
  make it the responsibility of one of the variant classes.
2018-11-09 00:31:24 -08:00
Todd Gamblin
a41bce2148 fix bad regular expressions and docstrings with '\' 2018-11-09 00:31:24 -08:00
Todd Gamblin
1137b183e3 commands: rework command categories and arguments
- split 'environment' section into 'environments' and 'modules'
- move location to 'query packages' section
- move cd to developer section

- --env-dir no longer has a short optino (was -E)
- -E now means "run without an environment" (no longer same as --env-dir)
- -D now means "run with this directory environment"
- remove short options for may infrequently used top-level commands
2018-11-09 00:31:24 -08:00
Todd Gamblin
efad7ac81b env: consolidate most of spack env status into spack find
- `spack env status` used to show install status; consolidate that into
  `spack find`.

- `spack env status` will still print out whether there is an active
  environment
2018-11-09 00:31:24 -08:00
Todd Gamblin
26a55ff749 env: move env uninstall into spack uninstall
- uninstall now:
  - restricts its spec search to the current environment
  - removes uninstalled specs from the current environment
  - reports envs that still need specs you're trying to uninstall

- removed spack env uninstall command
- updated tests
2018-11-09 00:31:24 -08:00
Todd Gamblin
7136274f4b env: move spack env stage into spack stage command 2018-11-09 00:31:24 -08:00
Todd Gamblin
e62506571f env: spack env destroy is now spack env remove 2018-11-09 00:31:24 -08:00
Todd Gamblin
8b549f664c env: move add, remove, and concretize to top-level commands 2018-11-09 00:31:24 -08:00
Todd Gamblin
e63b45b293 env: moved all spack env install functionality into spack install
- moved get_env from cmd/env.py to environment.py

- spack install will now install into the active environment when no
  arguments are provided.  It looks:
  1. at the command line
  2. for a local spack.yaml file
  3. for any currently activated environment
2018-11-09 00:31:24 -08:00
Todd Gamblin
08e4720ed9 env: remove all -e arguments on subcommands
- add and remove now require an active environment
- update tests to use with <ENV> instead of -e
2018-11-09 00:31:24 -08:00
Todd Gamblin
d483e6e17b env: currently activated environment cannot be destroyed 2018-11-09 00:31:24 -08:00
Todd Gamblin
40af955b94 env: prevent any active environments from interfering with tests
- ensure that `SPACK_ENV` is unset before tests
- ensure that `spack.environment.active` is deactivated if set
2018-11-09 00:31:24 -08:00
Todd Gamblin
c27b78ee36 env: remove upgrade() and relocate() for now
- these won't be in the first release of environments
- they'll be added back in later
2018-11-09 00:31:24 -08:00
Todd Gamblin
36623a27fd env: add test to ensure config precedence is high-to-low 2018-11-09 00:31:24 -08:00
Todd Gamblin
66aa3426ac env: make install_status output more concise 2018-11-09 00:31:24 -08:00
Todd Gamblin
a1818f971f env: environments can be named or created in directories
- `spack env create <name>` works as before

- `spack env create <path>` now works as well -- environments can be
  created in their own directories outside of Spack.

- `spack install` will look for a `spack.yaml` file in the current
  directory, and will install the entire project from the environment

- The Environment class has been refactored so that it does not depend on
  the internal Spack environment root; it just takes a path and operates
  on an environment in that path (so internal and external envs are
  handled the same)

- The named environment interface has been hoisted to the
  spack.environment module level.

- env.yaml is now spack.yaml in all places.  It was easier to go with one
  name for these files than to try to handle logic for both env.yaml and
  spack.yaml.
2018-11-09 00:31:24 -08:00
Todd Gamblin
9fb37dfd76 env: spack install SPEC installs into currently active environment.
- install will now add (if necessary), concretize, and install a single
  spec into the active environment.
2018-11-09 00:31:24 -08:00
Todd Gamblin
3fd9fc8994 env: spack env install automatically concretizes specs 2018-11-09 00:31:24 -08:00
Todd Gamblin
d14f7b82bb env: add spack env activate/deactivate and shell support
- `spack env activate foo`: sets SPACK_ENV to the current active env name

- `spack env deactivate`: unsets SPACK_ENV, deactivates the environment

- added support to setup_env.sh and setup_env.csh

- other env commands work properly with SPACK_ENV, as with an environment
  arguments.

- command-line --env arguments take precedence over the active
  environment, if given.
2018-11-09 00:31:24 -08:00
Todd Gamblin
15c5c36eaf env: bugfix: spack env list won't fail if var/spack/environments doesn't exist 2018-11-09 00:31:24 -08:00
Todd Gamblin
ce230fa3f4 env: rework environments
- env.yaml is now meaningful; it contains authoritative user specs

  - concretize diffs user specs in env.yaml and env.json to allow user to
    add/remove by simply updating env.yaml

  - comments are preserved when env.yaml is updated by add/unadd

  - env.yaml can contain configuration and include external configuration
    either from merged files or from config scopes

  - there is only one file format to remember (env.yaml, no separate init
    format)

- env.json is now env.lock, and it stores the *last* user specs to be
  concretized, along with full provenance.
  - internal structure was modified slightly for readability
  - env.lock contains a _meta section with metadata, in case needed

- added more tests for environments

- env commands follow Spack conventions; no more `spack env foo install`
2018-11-09 00:31:24 -08:00
Todd Gamblin
6af5dfbbc2 config: allow env.yaml to contain configuration in a single file
- add `SingleFileScope` to configuration, which allows us to pull config
  sections from a single file.

- update `env.yaml` and tests to ensure that the env.yaml schema works
  when pulling configurtion from the env file.
2018-11-09 00:31:24 -08:00
Todd Gamblin
9ee2623486 env: rename spack env list to spack env status 2018-11-09 00:31:24 -08:00
Todd Gamblin
c19000038b schemas: rework schemas so that they can be included from other files
- Each schema now has a top-level `properties` and `schema` attribute.

- The `properties` is a fragment that can be included in other
  jsonschemas, via Python, not via '$ref'

- Th `schema` is a complete `jsonschema` with `title` and `$schema`
  properties.
2018-11-09 00:31:24 -08:00
Todd Gamblin
83323f4e71 bugfix: Avoid KeyError in compilers.yaml version check 2018-11-09 00:31:24 -08:00
Todd Gamblin
84140c6cd3 env: add -e as global spack argument, make spack -e <env> spec work
- add -E/--exact-env instead of --use-env-repo
- simplify env handling in `spack find`
2018-11-09 00:31:24 -08:00
Todd Gamblin
180d804615 env: add --env argument to spack location 2018-11-09 00:31:24 -08:00
Todd Gamblin
47e60d5ef8 env: add --env argument to spack find
- add a common argument for `-e/--env`
- modify the database to support queries on subsets of hashes
- allow `spack find` to be filtered by hashes in an environment
2018-11-09 00:31:24 -08:00
Todd Gamblin
ea7648ff84 bugfix: identical specs with different DAG hashes don't shadow each other
- logic used in `spack find` was hiding duplicate installations if their
  hashes were different

- short hash doesn't work in this scenario, since specs are structurally
  identical

- ConstraintAction always works on a DB query, so use the DAG hash to
  ensure uniqueness
2018-11-09 00:31:24 -08:00
Todd Gamblin
3e94c4d573 env: move main Environment class and logic to spack.environment
- `spack.environment` is now the home for most of the infrastructure
   around Spack environments

- refactor `cmd/env.py` to use everything from spack.environment

- refactor the cmd/env test to use pytest and fixtures
2018-11-09 00:31:24 -08:00
Todd Gamblin
0e60fcccfb utils: merge spack.environment into spack.util.environment
- `spack.util.environment` is the new home for routines that modify
  environment variables.

- This is to make room for `spack.environment` to contain new routines
  for dealing with spack environments
2018-11-09 00:31:24 -08:00
Todd Gamblin
cd075b04d2 env: refactor imports in cmd/env.py and tests 2018-11-09 00:31:24 -08:00
Todd Gamblin
b738b7b05b env: refactor subparsers in env.py
- Instead of one method with all parsers, each subcommand gets two
  functions: `setup_<cmd>_parser()` and `environment_<cmd>()`

- the `setup_parser()` and `env()` functions now generate the parser
  based on these and a list of subcommands.

- it is now easier to associate the arguments with the subcommand.
2018-11-09 00:31:24 -08:00
Peter Josef Scheibel
31cb2041c3 env: add spack env command, along with env.yaml schema and tests
Co-authored-by: Elizabeth Fischer <rpf2116@columbia.edu>
2018-11-09 00:31:24 -08:00
Peter Josef Scheibel
4b2f51d063 env: preserve command_line as the scope of highest precedence
Co-authored-by: Elizabeth Fischer <rpf2116@columbia.edu>
2018-11-09 00:31:24 -08:00
Elizabeth Fischer
d1cce990cd env: refactor common arguments 2018-11-09 00:31:24 -08:00