Commit graph

10367 commits

Author SHA1 Message Date
Todd Gamblin
a41bce2148 fix bad regular expressions and docstrings with '\' 2018-11-09 00:31:24 -08:00
Todd Gamblin
1137b183e3 commands: rework command categories and arguments
- split 'environment' section into 'environments' and 'modules'
- move location to 'query packages' section
- move cd to developer section

- --env-dir no longer has a short optino (was -E)
- -E now means "run without an environment" (no longer same as --env-dir)
- -D now means "run with this directory environment"
- remove short options for may infrequently used top-level commands
2018-11-09 00:31:24 -08:00
Todd Gamblin
efad7ac81b env: consolidate most of spack env status into spack find
- `spack env status` used to show install status; consolidate that into
  `spack find`.

- `spack env status` will still print out whether there is an active
  environment
2018-11-09 00:31:24 -08:00
Todd Gamblin
26a55ff749 env: move env uninstall into spack uninstall
- uninstall now:
  - restricts its spec search to the current environment
  - removes uninstalled specs from the current environment
  - reports envs that still need specs you're trying to uninstall

- removed spack env uninstall command
- updated tests
2018-11-09 00:31:24 -08:00
Todd Gamblin
7136274f4b env: move spack env stage into spack stage command 2018-11-09 00:31:24 -08:00
Todd Gamblin
e62506571f env: spack env destroy is now spack env remove 2018-11-09 00:31:24 -08:00
Todd Gamblin
8b549f664c env: move add, remove, and concretize to top-level commands 2018-11-09 00:31:24 -08:00
Todd Gamblin
e63b45b293 env: moved all spack env install functionality into spack install
- moved get_env from cmd/env.py to environment.py

- spack install will now install into the active environment when no
  arguments are provided.  It looks:
  1. at the command line
  2. for a local spack.yaml file
  3. for any currently activated environment
2018-11-09 00:31:24 -08:00
Todd Gamblin
08e4720ed9 env: remove all -e arguments on subcommands
- add and remove now require an active environment
- update tests to use with <ENV> instead of -e
2018-11-09 00:31:24 -08:00
Todd Gamblin
d483e6e17b env: currently activated environment cannot be destroyed 2018-11-09 00:31:24 -08:00
Todd Gamblin
40af955b94 env: prevent any active environments from interfering with tests
- ensure that `SPACK_ENV` is unset before tests
- ensure that `spack.environment.active` is deactivated if set
2018-11-09 00:31:24 -08:00
Todd Gamblin
c27b78ee36 env: remove upgrade() and relocate() for now
- these won't be in the first release of environments
- they'll be added back in later
2018-11-09 00:31:24 -08:00
Todd Gamblin
36623a27fd env: add test to ensure config precedence is high-to-low 2018-11-09 00:31:24 -08:00
Todd Gamblin
66aa3426ac env: make install_status output more concise 2018-11-09 00:31:24 -08:00
Todd Gamblin
a1818f971f env: environments can be named or created in directories
- `spack env create <name>` works as before

- `spack env create <path>` now works as well -- environments can be
  created in their own directories outside of Spack.

- `spack install` will look for a `spack.yaml` file in the current
  directory, and will install the entire project from the environment

- The Environment class has been refactored so that it does not depend on
  the internal Spack environment root; it just takes a path and operates
  on an environment in that path (so internal and external envs are
  handled the same)

- The named environment interface has been hoisted to the
  spack.environment module level.

- env.yaml is now spack.yaml in all places.  It was easier to go with one
  name for these files than to try to handle logic for both env.yaml and
  spack.yaml.
2018-11-09 00:31:24 -08:00
Todd Gamblin
9fb37dfd76 env: spack install SPEC installs into currently active environment.
- install will now add (if necessary), concretize, and install a single
  spec into the active environment.
2018-11-09 00:31:24 -08:00
Todd Gamblin
3fd9fc8994 env: spack env install automatically concretizes specs 2018-11-09 00:31:24 -08:00
Todd Gamblin
d14f7b82bb env: add spack env activate/deactivate and shell support
- `spack env activate foo`: sets SPACK_ENV to the current active env name

- `spack env deactivate`: unsets SPACK_ENV, deactivates the environment

- added support to setup_env.sh and setup_env.csh

- other env commands work properly with SPACK_ENV, as with an environment
  arguments.

- command-line --env arguments take precedence over the active
  environment, if given.
2018-11-09 00:31:24 -08:00
Todd Gamblin
15c5c36eaf env: bugfix: spack env list won't fail if var/spack/environments doesn't exist 2018-11-09 00:31:24 -08:00
Todd Gamblin
ce230fa3f4 env: rework environments
- env.yaml is now meaningful; it contains authoritative user specs

  - concretize diffs user specs in env.yaml and env.json to allow user to
    add/remove by simply updating env.yaml

  - comments are preserved when env.yaml is updated by add/unadd

  - env.yaml can contain configuration and include external configuration
    either from merged files or from config scopes

  - there is only one file format to remember (env.yaml, no separate init
    format)

- env.json is now env.lock, and it stores the *last* user specs to be
  concretized, along with full provenance.
  - internal structure was modified slightly for readability
  - env.lock contains a _meta section with metadata, in case needed

- added more tests for environments

- env commands follow Spack conventions; no more `spack env foo install`
2018-11-09 00:31:24 -08:00
Todd Gamblin
6af5dfbbc2 config: allow env.yaml to contain configuration in a single file
- add `SingleFileScope` to configuration, which allows us to pull config
  sections from a single file.

- update `env.yaml` and tests to ensure that the env.yaml schema works
  when pulling configurtion from the env file.
2018-11-09 00:31:24 -08:00
Todd Gamblin
9ee2623486 env: rename spack env list to spack env status 2018-11-09 00:31:24 -08:00
Todd Gamblin
c19000038b schemas: rework schemas so that they can be included from other files
- Each schema now has a top-level `properties` and `schema` attribute.

- The `properties` is a fragment that can be included in other
  jsonschemas, via Python, not via '$ref'

- Th `schema` is a complete `jsonschema` with `title` and `$schema`
  properties.
2018-11-09 00:31:24 -08:00
Todd Gamblin
83323f4e71 bugfix: Avoid KeyError in compilers.yaml version check 2018-11-09 00:31:24 -08:00
Todd Gamblin
84140c6cd3 env: add -e as global spack argument, make spack -e <env> spec work
- add -E/--exact-env instead of --use-env-repo
- simplify env handling in `spack find`
2018-11-09 00:31:24 -08:00
Todd Gamblin
180d804615 env: add --env argument to spack location 2018-11-09 00:31:24 -08:00
Todd Gamblin
47e60d5ef8 env: add --env argument to spack find
- add a common argument for `-e/--env`
- modify the database to support queries on subsets of hashes
- allow `spack find` to be filtered by hashes in an environment
2018-11-09 00:31:24 -08:00
Todd Gamblin
ea7648ff84 bugfix: identical specs with different DAG hashes don't shadow each other
- logic used in `spack find` was hiding duplicate installations if their
  hashes were different

- short hash doesn't work in this scenario, since specs are structurally
  identical

- ConstraintAction always works on a DB query, so use the DAG hash to
  ensure uniqueness
2018-11-09 00:31:24 -08:00
Todd Gamblin
3e94c4d573 env: move main Environment class and logic to spack.environment
- `spack.environment` is now the home for most of the infrastructure
   around Spack environments

- refactor `cmd/env.py` to use everything from spack.environment

- refactor the cmd/env test to use pytest and fixtures
2018-11-09 00:31:24 -08:00
Todd Gamblin
0e60fcccfb utils: merge spack.environment into spack.util.environment
- `spack.util.environment` is the new home for routines that modify
  environment variables.

- This is to make room for `spack.environment` to contain new routines
  for dealing with spack environments
2018-11-09 00:31:24 -08:00
Todd Gamblin
cd075b04d2 env: refactor imports in cmd/env.py and tests 2018-11-09 00:31:24 -08:00
Todd Gamblin
b738b7b05b env: refactor subparsers in env.py
- Instead of one method with all parsers, each subcommand gets two
  functions: `setup_<cmd>_parser()` and `environment_<cmd>()`

- the `setup_parser()` and `env()` functions now generate the parser
  based on these and a list of subcommands.

- it is now easier to associate the arguments with the subcommand.
2018-11-09 00:31:24 -08:00
Peter Josef Scheibel
31cb2041c3 env: add spack env command, along with env.yaml schema and tests
Co-authored-by: Elizabeth Fischer <rpf2116@columbia.edu>
2018-11-09 00:31:24 -08:00
Peter Josef Scheibel
4b2f51d063 env: preserve command_line as the scope of highest precedence
Co-authored-by: Elizabeth Fischer <rpf2116@columbia.edu>
2018-11-09 00:31:24 -08:00
Elizabeth Fischer
d1cce990cd env: refactor common arguments 2018-11-09 00:31:24 -08:00
Elizabeth Fischer
037457adc9 specs: save/restore concrete & patches when exporting/importing Specs 2018-11-09 00:31:24 -08:00
Peter Josef Scheibel
7f224b616d specs: _concrete should never be cleared when copying Specs 2018-11-09 00:31:24 -08:00
Peter Josef Scheibel
acdb391931 specs: add convenience function for reading spec dependencies from node dict 2018-11-09 00:31:24 -08:00
Peter Josef Scheibel
4daa164fbf specs: allow writing full spec (including build deps) to dict 2018-11-09 00:31:24 -08:00
Peter Josef Scheibel
e6c6ab64b8 env: rename 'spack env' command to 'spack build-env' 2018-11-09 00:31:24 -08:00
Peter Josef Scheibel
bb8da72f0c env: renamed 'environment' test 2018-11-09 00:31:24 -08:00
Justin Stanley
d1b17e8207 canu: use built-in makefile install (#9726) 2018-11-08 10:28:10 +01:00
Denis Davydov
8f4d6caf32 muparser: minor fixes to 2.2.5 and add 2.2.6 with CMake (#9724)
* muparser: minor fixes to 2.2.5 and add 2.2.6 with CMake
* dealii: make muparser optional
2018-11-08 10:05:52 +01:00
Michael Kuhn
da193a07a3 ncl: Add 6.5.0 (#9736) 2018-11-08 10:03:29 +01:00
Levi Baber
55b6105af7
r-delayedarray: new dep in latest vers (#9731) 2018-11-07 10:27:08 -06:00
Valentin Clement (バレンタイン クレメン)
768111c365 Use the raw compiler path for the preprocessing pass in CLAW driver (#9748) 2018-11-07 15:24:24 +01:00
Ricardo Silva
eed5bdad68 sra-toolkit: improvements and new version (#9465)
* files under `bin` were copied 3 times instead of symlink'ed
* sha256 checksums
* new release 2.9.2 of July 24, 2018
2018-11-07 10:52:43 +01:00
Peter Scheibel
7d97e8b067 New repo for advanced packaging tutorial (#9711)
* modified tutorial packages

* update hint in hdf5 tutorial file (typo for suggested argument)

* add repo.yaml to tutorial repository

* update tutorial docs to refer user to tutorial package repository

* flake edits

* recommend site scope vs. defaults

* you don't specify the repo's name when adding a repo, just the path
2018-11-06 19:57:32 -08:00
Peter Scheibel
058cf81312 Binary caching: remove symlinks, copy files instead (#9747)
* omit symlinks and create file copies when making a binary cache of a package

* unrelated flake edits involving regexes that recent flake is now angry about
2018-11-06 19:56:03 -08:00
Zack Galbreath
30eda8b837 cdash: report clean results to CDash server (#9564)
* Record stdout for packages without errors

  Previously our reporter only stored stdout if something went wrong
  while installing a package.  This prevented us from properly reporting
  on steps where everything went as expected.

* More robustly report all phases to CDash

  Previously if a phase generated no output it would not be reported to CDash.
  For example, consider the following output:

    ==> Executing phase: 'configure'
    ==> Executing phase: 'build'

  This would not generate a report for the configure phase. Now it does.

* Add test case for CDash reporting clean builds

* Fix default directory for CDash reports

  The default 'cdash_report' directory name was getting overwritten
  by 'junit-report'.

* Upload the build phase first to CDash

  Older versions of CDash expect Build.xml to be the first file uploaded
  for any given build.

* Define cdash_phase before referring to it
2018-11-06 16:09:47 -08:00