Commit graph

3031 commits

Author SHA1 Message Date
Todd Gamblin
1137b183e3 commands: rework command categories and arguments
- split 'environment' section into 'environments' and 'modules'
- move location to 'query packages' section
- move cd to developer section

- --env-dir no longer has a short optino (was -E)
- -E now means "run without an environment" (no longer same as --env-dir)
- -D now means "run with this directory environment"
- remove short options for may infrequently used top-level commands
2018-11-09 00:31:24 -08:00
Todd Gamblin
efad7ac81b env: consolidate most of spack env status into spack find
- `spack env status` used to show install status; consolidate that into
  `spack find`.

- `spack env status` will still print out whether there is an active
  environment
2018-11-09 00:31:24 -08:00
Todd Gamblin
26a55ff749 env: move env uninstall into spack uninstall
- uninstall now:
  - restricts its spec search to the current environment
  - removes uninstalled specs from the current environment
  - reports envs that still need specs you're trying to uninstall

- removed spack env uninstall command
- updated tests
2018-11-09 00:31:24 -08:00
Todd Gamblin
7136274f4b env: move spack env stage into spack stage command 2018-11-09 00:31:24 -08:00
Todd Gamblin
e62506571f env: spack env destroy is now spack env remove 2018-11-09 00:31:24 -08:00
Todd Gamblin
8b549f664c env: move add, remove, and concretize to top-level commands 2018-11-09 00:31:24 -08:00
Todd Gamblin
e63b45b293 env: moved all spack env install functionality into spack install
- moved get_env from cmd/env.py to environment.py

- spack install will now install into the active environment when no
  arguments are provided.  It looks:
  1. at the command line
  2. for a local spack.yaml file
  3. for any currently activated environment
2018-11-09 00:31:24 -08:00
Todd Gamblin
08e4720ed9 env: remove all -e arguments on subcommands
- add and remove now require an active environment
- update tests to use with <ENV> instead of -e
2018-11-09 00:31:24 -08:00
Todd Gamblin
d483e6e17b env: currently activated environment cannot be destroyed 2018-11-09 00:31:24 -08:00
Todd Gamblin
40af955b94 env: prevent any active environments from interfering with tests
- ensure that `SPACK_ENV` is unset before tests
- ensure that `spack.environment.active` is deactivated if set
2018-11-09 00:31:24 -08:00
Todd Gamblin
c27b78ee36 env: remove upgrade() and relocate() for now
- these won't be in the first release of environments
- they'll be added back in later
2018-11-09 00:31:24 -08:00
Todd Gamblin
36623a27fd env: add test to ensure config precedence is high-to-low 2018-11-09 00:31:24 -08:00
Todd Gamblin
66aa3426ac env: make install_status output more concise 2018-11-09 00:31:24 -08:00
Todd Gamblin
a1818f971f env: environments can be named or created in directories
- `spack env create <name>` works as before

- `spack env create <path>` now works as well -- environments can be
  created in their own directories outside of Spack.

- `spack install` will look for a `spack.yaml` file in the current
  directory, and will install the entire project from the environment

- The Environment class has been refactored so that it does not depend on
  the internal Spack environment root; it just takes a path and operates
  on an environment in that path (so internal and external envs are
  handled the same)

- The named environment interface has been hoisted to the
  spack.environment module level.

- env.yaml is now spack.yaml in all places.  It was easier to go with one
  name for these files than to try to handle logic for both env.yaml and
  spack.yaml.
2018-11-09 00:31:24 -08:00
Todd Gamblin
9fb37dfd76 env: spack install SPEC installs into currently active environment.
- install will now add (if necessary), concretize, and install a single
  spec into the active environment.
2018-11-09 00:31:24 -08:00
Todd Gamblin
3fd9fc8994 env: spack env install automatically concretizes specs 2018-11-09 00:31:24 -08:00
Todd Gamblin
d14f7b82bb env: add spack env activate/deactivate and shell support
- `spack env activate foo`: sets SPACK_ENV to the current active env name

- `spack env deactivate`: unsets SPACK_ENV, deactivates the environment

- added support to setup_env.sh and setup_env.csh

- other env commands work properly with SPACK_ENV, as with an environment
  arguments.

- command-line --env arguments take precedence over the active
  environment, if given.
2018-11-09 00:31:24 -08:00
Todd Gamblin
15c5c36eaf env: bugfix: spack env list won't fail if var/spack/environments doesn't exist 2018-11-09 00:31:24 -08:00
Todd Gamblin
ce230fa3f4 env: rework environments
- env.yaml is now meaningful; it contains authoritative user specs

  - concretize diffs user specs in env.yaml and env.json to allow user to
    add/remove by simply updating env.yaml

  - comments are preserved when env.yaml is updated by add/unadd

  - env.yaml can contain configuration and include external configuration
    either from merged files or from config scopes

  - there is only one file format to remember (env.yaml, no separate init
    format)

- env.json is now env.lock, and it stores the *last* user specs to be
  concretized, along with full provenance.
  - internal structure was modified slightly for readability
  - env.lock contains a _meta section with metadata, in case needed

- added more tests for environments

- env commands follow Spack conventions; no more `spack env foo install`
2018-11-09 00:31:24 -08:00
Todd Gamblin
6af5dfbbc2 config: allow env.yaml to contain configuration in a single file
- add `SingleFileScope` to configuration, which allows us to pull config
  sections from a single file.

- update `env.yaml` and tests to ensure that the env.yaml schema works
  when pulling configurtion from the env file.
2018-11-09 00:31:24 -08:00
Todd Gamblin
9ee2623486 env: rename spack env list to spack env status 2018-11-09 00:31:24 -08:00
Todd Gamblin
c19000038b schemas: rework schemas so that they can be included from other files
- Each schema now has a top-level `properties` and `schema` attribute.

- The `properties` is a fragment that can be included in other
  jsonschemas, via Python, not via '$ref'

- Th `schema` is a complete `jsonschema` with `title` and `$schema`
  properties.
2018-11-09 00:31:24 -08:00
Todd Gamblin
83323f4e71 bugfix: Avoid KeyError in compilers.yaml version check 2018-11-09 00:31:24 -08:00
Todd Gamblin
84140c6cd3 env: add -e as global spack argument, make spack -e <env> spec work
- add -E/--exact-env instead of --use-env-repo
- simplify env handling in `spack find`
2018-11-09 00:31:24 -08:00
Todd Gamblin
180d804615 env: add --env argument to spack location 2018-11-09 00:31:24 -08:00
Todd Gamblin
47e60d5ef8 env: add --env argument to spack find
- add a common argument for `-e/--env`
- modify the database to support queries on subsets of hashes
- allow `spack find` to be filtered by hashes in an environment
2018-11-09 00:31:24 -08:00
Todd Gamblin
ea7648ff84 bugfix: identical specs with different DAG hashes don't shadow each other
- logic used in `spack find` was hiding duplicate installations if their
  hashes were different

- short hash doesn't work in this scenario, since specs are structurally
  identical

- ConstraintAction always works on a DB query, so use the DAG hash to
  ensure uniqueness
2018-11-09 00:31:24 -08:00
Todd Gamblin
3e94c4d573 env: move main Environment class and logic to spack.environment
- `spack.environment` is now the home for most of the infrastructure
   around Spack environments

- refactor `cmd/env.py` to use everything from spack.environment

- refactor the cmd/env test to use pytest and fixtures
2018-11-09 00:31:24 -08:00
Todd Gamblin
0e60fcccfb utils: merge spack.environment into spack.util.environment
- `spack.util.environment` is the new home for routines that modify
  environment variables.

- This is to make room for `spack.environment` to contain new routines
  for dealing with spack environments
2018-11-09 00:31:24 -08:00
Todd Gamblin
cd075b04d2 env: refactor imports in cmd/env.py and tests 2018-11-09 00:31:24 -08:00
Todd Gamblin
b738b7b05b env: refactor subparsers in env.py
- Instead of one method with all parsers, each subcommand gets two
  functions: `setup_<cmd>_parser()` and `environment_<cmd>()`

- the `setup_parser()` and `env()` functions now generate the parser
  based on these and a list of subcommands.

- it is now easier to associate the arguments with the subcommand.
2018-11-09 00:31:24 -08:00
Peter Josef Scheibel
31cb2041c3 env: add spack env command, along with env.yaml schema and tests
Co-authored-by: Elizabeth Fischer <rpf2116@columbia.edu>
2018-11-09 00:31:24 -08:00
Peter Josef Scheibel
4b2f51d063 env: preserve command_line as the scope of highest precedence
Co-authored-by: Elizabeth Fischer <rpf2116@columbia.edu>
2018-11-09 00:31:24 -08:00
Elizabeth Fischer
d1cce990cd env: refactor common arguments 2018-11-09 00:31:24 -08:00
Elizabeth Fischer
037457adc9 specs: save/restore concrete & patches when exporting/importing Specs 2018-11-09 00:31:24 -08:00
Peter Josef Scheibel
7f224b616d specs: _concrete should never be cleared when copying Specs 2018-11-09 00:31:24 -08:00
Peter Josef Scheibel
acdb391931 specs: add convenience function for reading spec dependencies from node dict 2018-11-09 00:31:24 -08:00
Peter Josef Scheibel
4daa164fbf specs: allow writing full spec (including build deps) to dict 2018-11-09 00:31:24 -08:00
Peter Josef Scheibel
e6c6ab64b8 env: rename 'spack env' command to 'spack build-env' 2018-11-09 00:31:24 -08:00
Peter Josef Scheibel
bb8da72f0c env: renamed 'environment' test 2018-11-09 00:31:24 -08:00
Peter Scheibel
7d97e8b067 New repo for advanced packaging tutorial (#9711)
* modified tutorial packages

* update hint in hdf5 tutorial file (typo for suggested argument)

* add repo.yaml to tutorial repository

* update tutorial docs to refer user to tutorial package repository

* flake edits

* recommend site scope vs. defaults

* you don't specify the repo's name when adding a repo, just the path
2018-11-06 19:57:32 -08:00
Peter Scheibel
058cf81312 Binary caching: remove symlinks, copy files instead (#9747)
* omit symlinks and create file copies when making a binary cache of a package

* unrelated flake edits involving regexes that recent flake is now angry about
2018-11-06 19:56:03 -08:00
Zack Galbreath
30eda8b837 cdash: report clean results to CDash server (#9564)
* Record stdout for packages without errors

  Previously our reporter only stored stdout if something went wrong
  while installing a package.  This prevented us from properly reporting
  on steps where everything went as expected.

* More robustly report all phases to CDash

  Previously if a phase generated no output it would not be reported to CDash.
  For example, consider the following output:

    ==> Executing phase: 'configure'
    ==> Executing phase: 'build'

  This would not generate a report for the configure phase. Now it does.

* Add test case for CDash reporting clean builds

* Fix default directory for CDash reports

  The default 'cdash_report' directory name was getting overwritten
  by 'junit-report'.

* Upload the build phase first to CDash

  Older versions of CDash expect Build.xml to be the first file uploaded
  for any given build.

* Define cdash_phase before referring to it
2018-11-06 16:09:47 -08:00
Massimiliano Culpo
05779d911f Adapted the code of the non-daemonic pool to recent python versions
fixes #9739

The non-daemonic pool relies heavily on implementation details of the
multiprocessing package. In this commit we provide an implementation
that fits recent python versions.
2018-11-06 16:02:37 -08:00
Nick Forrington
9f5865a68d compilers: add arm compiler detection to Spack
- added arm.py with support for detecting `armclang` and `armflang`

Co-authored-by: Srinath Vadlamani <srinath.vadlamani@arm.com>
2018-11-02 12:53:50 -07:00
Tim Fuller
fc25ba1b22 Parse the ${NAMESPACE} format string in a spec's format method. (#9686)
This allows installing software on a namespace basis by including ${NAMESPACE} in `install_path_scheme`. e.g.,

```
cat ~/.spack/config.yaml
config:
  install_path_scheme:
      "${ARCHITECTURE}/${NAMESPACE}/${COMPILERNAME}-${COMPILERVER}/${PACKAGE}-${VERSION}-${HASH}"
```
2018-10-31 20:07:07 -07:00
Satish Balay
fb849a4b7f static_to_shared_library: separate options from option values (#9690)
The 'static_to_shared_library' function takes a compiler Executable,
which is intended to be invoked with a list of arguments; the
arguments must be separated from their values in the list, given
the way that 'Executable.__call__' invokes the underlying executable.
'static_to_shared_library' was not doing this, which this commit fixes.
2018-10-31 12:49:04 -07:00
Todd Gamblin
2912cf3e17
compilers: update clang fortran compiler wrapper selection (#9678)
Clang has support for using different fortran compilers with the Clang executable.

Spack includes logic to select a compiler wrapper symlink which refers to the fortran executable (since some build systems depend on the name of the compiler, e.g. 'gfortran' or 'flang').

This selection was previously based on the architecture, and chose incorrectly in some situations (e.g. for clang/gfortran on Linux). This replaces architecture-based wrapper selection with a selection that is based on the name of the Fortran compiler executable.
2018-10-30 23:00:43 -07:00
Levi Baber
72fa122101 tests: add test for FIXME boilerplate to package_sanity (#9285)
* package_sanity: add test_no_fixme
* cleanup & better assert message
2018-10-30 13:44:25 -07:00
Todd Gamblin
b27fbfb379
config: spack config blame now colors filenames in config output (#9656)
- it was hard to distinguish all-gray filenames
- added rotating colors to `spack config blame`
2018-10-26 22:19:11 -07:00