Commit graph

2911 commits

Author SHA1 Message Date
Stephen Herbein
de60e9d582 lmod: fix use of custom separator in prepend_path etc. (#8737)
fixes #8736
2018-08-01 12:58:54 +02:00
Stephen Herbein
cdbc545793 docs: update spack view argument synatx (#8839)
The `--dependencies` and `--exclude` flags are now arguments to the `spack view`
comand rather than the `spack view symlink` command.
2018-07-31 10:41:41 -07:00
Todd Gamblin
2c45c3c5b3
spec: Spec.tree() merges deptypes when only covering nodes (#8821)
- previously, output could be confusing when deptypes were only shown for
  one dependent when a node had *multiple* dependents

- also fix default coverage of `Spec.tree()`: it previously defaulted to
  cover only build and link dependencies, but this is a holdover from
  when those were the only types.
2018-07-29 11:54:20 -07:00
Massimiliano Culpo
1fd9574353 Fixed a typo in the docs (the docker image to pull was wrong) (#8814) 2018-07-27 09:56:01 -05:00
Adam J. Stewart
7902f3eb49 Add docs on new top-level VCS attributes 2018-07-25 23:10:10 -07:00
Todd Gamblin
5f00056490 tests: convert mirror test to pytest with free functions. 2018-07-25 23:10:10 -07:00
Todd Gamblin
020c34e136 tests: add checks and tests for consistent version() arguments
- Previously, Spack didn't check the arguments you put in version()
  directives.

- So, you could do something like this, where there are arguments for a
  URL fetcher AND for a git fetcher:

  version('1.0', md5='abc123', git='https://foo.bar', commit='feda2343')

- Now, we check the arguments before constructing a fetcher, to ensure
  that each package has *only* arguments for a single type of fetcher.

- Also added `test_package_version_consistency()` to the `package_sanity`
  test, so that all builtin packages are required to have valid
  `version()` directives.
2018-07-25 23:10:10 -07:00
Todd Gamblin
6f7eaecfa0 tests: correct name of check_db to check_repo
- this test is checking the package *repository*, not the database.
2018-07-25 23:10:10 -07:00
Todd Gamblin
773cfe088f core: differentiate package-level fetch URLs by args to version()
- packagers can specify two top-level fetch URLs if one is `url`
  - e.g., `url` and `git` or `url` and `svn`

- allow only one VCS fetcher so we can differentiate between URL and VCS.

- also clean up fetcher logic and class structure
2018-07-25 23:10:10 -07:00
Todd Gamblin
04aec9d6f8 core: add check for conflicting top-level fetch attributes in packages
- ensure that packages can't have more than one of git, hg, svn, or url
2018-07-25 23:10:10 -07:00
Todd Gamblin
6837da2ad7 style: clean up exception definitions in package.py 2018-07-25 23:10:10 -07:00
Todd Gamblin
498d8cf04b core: fixes and tests for handling of fetcher attributes in packages
- Packages can remove the top-level `url` attribute and still work

- These are now legal:
  - Packages with *only* version-specific URLs (even with gaps)

  - Packages with a top-level git/hg/svn attribute and `version`
    directives for that.

- If a package has both a top-level hg/git/svn attribute AND a top-level
  url attribute, the url attribute takes precedence.
2018-07-25 23:10:10 -07:00
Adam J. Stewart
5792a805fc core: quick fix for packages without URLs
Some packages do not have a `url` and are instead downloaded via `git`,
`hg`, or `svn`. Some packages like `spectrum-mpi` cannot be downloaded at
all, and are placeholder packages for system installations. Previously,
`__init__()` in `PackageBase` crashed if a package did not have a `url`
attribute defined.

I hacked this section of code out, but I have no idea what the
repercussions of that are.
2018-07-25 23:10:10 -07:00
Adam J. Stewart
13c52066a5 Fix order of regexes reported by spack url summary 2018-07-24 21:02:28 -07:00
Todd Gamblin
656e935e50 core: make spack.util.crypto initialization less expensive.
- This hard-codes the hash lengths rather than computing them on import.

- Also cleans up the code in `spack.util.crypto` to make it easier to
  understand.
2018-07-24 18:35:02 -07:00
Todd Gamblin
674eb00e53
bugfix: spack module -m error message (#8794)
Fix this output error:

```
$ spack -m module loads mpileaks
==> Error: `spack module loads -m t -m c -m l ...` has been moved. Try this instead:

    $ spack module t loads mpileaks
    $ spack module c loads mpileaks
    $ spack module l loads mpileaks
```
2018-07-24 16:12:47 -07:00
Massimiliano Culpo
6bb2256043 Die with a meaningful error if a deprecated command is used
In case a deprecated form of the module command is used, the program
will exit non-zero and print an informative error message suggesting
which command should be used instead.
2018-07-24 11:27:17 -07:00
Massimiliano Culpo
35cef16974 Grouped all the module commands under spack module
As requested in the review all the commands meant to manage module
files have been grouped under the `spack module` command.

Unit tests have been refactored to match the new command structure.
2018-07-24 11:27:17 -07:00
Massimiliano Culpo
7cbe1aedb7 module file tutorial: rebuilt docker image and updated the script 2018-07-24 11:27:17 -07:00
Massimiliano Culpo
576f0c98d2 Added --dependencies to the help text of spack load
fixes #6321
2018-07-24 11:27:17 -07:00
Massimiliano Culpo
443d702971 spack load exits with 1 if module does not exist or is not installed
fixes #2215
fixes #2570
fixes #6676
fixes #7281
closes #3827

This PR reverts the use of `spack module loads` in favor of
`spack module find` when loading module files via Spack. After this PR
`spack load` will accept a single spec at a time, and will be able
to interpret correctly the `--dependencies` option.
2018-07-24 11:27:17 -07:00
Massimiliano Culpo
7a49ba56b6 Added blacklisting of implicit modules in docs + regression tests
fixes #4400

The feature requested in #4400 was already part of the module file
configuration, but it was neither tested nor documented. This
commit takes care of adding a few lines in the documentation and a
regression test.
2018-07-24 11:27:17 -07:00
Massimiliano Culpo
ff83003eaf Renamed 'patch_configuration' fixture to 'module_configuration'
This just because the fixture has been moved one level above the one
it was originally defined. In this more general context there's more
than one configuration file that could be patched for tests.
2018-07-24 11:27:17 -07:00
Massimiliano Culpo
008f171a7e Added a unit test for 'spack lmod setdefault' 2018-07-24 11:27:17 -07:00
Massimiliano Culpo
e81c0c3e2c Updated references to spack module in the documentation. 2018-07-24 11:27:17 -07:00
Massimiliano Culpo
6ab57571c2 Added 'setdefault' subcommand to 'spack lmod'
Added a subcommand that sets the default module to be loaded, if
multiple installations of the same package are present.
2018-07-24 11:27:17 -07:00
Massimiliano Culpo
0457c1fbee Split 'spack module' into multiple commands
'spack module' has been split into multiple commands, each one tied to a
specific module type. This permits the specialization of the new
commands with features that are module type specific (e.g. set the
default module file in lmod when multiple versions of the same package
are installed at the same time).
2018-07-24 11:27:17 -07:00
Todd Gamblin
fdcaf5c4c8 bugfix: fix spack spec --yaml
- repo membership test was broken by the refactor of spack/__init__.py

- refactor singleton so that 'spec in repo' works again for `spack.repo.path`

- fix spec command and add basic tests for `spack spec` and `spack spec --yaml`
2018-07-24 11:25:34 -07:00
Todd Gamblin
d4c4effb5e docs: update old docs for spack.package.PackageBase
- There was a lot of documentation in `PackageBase` dating back to the
  very first versions of Spack.

- It was repetitive and out of date, and the docs at spack.readthedocs.io
  are better.

- Remove the outdated specifics, and leave the minimal useful set of
  developer docs in `package.py`.
2018-07-23 11:57:52 -07:00
Todd Gamblin
bd3ffc7b76 core: use sha256 instead of md5 for spack checksum and spack create
- This changes `get_checksums_for_versions` to generate code that uses an
  explicit `sha256` argument instead if the bare `md5` hash we used to
  generate.

- also use a generic digest parameter for the `version` directive, rather
  than a specific `md5` parameter.
2018-07-23 11:57:52 -07:00
Todd Gamblin
a1d6909864 bugfix: stage test no longer modifies ~/.spack/config.yaml
- two stage tests weren't properly using the mutable_config fixture.
- this caused running `spack test` to modify the user's config.yaml
2018-07-21 17:12:36 -07:00
Todd Gamblin
b2f3b07ec0 docs: doucment spack config blame 2018-07-21 14:04:18 -07:00
Todd Gamblin
331da779f8 docs: updates to spack config get docs for command-line scopes 2018-07-21 14:04:18 -07:00
Elizabeth Fischer
46f87cb9f2 docs: command-line scopes
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
2018-07-21 14:04:18 -07:00
Elizabeth Fischer
52fbbdf5a1 config: allow user to add configuration scopes on the command line.
- Add command-line scope option to Spack

- Rework structure of main to allow configuration system to raise
  errors more naturally

Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
2018-07-21 14:04:18 -07:00
Todd Gamblin
2b0d944341 locks: fix bug when creating lockfiles in the current directory.
- Fixes a bug in `llnl.util.lock`

- Locks in the current directory would fail because the parent directory
  was the empty string.

- Fix this and return '.' for the parent of locks in the current
  directory.
2018-07-21 10:39:47 -07:00
Todd Gamblin
20e4038a72 style: make core comply with pep8-naming 2018-07-19 00:41:36 -07:00
Todd Gamblin
264761ccb8 add py-pep8-naming package 2018-07-19 00:41:36 -07:00
Adam J. Stewart
a67139f6c5 Better Makefile target detection (#8223)
Replace regex-based target detection for Makefiles with a preliminary "make -q"
to check if a target exists. This does not work for NetBSD make; additional work
is required to detect if NetBSD make is present and to use a regex in that case.
The affected makefile target checks are only performed when the "--test" flag is
added to a "spack install" invocation.
2018-07-18 11:11:10 -07:00
Massimiliano Culpo
33aa06e1cf shell: fixed a typo in spack --print-shell-vars (#8732) 2018-07-17 17:03:45 -07:00
Massimiliano Culpo
373b3d2444 Packages must be added to DB to be considered installed (#8038)
Fixes #8036

Before this PR Package.installed was returning True if the spec prefix
existed, without checking the DB. This is wrong for external packages,
whose prefix exists before being registered into the DB. Now the property
checks for both the prefix and a DB entry.
2018-07-17 11:43:30 -07:00
Adam J. Stewart
8ce62ba513 Add documentation on build systems (#5015)
Spack provides a number of classes based on commonly-used build systems
that users can extend when writing packages; the classes provide functionality
to perform the actions relevant to the build system (e.g. running "configure" for
an Autotools-based package). This adds documentation for classes supporting the
following build systems:

* Makefile
* Autotools
* CMake
* QMake
* SCons
* Waf

This includes build systems for managing extensions of the following packages:

* Perl
* Python
* R
* Octave

This also adds documentation on implementing packages that use a custom build
system (e.g. Perl/CMake).

Spack also provides extendable classes which aggregate functionality for related
sets of packages, e.g. those using CUDA. Documentation is added for
CudaPackage.
2018-07-17 11:28:38 -07:00
Todd Gamblin
06418a3dcd
add spack --print-shell-vars to speed up setup-env.[c]sh (#8101)
- The setup-env.sh script currently makes two calls to spack, but it
  should only need to make one.

- Add a fast-path shell setup routine in `main.py` to allow the shell
  setup to happen in a single, fast call that doesn't load more than it
  needs to.

- This simplifies setup code, as it has to eval what Spack prints

- TODO: consider eventually making the whole setup script the output of a
  spack command
2018-07-16 15:43:44 -07:00
Andreas Baumbach
79b407f59e clean up of spack clean (#8610)
* update help of `clean --all` to include `-p`
* remove old orphaned `.pyc` removal
* restrict removal or orphaned pyc files to `lib/spack` and `var/spack`
2018-07-14 17:20:49 -07:00
Todd Gamblin
650786c812 locks: improve errors and permission checking
- Clean up error messages for when a lock can't be created, or when an
  exclusive (write) lock can't be taken on a file.

- Add a number of subclasses of LockError to distinguish timeouts from
  permission issues.

- Add an explicit check to prevent the user from taking a write lock on a
  read-only file.
  - We had a check for this for when we try to *upgrade* a lock on an RO
    file, but not for an initial write lock attempt.

- Add more tests for different lock permission scenarios.
2018-07-12 19:59:53 +02:00
Todd Gamblin
ab794fa741 locks: llnl.util.lock now only writes host info when in debug mode
- write locks previously wrote information about the lock holder (host
  and pid), and read locks woudl read this in.

- This is really only for debugging, so only enable it then

- add some tests that target debug info, and improve multiproc lock test
  output
2018-07-12 19:59:53 +02:00
Todd Gamblin
b9af52a888 tests: Separate tests for llnl.util.lock and spack.util.lock
- llnl.util.lock tests are now independent of Spack
2018-07-12 19:59:53 +02:00
Todd Gamblin
7626ec4579 refactor: move spack.util.multiproc to llnl.util.multiproc
- multiproc doesn't depend on Spack
- llnl.util.lock test uses it, but shouldn't use parts of Spack.
2018-07-12 19:59:53 +02:00
Massimiliano Culpo
e3a556cd2a Don't propagate version-specific URL overrides to newer versions (#8565)
When a user specifies a URL for a specific version of a package, Spack originally
would use that URL for all newer versions of the package. This behavior has
proven to be generally more harmful than useful, so this PR removes the feature
such that a version-specific URL override affects only that version.
2018-07-11 13:12:19 -07:00
Matthias Diener
90042749e8 fetch: remove 'trying' when cloning repos (#8651)
* fetch: remove 'trying' when cloning repos

This makes it similar to what the URL strategy prints.

* change svn/hg names
2018-07-10 10:22:51 -05:00