Commit graph

5146 commits

Author SHA1 Message Date
Tamara Dahlgren
e470131a77
Tests: remove test_get_stage_root_in_spack since it writes to the spack directory (#28056) 2021-12-19 11:31:38 -08:00
Paul Spencer
30244c2c40
sbang: respect package permissive package permissions for sbang (#25764)
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
2021-12-18 22:07:20 -08:00
Vanessasaurus
d9c4b91af3
Remove ability to run spack monitor without auth (#27888)
spack monitor now requires authentication as each build must be associated
with a user, so it does not make sense to allow the --monitor-no-auth flag
and this commit will remove it
2021-12-17 18:00:43 +01:00
victorusu
18615b1485
Add setdefault option to tcl module (#14686)
This commit introduces the command

spack module tcl setdefault <package>

similar to the one already available for lmod

Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2021-12-17 10:05:32 +01:00
Harmen Stoppels
fb93979b94
Set backup=False by default in filter_file (#28036) 2021-12-16 13:50:20 +01:00
Harmen Stoppels
6357de4e61
Fix table formatting (#28037) 2021-12-16 12:13:12 +00:00
Andrew W Elble
96535cc4f9
MANPATH needs a trailing ':' to utilize system defaults (#21682)
otherwise spack breaks using system man pages by default.

Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
2021-12-16 10:54:35 +00:00
Christian Goll
bd0ffa8a3c
Added opensuse/leap:15 to spack containerize (#27837)
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2021-12-16 09:47:15 +00:00
Tamara Dahlgren
ac49ce8b3b
Provide meaningful message for empty environment installs (#28031)
* Provide a meaningful failure message for installation of an empty environment

* Allow regenerating view per offline discussion
2021-12-15 22:56:54 -08:00
Tamara Dahlgren
9240614928
Bugfix: Simplify preferred-test versions; set checksum defaults (#28026)
* Simplify preferred-test versions; set checksum defaults

* Fix test_preferred failure
2021-12-16 06:55:28 +01:00
Harmen Stoppels
72ca7d6ee5
Revert "patches: make re-applied patches idempotent (#26784)" (#27625)
This reverts commit c5ca0db27f.
2021-12-15 08:56:03 -08:00
Greg Becker
5319b6e3b1
Add option to minimize full debug cores. include warning message about performance (#27970)
Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
2021-12-15 08:52:53 +01:00
Zack Galbreath
3139894794
ci: Catch Exceptions when attempting to register a CDash build (#27883)
This PR also slightly changes the behavior in ci_rebuild().
We now still attempt to submit `spack install` results to CDash
even if the initial registration failed due to connection issues.

This commit follows in the spirit of #24299. We do not want `spack install`
to exit with a non-zero status when something goes wrong while attempting to
report results to CDash.
2021-12-14 14:18:08 -07:00
Harmen Stoppels
f6802b733a
Install dir creation message demoted to "debug" level (#27911) 2021-12-13 11:45:31 +01:00
Adam J. Stewart
0960c0810c
Extends: support spec, not just package name (#27754) 2021-12-10 13:30:21 +01:00
Greg Becker
dc87157e80
Improve debug info from concretizer (#27707) 2021-12-10 10:49:33 +01:00
Massimiliano Culpo
d17511a806
Refactor "spack buildcache" command (#27776)
This PR is meant to move code with "business logic" from `spack.cmd.buildcache` to appropriate core modules[^1]. 

Modifications:
- [x] Add `spack.binary_distribution.push` to create a binary package from a spec and push it to a mirror
- [x] Add `spack.binary_distribution.install_root_node` to install only the root node of a concrete spec from a buildcache (may check the sha256 sum if it is passed in as input)
- [x] Add `spack.binary_distribution.install_single_spec` to install a single concrete spec from a buildcache
- [x] Add `spack.binary_distribution.download_single_spec` to download a single concrete spec from a buildcache to a local destination
- [x] Add `Spec.from_specfile` that construct a spec given the path of a JSON or YAML spec file
- [x] Removed logic from `spack.cmd.buildcache`
- [x] Removed calls to `spack.cmd.buildcache` in `spack.bootstrap`
- [x] Deprecate `spack buildcache copy` with a message that says it will be removed in v0.19.0

[^1]: The rationale is that commands should be lightweight wrappers of the core API, since that helps with both testing and scripting (easier mocking and no need to invoke `SpackCommand`s in a script).
2021-12-10 10:23:14 +01:00
Massimiliano Culpo
01d077d4bc
Make external detection more resilient to individual package errors (#27854)
After this PR an error in a single package while detecting
external software won't abort the entire procedure.

The error is reported to screen as a warning.
2021-12-08 13:58:21 -08:00
Tamara Dahlgren
d458e82286
rocm: add gfx90a (#27823) 2021-12-07 17:19:48 -08:00
Harmen Stoppels
3d1b9e4dbc
"spack buildcache install": don't catch exception (#27674)
Remove a try/catch for an error with no handling. If the affected
code doesn't execute successfully, then the associated variable
is undefined and another (more-obscure) error occurs shortly after.
2021-12-07 13:17:17 -08:00
Massimiliano Culpo
f81d84dfc6
Release procedure: add a step to update docs (#27734) 2021-12-07 11:30:14 +00:00
Peter Scheibel
edb971a10e
Support packages which need to explicitly refer to dpcpp by name (#27168)
* Hack to support packages which need to explicitly refer to dpcpp by name
* cc script needs to know about dpcpp
2021-12-02 15:49:20 -08:00
Massimiliano Culpo
645a7dc14c
spack audit: fix API calls (#27713)
This broke in #24858
2021-11-30 14:59:55 +01:00
Thomas Madlener
ecb588740a
Speed up install of environments with dev packages (#27167)
* only check file modification times once per dev package
2021-11-29 09:34:23 -08:00
Harmen Stoppels
037ece674b
distro: don't use deprecated linux_distribution (#27658) 2021-11-29 16:26:19 +01:00
Massimiliano Culpo
0d10408a25
bootstrap: restrict patchelf to v0.13.x (#27685) 2021-11-29 15:41:25 +01:00
Paul Ferrell
c0edb17b93
Handle byte sequences which are not encoded as UTF8 while logging. (#21447)
Fix builds which produce a lines with non-UTF8 output while logging
The alternative is to read in binary mode, and then decode while
ignoring errors.
2021-11-29 13:27:02 +01:00
Massimiliano Culpo
a96f2f603b
Bootstrap patchelf like GnuPG (#27532)
Remove a custom bootstrapping procedure to
use spack.bootstrap instead

Modifications:
* Reference count the bootstrap context manager
* Avoid SpackCommand to make the bootstrapping
  procedure more transparent
* Put back requirement on patchelf being in PATH for unit tests
* Add an e2e test to check bootstrapping patchelf
2021-11-26 15:32:13 +01:00
Maxim Belkin
6e095a9741
module_file_support: update format for configuration (#27598) 2021-11-25 08:41:32 +01:00
Massimiliano Culpo
270ba10962
spack flake8: remove deprecated command (#27290)
The "spack flake8" command wwas deprecated in favor
of "spack style". The deprecation wwarning is in the
0.17.X series, so removing it for v0.18.x
2021-11-24 14:20:11 -08:00
Robert Cohn
76ad803f25
intel-oneapi-mkl: add cluster libs and option for static linking (#26256) 2021-11-24 11:04:05 -08:00
Harmen Stoppels
dbf67d912c
Make patchelf test use the realpath (#27618)
I think this test should be removed, but when it stays, it should at
least follow the symlink, cause it fails for me if I let spack build
patchelf and have a symlink in a view.
2021-11-24 11:17:23 +01:00
Massimiliano Culpo
5c3dfacdc5
Update distro to v1.6.0 (#27263) 2021-11-24 10:10:11 +00:00
Massimiliano Culpo
70d5d234db
Update Jinja2 to v2.11.3 and MarkupSafe to v1.1.1 (#27264) 2021-11-24 10:21:35 +01:00
Massimiliano Culpo
12da0a9a69
Update six to v1.16.0 (#27265) 2021-11-24 10:20:04 +01:00
Harmen Stoppels
cced832cac
Fix leaky tests (#27616)
* fix: cc.py should use a function not session scope
* fix: don't let build env vars leak to other tests
* fix: don't leak build env in dev_build test
2021-11-23 14:10:48 -08:00
Massimiliano Culpo
fa7189b480
Remove support for Python 2.6 (#27256)
Modifications:
- [x] Removed `centos:6` unit test, adjusted vermin checks
- [x] Removed backport of `collections.OrderedDict`
- [x] Removed backport of `functools.total_ordering`
- [x] Removed Python 2.6 specific skip markers in unit tests
- [x] Fixed a few minor Python 2.6 related TODOs in code

Updating the vendored dependencies will be done in separate PRs
2021-11-23 09:06:17 -08:00
Nathan Hanford
2104f1273a
bugfix: Allow legacy tests to be read after hash break (#26078)
* added a test case

Co-authored-by: Nathan Hanford <hanford1@llnl.gov>
2021-11-22 21:49:41 -07:00
Piotr Luszczek
e90d5ad6cf
Intel packages: add support for LLVM OpenMP (#26517) 2021-11-22 13:41:43 -08:00
Massimiliano Culpo
5eba5dc271
Make CUDA and ROCm architecture conditional (#27185)
* Make CUDA and ROCm architecture conditional

fixes #14337

The variant to specify which architecture to use
for CUDA and ROCm are now conditional on +cuda and
+rocm respectively.

* cp2k: make all CUDA related variants conditional on +cuda
2021-11-22 07:54:19 -05:00
Harmen Stoppels
0024e5cc9b
Make _enable_or_disable(...) return an empty array for conditional variants whose condition is not met (#27504) 2021-11-22 10:47:09 +01:00
Joseph Snyder
d759612523
Add connection specification to mirror creation (#24244)
* Add connection specification to mirror creation

This allows each mirror to contain information about the credentials
used to access it.

Update command and tests based on comments

Switch to only "long form" flags for the s3 connection information.
Use the "any" function instead of checking for an empty list when looking
for s3 connection information.

Split test to use the access token separately from the access id and key.
Use long flag form in test.

Add endpoint_url to available S3 options.

Extend the special parameters for an S3 mirror to accept the
endpoint_url parameter.

Add a test.

* Add connection information per URL not per mirror

Expand the mirror-based connection information to be per-URL.
This will allow a user to specify different S3 connection information
for both the fetch and the push URLs.

Add a parameter for "profile", another way of storing the id/secret pair.

* Switch from "access_profile" to "profile"
2021-11-19 15:28:34 -05:00
Harmen Stoppels
c5aee4d9b4
define_from_variant: return an empty string for non-existing variants (#27503)
This permits to use conditional variants without a lot of boilerplate.
2021-11-19 14:10:00 +01:00
Michael Davis
3375db12a5
Adding --reuse to dev-build command. (#27487) 2021-11-19 09:25:45 +01:00
Massimiliano Culpo
57d3b02800
Remove spurious debug print (#27541) 2021-11-19 09:02:22 +01:00
Massimiliano Culpo
e3cd91af53
Refactor bootstrap of "spack style" dependencies (#27485)
Remove the "get_executable" function from the
spack.bootstrap module. Now "flake8", "isort",
"mypy" and "black" will use the same
bootstrapping method as GnuPG.
2021-11-18 15:23:09 +01:00
Massimiliano Culpo
f981682bdc
Allow recent pytest versions to be used with Spack (#25371)
Currently Spack vendors `pytest` at a version which is three major 
versions behind the latest (3.2.5 vs. 6.2.4). We do that since v3.2.5 
is the latest version supporting Python 2.6. Remaining so much 
behind the currently supported versions though might introduce 
some incompatibilities and is surely a technical debt.

This PR modifies Spack to:
- Use the vendored `pytest@3.2.5` only as a fallback solution, 
  if the Python interpreter used for Spack doesn't provide a newer one
- Be able to parse `pytest --collect-only` in all the different output 
  formats from v3.2.5 to v6.2.4 and use it consistently for `spack unit-test --list-*`
- Updating the unit tests in Github Actions to use a more recent `pytest` version
2021-11-18 15:08:59 +01:00
Harmen Stoppels
8f7640dbef
ci: run style unit tests only if we target develop (#27472)
Some tests assume the base branch is develop, but this branch may not
have been checked out.
2021-11-18 13:00:39 +01:00
Harmen Stoppels
cc62689504
Fix overly generic exceptions in log parser (#27413)
This type of error is skipped:

make[1]: *** [Makefile:222: /tmp/user/spack-stage/.../spack-src/usr/lib/julia/libopenblas64_.so.so] Error 1

but it's useful to have it, especially when a package sets a variable
incorrectly in makefiles
2021-11-17 11:24:14 +01:00
Robert Cohn
67cba372e8
Intel mpi: allow use of external libfabric (#27292)
Intel mpi comes with an installation of libfabric (which it needs as a
dependency). It can use other implementations of libfabric at runtime
though, so if you install a package that depends on `mpi` and
`libfabric`, you can specify `intel-mpi+external-libfabric` and ensure
that the Spack-built instance is used (both by `intel-mpi` and the
root).

Apply analogous change to intel-oneapi-mpi.
2021-11-16 12:55:24 -08:00