Commit graph

5152 commits

Author SHA1 Message Date
Vanessasaurus
da9e152ed1
Fix bugs in spack monitor (#27511)
Updates to installer.py did not account for spack monitor, so as currently implemented
there are three cases of failure that spack monitor will not account for. To fix this we add additional
hooks, including an on cancel and also do a custom action on concretization fail.

Signed-off-by: vsoch <vsoch@users.noreply.github.com>

Co-authored-by: vsoch <vsoch@users.noreply.github.com>
2021-12-20 06:54:41 +01:00
Todd Gamblin
bfc69f0e4b jsonschema: use more specific schema identifiers
The latest version of `jsonschema` fails if we're not specific about which schema draft
specification we're using.  Update all of them to use the latest one (draft-07).
2021-12-19 12:55:42 -08:00
Todd Gamblin
7703043195 externals: Upgrade jsonschema to v3.2.0
Our `jsonschema` external won't support Python 3.10, so we need to upgrade it.
It currently generates this warning:

    lib/spack/external/jsonschema/compat.py:6: DeprecationWarning: Using or importing the ABCs
        from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and
        in 3.10 it will stop working

This upgrades `jsonschema` to 3.2.0, the latest version with support for Python 2.7.  The next
version after this (4.0.0) drops support for 2.7 and 3.6, so we'll have to wait to upgrade to it.

Dependencies have been added in prior commits.
2021-12-19 12:55:42 -08:00
Todd Gamblin
f83e0fb81a externals: add attrs for new jsonschema
Updating `jsonschema` to 3.2.0 requires `attrs`. Add it to externals.
2021-12-19 12:55:42 -08:00
Todd Gamblin
90592b3cbe externals: add pyrsistent for new jsonschema
Updating `jsonschema` to 3.2.0 requires `pyrsistent`. Adding just the pieces of it
that are needed for `jsonschema`.
2021-12-19 12:55:42 -08:00
Todd Gamblin
04536db387 externals: add functools32 for new jsonschema
Updating `jsonschema` to 3.2.0 requires `functools32`, just for Python 2.
2021-12-19 12:55:42 -08:00
Tamara Dahlgren
e470131a77
Tests: remove test_get_stage_root_in_spack since it writes to the spack directory (#28056) 2021-12-19 11:31:38 -08:00
Paul Spencer
30244c2c40
sbang: respect package permissive package permissions for sbang (#25764)
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
2021-12-18 22:07:20 -08:00
Vanessasaurus
d9c4b91af3
Remove ability to run spack monitor without auth (#27888)
spack monitor now requires authentication as each build must be associated
with a user, so it does not make sense to allow the --monitor-no-auth flag
and this commit will remove it
2021-12-17 18:00:43 +01:00
victorusu
18615b1485
Add setdefault option to tcl module (#14686)
This commit introduces the command

spack module tcl setdefault <package>

similar to the one already available for lmod

Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2021-12-17 10:05:32 +01:00
Harmen Stoppels
fb93979b94
Set backup=False by default in filter_file (#28036) 2021-12-16 13:50:20 +01:00
Harmen Stoppels
6357de4e61
Fix table formatting (#28037) 2021-12-16 12:13:12 +00:00
Andrew W Elble
96535cc4f9
MANPATH needs a trailing ':' to utilize system defaults (#21682)
otherwise spack breaks using system man pages by default.

Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
2021-12-16 10:54:35 +00:00
Christian Goll
bd0ffa8a3c
Added opensuse/leap:15 to spack containerize (#27837)
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2021-12-16 09:47:15 +00:00
Tamara Dahlgren
ac49ce8b3b
Provide meaningful message for empty environment installs (#28031)
* Provide a meaningful failure message for installation of an empty environment

* Allow regenerating view per offline discussion
2021-12-15 22:56:54 -08:00
Tamara Dahlgren
9240614928
Bugfix: Simplify preferred-test versions; set checksum defaults (#28026)
* Simplify preferred-test versions; set checksum defaults

* Fix test_preferred failure
2021-12-16 06:55:28 +01:00
Harmen Stoppels
72ca7d6ee5
Revert "patches: make re-applied patches idempotent (#26784)" (#27625)
This reverts commit c5ca0db27f.
2021-12-15 08:56:03 -08:00
Greg Becker
5319b6e3b1
Add option to minimize full debug cores. include warning message about performance (#27970)
Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
2021-12-15 08:52:53 +01:00
Zack Galbreath
3139894794
ci: Catch Exceptions when attempting to register a CDash build (#27883)
This PR also slightly changes the behavior in ci_rebuild().
We now still attempt to submit `spack install` results to CDash
even if the initial registration failed due to connection issues.

This commit follows in the spirit of #24299. We do not want `spack install`
to exit with a non-zero status when something goes wrong while attempting to
report results to CDash.
2021-12-14 14:18:08 -07:00
Harmen Stoppels
f6802b733a
Install dir creation message demoted to "debug" level (#27911) 2021-12-13 11:45:31 +01:00
Adam J. Stewart
0960c0810c
Extends: support spec, not just package name (#27754) 2021-12-10 13:30:21 +01:00
Greg Becker
dc87157e80
Improve debug info from concretizer (#27707) 2021-12-10 10:49:33 +01:00
Massimiliano Culpo
d17511a806
Refactor "spack buildcache" command (#27776)
This PR is meant to move code with "business logic" from `spack.cmd.buildcache` to appropriate core modules[^1]. 

Modifications:
- [x] Add `spack.binary_distribution.push` to create a binary package from a spec and push it to a mirror
- [x] Add `spack.binary_distribution.install_root_node` to install only the root node of a concrete spec from a buildcache (may check the sha256 sum if it is passed in as input)
- [x] Add `spack.binary_distribution.install_single_spec` to install a single concrete spec from a buildcache
- [x] Add `spack.binary_distribution.download_single_spec` to download a single concrete spec from a buildcache to a local destination
- [x] Add `Spec.from_specfile` that construct a spec given the path of a JSON or YAML spec file
- [x] Removed logic from `spack.cmd.buildcache`
- [x] Removed calls to `spack.cmd.buildcache` in `spack.bootstrap`
- [x] Deprecate `spack buildcache copy` with a message that says it will be removed in v0.19.0

[^1]: The rationale is that commands should be lightweight wrappers of the core API, since that helps with both testing and scripting (easier mocking and no need to invoke `SpackCommand`s in a script).
2021-12-10 10:23:14 +01:00
Massimiliano Culpo
01d077d4bc
Make external detection more resilient to individual package errors (#27854)
After this PR an error in a single package while detecting
external software won't abort the entire procedure.

The error is reported to screen as a warning.
2021-12-08 13:58:21 -08:00
Tamara Dahlgren
d458e82286
rocm: add gfx90a (#27823) 2021-12-07 17:19:48 -08:00
Harmen Stoppels
3d1b9e4dbc
"spack buildcache install": don't catch exception (#27674)
Remove a try/catch for an error with no handling. If the affected
code doesn't execute successfully, then the associated variable
is undefined and another (more-obscure) error occurs shortly after.
2021-12-07 13:17:17 -08:00
Massimiliano Culpo
f81d84dfc6
Release procedure: add a step to update docs (#27734) 2021-12-07 11:30:14 +00:00
Peter Scheibel
edb971a10e
Support packages which need to explicitly refer to dpcpp by name (#27168)
* Hack to support packages which need to explicitly refer to dpcpp by name
* cc script needs to know about dpcpp
2021-12-02 15:49:20 -08:00
Massimiliano Culpo
645a7dc14c
spack audit: fix API calls (#27713)
This broke in #24858
2021-11-30 14:59:55 +01:00
Thomas Madlener
ecb588740a
Speed up install of environments with dev packages (#27167)
* only check file modification times once per dev package
2021-11-29 09:34:23 -08:00
Harmen Stoppels
037ece674b
distro: don't use deprecated linux_distribution (#27658) 2021-11-29 16:26:19 +01:00
Massimiliano Culpo
0d10408a25
bootstrap: restrict patchelf to v0.13.x (#27685) 2021-11-29 15:41:25 +01:00
Paul Ferrell
c0edb17b93
Handle byte sequences which are not encoded as UTF8 while logging. (#21447)
Fix builds which produce a lines with non-UTF8 output while logging
The alternative is to read in binary mode, and then decode while
ignoring errors.
2021-11-29 13:27:02 +01:00
Massimiliano Culpo
a96f2f603b
Bootstrap patchelf like GnuPG (#27532)
Remove a custom bootstrapping procedure to
use spack.bootstrap instead

Modifications:
* Reference count the bootstrap context manager
* Avoid SpackCommand to make the bootstrapping
  procedure more transparent
* Put back requirement on patchelf being in PATH for unit tests
* Add an e2e test to check bootstrapping patchelf
2021-11-26 15:32:13 +01:00
Maxim Belkin
6e095a9741
module_file_support: update format for configuration (#27598) 2021-11-25 08:41:32 +01:00
Massimiliano Culpo
270ba10962
spack flake8: remove deprecated command (#27290)
The "spack flake8" command wwas deprecated in favor
of "spack style". The deprecation wwarning is in the
0.17.X series, so removing it for v0.18.x
2021-11-24 14:20:11 -08:00
Robert Cohn
76ad803f25
intel-oneapi-mkl: add cluster libs and option for static linking (#26256) 2021-11-24 11:04:05 -08:00
Harmen Stoppels
dbf67d912c
Make patchelf test use the realpath (#27618)
I think this test should be removed, but when it stays, it should at
least follow the symlink, cause it fails for me if I let spack build
patchelf and have a symlink in a view.
2021-11-24 11:17:23 +01:00
Massimiliano Culpo
5c3dfacdc5
Update distro to v1.6.0 (#27263) 2021-11-24 10:10:11 +00:00
Massimiliano Culpo
70d5d234db
Update Jinja2 to v2.11.3 and MarkupSafe to v1.1.1 (#27264) 2021-11-24 10:21:35 +01:00
Massimiliano Culpo
12da0a9a69
Update six to v1.16.0 (#27265) 2021-11-24 10:20:04 +01:00
Harmen Stoppels
cced832cac
Fix leaky tests (#27616)
* fix: cc.py should use a function not session scope
* fix: don't let build env vars leak to other tests
* fix: don't leak build env in dev_build test
2021-11-23 14:10:48 -08:00
Massimiliano Culpo
fa7189b480
Remove support for Python 2.6 (#27256)
Modifications:
- [x] Removed `centos:6` unit test, adjusted vermin checks
- [x] Removed backport of `collections.OrderedDict`
- [x] Removed backport of `functools.total_ordering`
- [x] Removed Python 2.6 specific skip markers in unit tests
- [x] Fixed a few minor Python 2.6 related TODOs in code

Updating the vendored dependencies will be done in separate PRs
2021-11-23 09:06:17 -08:00
Nathan Hanford
2104f1273a
bugfix: Allow legacy tests to be read after hash break (#26078)
* added a test case

Co-authored-by: Nathan Hanford <hanford1@llnl.gov>
2021-11-22 21:49:41 -07:00
Piotr Luszczek
e90d5ad6cf
Intel packages: add support for LLVM OpenMP (#26517) 2021-11-22 13:41:43 -08:00
Massimiliano Culpo
5eba5dc271
Make CUDA and ROCm architecture conditional (#27185)
* Make CUDA and ROCm architecture conditional

fixes #14337

The variant to specify which architecture to use
for CUDA and ROCm are now conditional on +cuda and
+rocm respectively.

* cp2k: make all CUDA related variants conditional on +cuda
2021-11-22 07:54:19 -05:00
Harmen Stoppels
0024e5cc9b
Make _enable_or_disable(...) return an empty array for conditional variants whose condition is not met (#27504) 2021-11-22 10:47:09 +01:00
Joseph Snyder
d759612523
Add connection specification to mirror creation (#24244)
* Add connection specification to mirror creation

This allows each mirror to contain information about the credentials
used to access it.

Update command and tests based on comments

Switch to only "long form" flags for the s3 connection information.
Use the "any" function instead of checking for an empty list when looking
for s3 connection information.

Split test to use the access token separately from the access id and key.
Use long flag form in test.

Add endpoint_url to available S3 options.

Extend the special parameters for an S3 mirror to accept the
endpoint_url parameter.

Add a test.

* Add connection information per URL not per mirror

Expand the mirror-based connection information to be per-URL.
This will allow a user to specify different S3 connection information
for both the fetch and the push URLs.

Add a parameter for "profile", another way of storing the id/secret pair.

* Switch from "access_profile" to "profile"
2021-11-19 15:28:34 -05:00
Harmen Stoppels
c5aee4d9b4
define_from_variant: return an empty string for non-existing variants (#27503)
This permits to use conditional variants without a lot of boilerplate.
2021-11-19 14:10:00 +01:00
Michael Davis
3375db12a5
Adding --reuse to dev-build command. (#27487) 2021-11-19 09:25:45 +01:00