Commit graph

5188 commits

Author SHA1 Message Date
Todd Gamblin
b324fe5d95 externals: add astunparse
Add `astunparse` as `spack_astunparse`. This library unparses Python ASTs and we're
adding it under our own name so that we can make modifications to it.

Ultimately this will be used to make `package_hash` consistent across Python versions.
2022-01-12 06:14:18 -08:00
Harmen Stoppels
c8e01752a1
Use depends_on over load in lmod module files generated by Spack (#28352) 2022-01-12 13:29:22 +00:00
Peter Scheibel
9f7fb6d01a
stage.steal_source: preserve symlinks
This avoids dangling symlink errors. ignore_dangling_symlinks option would be more-targeted but is only available for Python >= 3.2 (#28318)
2022-01-10 10:10:49 -08:00
Adam J. Stewart
cc32b08205
Python: set default config_vars (#28290)
* Python: set default config_vars

* Add missing commas

* dso_suffix not present for some reason

* Remove use of default_site_packages_dir

* Use config_vars during bootstrapping too

* Catch more errors

* Fix unit tests

* Catch more errors

* Update docstring
2022-01-10 12:00:06 -06:00
Harmen Stoppels
f8d4b4746c
Fix double space in prompt after 'spack env activate -p' (#28279) 2022-01-06 04:59:07 -07:00
Andrew W Elble
a4f0fbafbb
Add function to determine Linux kernel version (#27855)
This reports the kernel version (vs. the distro version) on Linux and
returns a valid Version (stripping characters like '+' which may be
present for custom-built kernels).
2022-01-05 10:22:43 -08:00
Massimiliano Culpo
c2e1a12cdf
Ensure some version known to Spack can satisfy constraints in depends_on (#28131)
Add a new check to `spack audit` to scan and verify that version constraints may be satisfied

Modifications:
 - [x] Add a new check to `spack audit` to scan and verify that version constraints may be satisfied by some version declared in the built-in repository
- [x] Fix issues found by CI

Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
2021-12-29 12:16:07 -08:00
Martin Diehl
3cd599d6f6
Fixed typos: 'wtih' instead of 'with' (#28166) 2021-12-28 09:45:43 +01:00
Massimiliano Culpo
60a5b2a85e Merge tag 'v0.17.1' into develop 2021-12-23 19:48:36 +01:00
Massimiliano Culpo
4381cb5957
New subcommand: spack bootstrap status (#28004)
This command pokes the environment, Python interpreter
and bootstrap store to check if dependencies needed by
Spack are available.

If any are missing, it shows a comprehensible message.
2021-12-23 10:34:04 -08:00
Massimiliano Culpo
69b8cddb1b Bump version and update CHANGELOG.md 2021-12-23 16:02:09 +01:00
Tom Scogland
8e659f512e locks: allow locks to work under high contention (#27846)
* locks: allow locks to work under high contention

This is a bug found by Harshitha Menon.  

The `lock=None` line shouldn't be a release but should be 
```
return (lock_type, None)
``` 
to inform the caller it couldn't get the lock type requested without
disturbing the existing lock object in the database.  There were also a
couple of bugs due to taking write locks at the beginning without any
checking or release, and not releasing read locks before requeueing.
This version no longer gives me read upgrade to write errors, even
running 200 instances on one box.

* Change lock in check_deps_status to read, release if not installed,
  not sure why this was ever write, but read definitely is more
  appropriate here, and the read lock is only held out of the scope if
  the package is installed.

* Release read lock before requeueing to reduce chance of livelock, the
  timeout that caused the original issue now happens in roughly 3 of 200
  workers instead of 199 on average.
2021-12-23 16:02:09 +01:00
Harmen Stoppels
5daf023aec Regenerate views when specs already installed (#28113)
With this commit:
```
$ spack env activate --temp
$ spack install zlib
==> All of the packages are already installed
==> Updating view at /tmp/spack-faiirgmt/.spack-env/view
$ spack install zlib
==> All of the packages are already installed
```

Before this PR:
```
$ spack env activate --temp
$ spack install zlib
==> All of the packages are already installed
$ spack install zlib
==> All of the packages are already installed
```

No view was generated
2021-12-23 16:02:09 +01:00
Paul Spencer
e1cc28a30a sbang: respect package permissive package permissions for sbang (#25764)
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
2021-12-23 16:02:09 +01:00
victorusu
17edf1ae90 Add setdefault option to tcl module (#14686)
This commit introduces the command

spack module tcl setdefault <package>

similar to the one already available for lmod

Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2021-12-23 16:02:09 +01:00
Harmen Stoppels
79fd1c5114 Set backup=False by default in filter_file (#28036) 2021-12-23 16:02:09 +01:00
Harmen Stoppels
13e36c5457 Fix table formatting (#28037) 2021-12-23 16:02:09 +01:00
Harmen Stoppels
b2694013d4 Revert "patches: make re-applied patches idempotent (#26784)" (#27625)
This reverts commit c5ca0db27f.
2021-12-23 16:02:09 +01:00
Andrew W Elble
8f3b025b55 MANPATH needs a trailing ':' to utilize system defaults (#21682)
otherwise spack breaks using system man pages by default.

Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
2021-12-23 16:02:09 +01:00
Christian Goll
37fbe30c4a Added opensuse/leap:15 to spack containerize (#27837)
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2021-12-23 16:02:09 +01:00
Tamara Dahlgren
314867e635 Provide meaningful message for empty environment installs (#28031)
* Provide a meaningful failure message for installation of an empty environment

* Allow regenerating view per offline discussion
2021-12-23 16:02:09 +01:00
Greg Becker
9345bf81b9 Add option to minimize full debug cores. include warning message about performance (#27970)
Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
2021-12-23 16:02:09 +01:00
Harmen Stoppels
20ad47f9e1 Install dir creation message demoted to "debug" level (#27911) 2021-12-23 16:02:09 +01:00
Greg Becker
7e5de95a30 Improve debug info from concretizer (#27707) 2021-12-23 16:02:09 +01:00
Massimiliano Culpo
e9f7fb03c9 spack audit: fix API calls (#27713)
This broke in #24858
2021-12-23 16:02:09 +01:00
Paul Ferrell
9d4291e590 Handle byte sequences which are not encoded as UTF8 while logging. (#21447)
Fix builds which produce a lines with non-UTF8 output while logging
The alternative is to read in binary mode, and then decode while
ignoring errors.
2021-12-23 16:02:09 +01:00
Harmen Stoppels
654f6839eb ci: run style unit tests only if we target develop (#27472)
Some tests assume the base branch is develop, but this branch may not
have been checked out.
2021-12-23 16:02:09 +01:00
Harmen Stoppels
c8daa7218d Turn some verbose messages into debug messages (#27408) 2021-12-23 16:02:09 +01:00
Harmen Stoppels
d862507bcf Fix overloaded argparse keys (#27379)
Commands should not reuse option names defined in main.
2021-12-23 16:02:09 +01:00
Jordan Galby
7c6b253d89 Fix log-format reporter ignoring install errors (#25961)
When running `spack install --log-format junit|cdash ...`, install
errors were ignored. This made spack continue building dependents of
failed install, ignoring `--fail-fast`, and exit 0 at the end.
2021-12-23 16:02:09 +01:00
Dylan Simon
544826c825 make --enable-locks actually enable locks (#24675) 2021-12-23 16:02:09 +01:00
Tom Scogland
b7b6542804
locks: allow locks to work under high contention (#27846)
* locks: allow locks to work under high contention

This is a bug found by Harshitha Menon.  

The `lock=None` line shouldn't be a release but should be 
```
return (lock_type, None)
``` 
to inform the caller it couldn't get the lock type requested without
disturbing the existing lock object in the database.  There were also a
couple of bugs due to taking write locks at the beginning without any
checking or release, and not releasing read locks before requeueing.
This version no longer gives me read upgrade to write errors, even
running 200 instances on one box.

* Change lock in check_deps_status to read, release if not installed,
  not sure why this was ever write, but read definitely is more
  appropriate here, and the read lock is only held out of the scope if
  the package is installed.

* Release read lock before requeueing to reduce chance of livelock, the
  timeout that caused the original issue now happens in roughly 3 of 200
  workers instead of 199 on average.
2021-12-22 16:25:05 +01:00
Joseph Snyder
34873f5fe7
Use consistent method of checking for presence of info in connection (#27694)
Fixes #27652

Ensure that mirror's to_dict function returns a syaml_dict object for all code
paths.

Switch to using the .get function for accessing the potential information from
the S3 mirror objects.  If the key is not there, it will gracefully return
None instead of failing with a KeyError

Additionally, check that the connection object is a dictionary before trying
to "get" from it.

Add a test for the capturing of the new S3 information.
2021-12-22 16:15:49 +01:00
Harmen Stoppels
db69a291d4
Regenerate views when specs already installed (#28113)
With this commit:
```
$ spack env activate --temp
$ spack install zlib
==> All of the packages are already installed
==> Updating view at /tmp/spack-faiirgmt/.spack-env/view
$ spack install zlib
==> All of the packages are already installed
```

Before this PR:
```
$ spack env activate --temp
$ spack install zlib
==> All of the packages are already installed
$ spack install zlib
==> All of the packages are already installed
```

No view was generated
2021-12-21 18:41:12 +01:00
Robert Cohn
d45280369f
Intel oneAPI packages: add 2022.1.1 release (#28096)
Co-authored-by: Egorov, Andrey <andrey.egorov@intel.com>
2021-12-20 14:06:32 -08:00
Vanessasaurus
a94b4eef79
Fixing spacing of libabigail to : (#28085) 2021-12-20 03:56:03 -07:00
Vanessasaurus
da9e152ed1
Fix bugs in spack monitor (#27511)
Updates to installer.py did not account for spack monitor, so as currently implemented
there are three cases of failure that spack monitor will not account for. To fix this we add additional
hooks, including an on cancel and also do a custom action on concretization fail.

Signed-off-by: vsoch <vsoch@users.noreply.github.com>

Co-authored-by: vsoch <vsoch@users.noreply.github.com>
2021-12-20 06:54:41 +01:00
Todd Gamblin
bfc69f0e4b jsonschema: use more specific schema identifiers
The latest version of `jsonschema` fails if we're not specific about which schema draft
specification we're using.  Update all of them to use the latest one (draft-07).
2021-12-19 12:55:42 -08:00
Todd Gamblin
7703043195 externals: Upgrade jsonschema to v3.2.0
Our `jsonschema` external won't support Python 3.10, so we need to upgrade it.
It currently generates this warning:

    lib/spack/external/jsonschema/compat.py:6: DeprecationWarning: Using or importing the ABCs
        from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and
        in 3.10 it will stop working

This upgrades `jsonschema` to 3.2.0, the latest version with support for Python 2.7.  The next
version after this (4.0.0) drops support for 2.7 and 3.6, so we'll have to wait to upgrade to it.

Dependencies have been added in prior commits.
2021-12-19 12:55:42 -08:00
Todd Gamblin
f83e0fb81a externals: add attrs for new jsonschema
Updating `jsonschema` to 3.2.0 requires `attrs`. Add it to externals.
2021-12-19 12:55:42 -08:00
Todd Gamblin
90592b3cbe externals: add pyrsistent for new jsonschema
Updating `jsonschema` to 3.2.0 requires `pyrsistent`. Adding just the pieces of it
that are needed for `jsonschema`.
2021-12-19 12:55:42 -08:00
Todd Gamblin
04536db387 externals: add functools32 for new jsonschema
Updating `jsonschema` to 3.2.0 requires `functools32`, just for Python 2.
2021-12-19 12:55:42 -08:00
Tamara Dahlgren
e470131a77
Tests: remove test_get_stage_root_in_spack since it writes to the spack directory (#28056) 2021-12-19 11:31:38 -08:00
Paul Spencer
30244c2c40
sbang: respect package permissive package permissions for sbang (#25764)
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
2021-12-18 22:07:20 -08:00
Vanessasaurus
d9c4b91af3
Remove ability to run spack monitor without auth (#27888)
spack monitor now requires authentication as each build must be associated
with a user, so it does not make sense to allow the --monitor-no-auth flag
and this commit will remove it
2021-12-17 18:00:43 +01:00
victorusu
18615b1485
Add setdefault option to tcl module (#14686)
This commit introduces the command

spack module tcl setdefault <package>

similar to the one already available for lmod

Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2021-12-17 10:05:32 +01:00
Harmen Stoppels
fb93979b94
Set backup=False by default in filter_file (#28036) 2021-12-16 13:50:20 +01:00
Harmen Stoppels
6357de4e61
Fix table formatting (#28037) 2021-12-16 12:13:12 +00:00
Andrew W Elble
96535cc4f9
MANPATH needs a trailing ':' to utilize system defaults (#21682)
otherwise spack breaks using system man pages by default.

Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
2021-12-16 10:54:35 +00:00
Christian Goll
bd0ffa8a3c
Added opensuse/leap:15 to spack containerize (#27837)
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2021-12-16 09:47:15 +00:00