Commit graph

5128 commits

Author SHA1 Message Date
Tamara Dahlgren
d458e82286
rocm: add gfx90a (#27823) 2021-12-07 17:19:48 -08:00
Harmen Stoppels
3d1b9e4dbc
"spack buildcache install": don't catch exception (#27674)
Remove a try/catch for an error with no handling. If the affected
code doesn't execute successfully, then the associated variable
is undefined and another (more-obscure) error occurs shortly after.
2021-12-07 13:17:17 -08:00
Massimiliano Culpo
f81d84dfc6
Release procedure: add a step to update docs (#27734) 2021-12-07 11:30:14 +00:00
Peter Scheibel
edb971a10e
Support packages which need to explicitly refer to dpcpp by name (#27168)
* Hack to support packages which need to explicitly refer to dpcpp by name
* cc script needs to know about dpcpp
2021-12-02 15:49:20 -08:00
Massimiliano Culpo
645a7dc14c
spack audit: fix API calls (#27713)
This broke in #24858
2021-11-30 14:59:55 +01:00
Thomas Madlener
ecb588740a
Speed up install of environments with dev packages (#27167)
* only check file modification times once per dev package
2021-11-29 09:34:23 -08:00
Harmen Stoppels
037ece674b
distro: don't use deprecated linux_distribution (#27658) 2021-11-29 16:26:19 +01:00
Massimiliano Culpo
0d10408a25
bootstrap: restrict patchelf to v0.13.x (#27685) 2021-11-29 15:41:25 +01:00
Paul Ferrell
c0edb17b93
Handle byte sequences which are not encoded as UTF8 while logging. (#21447)
Fix builds which produce a lines with non-UTF8 output while logging
The alternative is to read in binary mode, and then decode while
ignoring errors.
2021-11-29 13:27:02 +01:00
Massimiliano Culpo
a96f2f603b
Bootstrap patchelf like GnuPG (#27532)
Remove a custom bootstrapping procedure to
use spack.bootstrap instead

Modifications:
* Reference count the bootstrap context manager
* Avoid SpackCommand to make the bootstrapping
  procedure more transparent
* Put back requirement on patchelf being in PATH for unit tests
* Add an e2e test to check bootstrapping patchelf
2021-11-26 15:32:13 +01:00
Maxim Belkin
6e095a9741
module_file_support: update format for configuration (#27598) 2021-11-25 08:41:32 +01:00
Massimiliano Culpo
270ba10962
spack flake8: remove deprecated command (#27290)
The "spack flake8" command wwas deprecated in favor
of "spack style". The deprecation wwarning is in the
0.17.X series, so removing it for v0.18.x
2021-11-24 14:20:11 -08:00
Robert Cohn
76ad803f25
intel-oneapi-mkl: add cluster libs and option for static linking (#26256) 2021-11-24 11:04:05 -08:00
Harmen Stoppels
dbf67d912c
Make patchelf test use the realpath (#27618)
I think this test should be removed, but when it stays, it should at
least follow the symlink, cause it fails for me if I let spack build
patchelf and have a symlink in a view.
2021-11-24 11:17:23 +01:00
Massimiliano Culpo
5c3dfacdc5
Update distro to v1.6.0 (#27263) 2021-11-24 10:10:11 +00:00
Massimiliano Culpo
70d5d234db
Update Jinja2 to v2.11.3 and MarkupSafe to v1.1.1 (#27264) 2021-11-24 10:21:35 +01:00
Massimiliano Culpo
12da0a9a69
Update six to v1.16.0 (#27265) 2021-11-24 10:20:04 +01:00
Harmen Stoppels
cced832cac
Fix leaky tests (#27616)
* fix: cc.py should use a function not session scope
* fix: don't let build env vars leak to other tests
* fix: don't leak build env in dev_build test
2021-11-23 14:10:48 -08:00
Massimiliano Culpo
fa7189b480
Remove support for Python 2.6 (#27256)
Modifications:
- [x] Removed `centos:6` unit test, adjusted vermin checks
- [x] Removed backport of `collections.OrderedDict`
- [x] Removed backport of `functools.total_ordering`
- [x] Removed Python 2.6 specific skip markers in unit tests
- [x] Fixed a few minor Python 2.6 related TODOs in code

Updating the vendored dependencies will be done in separate PRs
2021-11-23 09:06:17 -08:00
Nathan Hanford
2104f1273a
bugfix: Allow legacy tests to be read after hash break (#26078)
* added a test case

Co-authored-by: Nathan Hanford <hanford1@llnl.gov>
2021-11-22 21:49:41 -07:00
Piotr Luszczek
e90d5ad6cf
Intel packages: add support for LLVM OpenMP (#26517) 2021-11-22 13:41:43 -08:00
Massimiliano Culpo
5eba5dc271
Make CUDA and ROCm architecture conditional (#27185)
* Make CUDA and ROCm architecture conditional

fixes #14337

The variant to specify which architecture to use
for CUDA and ROCm are now conditional on +cuda and
+rocm respectively.

* cp2k: make all CUDA related variants conditional on +cuda
2021-11-22 07:54:19 -05:00
Harmen Stoppels
0024e5cc9b
Make _enable_or_disable(...) return an empty array for conditional variants whose condition is not met (#27504) 2021-11-22 10:47:09 +01:00
Joseph Snyder
d759612523
Add connection specification to mirror creation (#24244)
* Add connection specification to mirror creation

This allows each mirror to contain information about the credentials
used to access it.

Update command and tests based on comments

Switch to only "long form" flags for the s3 connection information.
Use the "any" function instead of checking for an empty list when looking
for s3 connection information.

Split test to use the access token separately from the access id and key.
Use long flag form in test.

Add endpoint_url to available S3 options.

Extend the special parameters for an S3 mirror to accept the
endpoint_url parameter.

Add a test.

* Add connection information per URL not per mirror

Expand the mirror-based connection information to be per-URL.
This will allow a user to specify different S3 connection information
for both the fetch and the push URLs.

Add a parameter for "profile", another way of storing the id/secret pair.

* Switch from "access_profile" to "profile"
2021-11-19 15:28:34 -05:00
Harmen Stoppels
c5aee4d9b4
define_from_variant: return an empty string for non-existing variants (#27503)
This permits to use conditional variants without a lot of boilerplate.
2021-11-19 14:10:00 +01:00
Michael Davis
3375db12a5
Adding --reuse to dev-build command. (#27487) 2021-11-19 09:25:45 +01:00
Massimiliano Culpo
57d3b02800
Remove spurious debug print (#27541) 2021-11-19 09:02:22 +01:00
Massimiliano Culpo
e3cd91af53
Refactor bootstrap of "spack style" dependencies (#27485)
Remove the "get_executable" function from the
spack.bootstrap module. Now "flake8", "isort",
"mypy" and "black" will use the same
bootstrapping method as GnuPG.
2021-11-18 15:23:09 +01:00
Massimiliano Culpo
f981682bdc
Allow recent pytest versions to be used with Spack (#25371)
Currently Spack vendors `pytest` at a version which is three major 
versions behind the latest (3.2.5 vs. 6.2.4). We do that since v3.2.5 
is the latest version supporting Python 2.6. Remaining so much 
behind the currently supported versions though might introduce 
some incompatibilities and is surely a technical debt.

This PR modifies Spack to:
- Use the vendored `pytest@3.2.5` only as a fallback solution, 
  if the Python interpreter used for Spack doesn't provide a newer one
- Be able to parse `pytest --collect-only` in all the different output 
  formats from v3.2.5 to v6.2.4 and use it consistently for `spack unit-test --list-*`
- Updating the unit tests in Github Actions to use a more recent `pytest` version
2021-11-18 15:08:59 +01:00
Harmen Stoppels
8f7640dbef
ci: run style unit tests only if we target develop (#27472)
Some tests assume the base branch is develop, but this branch may not
have been checked out.
2021-11-18 13:00:39 +01:00
Harmen Stoppels
cc62689504
Fix overly generic exceptions in log parser (#27413)
This type of error is skipped:

make[1]: *** [Makefile:222: /tmp/user/spack-stage/.../spack-src/usr/lib/julia/libopenblas64_.so.so] Error 1

but it's useful to have it, especially when a package sets a variable
incorrectly in makefiles
2021-11-17 11:24:14 +01:00
Robert Cohn
67cba372e8
Intel mpi: allow use of external libfabric (#27292)
Intel mpi comes with an installation of libfabric (which it needs as a
dependency). It can use other implementations of libfabric at runtime
though, so if you install a package that depends on `mpi` and
`libfabric`, you can specify `intel-mpi+external-libfabric` and ensure
that the Spack-built instance is used (both by `intel-mpi` and the
root).

Apply analogous change to intel-oneapi-mpi.
2021-11-16 12:55:24 -08:00
Harmen Stoppels
21308eb5cc
Turn some verbose messages into debug messages (#27408) 2021-11-15 11:21:37 +01:00
Seth R. Johnson
a04cc4470e
Add PyPI docs and warning in auto-generated package (#27404)
* docs: Add cross-references for pypi setup

* create: add warning for missing pypi
2021-11-12 10:58:44 -06:00
Harmen Stoppels
9637ed05f5
Fix overloaded argparse keys (#27379)
Commands should not reuse option names defined in main.
2021-11-11 23:34:18 -08:00
Massimiliano Culpo
b16bfe4f5f
spack tutorial: fix output to screen (#27316)
Spack was checking out v0.17, the output reported v0.16
2021-11-09 15:10:22 -08:00
Maxim Belkin
0ab5d42bd5
Fix typos (ouptut) (#27317) 2021-11-09 16:11:34 -06:00
Jordan Galby
6e9c0a8155
Fix log-format reporter ignoring install errors (#25961)
When running `spack install --log-format junit|cdash ...`, install
errors were ignored. This made spack continue building dependents of
failed install, ignoring `--fail-fast`, and exit 0 at the end.
2021-11-09 15:47:32 +00:00
Dylan Simon
2b990b400e
make --enable-locks actually enable locks (#24675) 2021-11-09 10:52:08 +00:00
Valentin Volkl
a3dd0e7861
build_environment: clean *_ROOT variables (#26474)
Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
2021-11-09 11:16:42 +01:00
iarspider
f164bae4a3
Python tests: skip importing weirdly-named modules (#27151)
* Python tests: allow importing weirdly-named modules

e.g. with dashes in name

* SIP tests: allow importing weirdly-named modules

* Skip modules with invalid names

* Changes from review

* Update from review

* Update from review

* Cleanup
2021-11-08 19:22:33 +00:00
Todd Gamblin
0e3d0516e6 bump version number to 0.17.0 2021-11-05 17:11:37 -07:00
Massimiliano Culpo
0feb5ec70a
Prevent additional properties to be in the answer set when reusing specs (#27240)
* Prevent additional properties to be in the answer set when reusing specs

fixes #27237

The mechanism to reuse concrete specs relies on imposing
the set of constraints stemming from the concrete spec
being reused.

We also need to prevent that other constraints get added
to this set.
2021-11-05 10:52:44 -06:00
Harmen Stoppels
8bb5ed8464
make version docs reflect reality (#27149)
* make version docs reflect reality

* typo and make things

* 2.6 -> 2.7 in example
2021-11-05 09:39:31 +00:00
Todd Gamblin
e13e697067
commands: spack load --list alias for spack find --loaded (#27184)
See #25249 and https://github.com/spack/spack/pull/27159#issuecomment-958163679.
This adds `spack load --list` as an alias for `spack find --loaded`.  The new command is
not as powerful as `spack find --loaded`, as you can't combine it with all the queries or
formats that `spack find` provides.  However, it is more intuitively located in the command
structure in that it appears in the output of `spack load --help`.

The idea here is that people can use `spack load --list`  for simple stuff but fall back to
`spack find --loaded` if they need more.

- add help to `spack load --list` that references `spack find`
- factor some parts of `spack find` out to be called from `spack load`
- add shell tests
- update docs

Co-authored-by: Peter Josef Scheibel <scheibel1@llnl.gov>
Co-authored-by: Richarda Butler <39577672+RikkiButler20@users.noreply.github.com>
2021-11-05 00:58:29 -07:00
Todd Gamblin
8e76244266 docs for experimental --reuse argument to spack install
Add docs for `--reuse`, along with a warning that it will likely be
removed and refactored.
2021-11-05 00:15:47 -07:00
Gregory Becker
5efa76a033 error message for reusing multiple hashes for package 2021-11-05 00:15:47 -07:00
Todd Gamblin
ac1e05fe1b concretizer: add error messages and simplify asp.py 2021-11-05 00:15:47 -07:00
Massimiliano Culpo
0186f0f955 Fix logic program for multi-valued variants
Reformulate variant rules so that we minimize both

1. The number of non-default values being used
2. The number of default values not-being used

This is crucial for MV variants where we may have
more than one default value
2021-11-05 00:15:47 -07:00
Todd Gamblin
e0c3d074c0 bugfix: handle hashes that only exist in input specs
In our tests, we use concrete specs generated from mock packages,
which *only* occur as inputs to the solver. This fixes two problems:

1. We weren't previously adding facts to encode the necessary
   `depends_on()` relationships, and specs were unsatisfiable on
   reachability.

2. Our hash lookup for reconstructing the DAG does not
   consider that a hash may have come from the inputs.
2021-11-05 00:15:47 -07:00