Commit graph

5114 commits

Author SHA1 Message Date
Massimiliano Culpo
5c3dfacdc5
Update distro to v1.6.0 (#27263) 2021-11-24 10:10:11 +00:00
Massimiliano Culpo
70d5d234db
Update Jinja2 to v2.11.3 and MarkupSafe to v1.1.1 (#27264) 2021-11-24 10:21:35 +01:00
Massimiliano Culpo
12da0a9a69
Update six to v1.16.0 (#27265) 2021-11-24 10:20:04 +01:00
Harmen Stoppels
cced832cac
Fix leaky tests (#27616)
* fix: cc.py should use a function not session scope
* fix: don't let build env vars leak to other tests
* fix: don't leak build env in dev_build test
2021-11-23 14:10:48 -08:00
Massimiliano Culpo
fa7189b480
Remove support for Python 2.6 (#27256)
Modifications:
- [x] Removed `centos:6` unit test, adjusted vermin checks
- [x] Removed backport of `collections.OrderedDict`
- [x] Removed backport of `functools.total_ordering`
- [x] Removed Python 2.6 specific skip markers in unit tests
- [x] Fixed a few minor Python 2.6 related TODOs in code

Updating the vendored dependencies will be done in separate PRs
2021-11-23 09:06:17 -08:00
Nathan Hanford
2104f1273a
bugfix: Allow legacy tests to be read after hash break (#26078)
* added a test case

Co-authored-by: Nathan Hanford <hanford1@llnl.gov>
2021-11-22 21:49:41 -07:00
Piotr Luszczek
e90d5ad6cf
Intel packages: add support for LLVM OpenMP (#26517) 2021-11-22 13:41:43 -08:00
Massimiliano Culpo
5eba5dc271
Make CUDA and ROCm architecture conditional (#27185)
* Make CUDA and ROCm architecture conditional

fixes #14337

The variant to specify which architecture to use
for CUDA and ROCm are now conditional on +cuda and
+rocm respectively.

* cp2k: make all CUDA related variants conditional on +cuda
2021-11-22 07:54:19 -05:00
Harmen Stoppels
0024e5cc9b
Make _enable_or_disable(...) return an empty array for conditional variants whose condition is not met (#27504) 2021-11-22 10:47:09 +01:00
Joseph Snyder
d759612523
Add connection specification to mirror creation (#24244)
* Add connection specification to mirror creation

This allows each mirror to contain information about the credentials
used to access it.

Update command and tests based on comments

Switch to only "long form" flags for the s3 connection information.
Use the "any" function instead of checking for an empty list when looking
for s3 connection information.

Split test to use the access token separately from the access id and key.
Use long flag form in test.

Add endpoint_url to available S3 options.

Extend the special parameters for an S3 mirror to accept the
endpoint_url parameter.

Add a test.

* Add connection information per URL not per mirror

Expand the mirror-based connection information to be per-URL.
This will allow a user to specify different S3 connection information
for both the fetch and the push URLs.

Add a parameter for "profile", another way of storing the id/secret pair.

* Switch from "access_profile" to "profile"
2021-11-19 15:28:34 -05:00
Harmen Stoppels
c5aee4d9b4
define_from_variant: return an empty string for non-existing variants (#27503)
This permits to use conditional variants without a lot of boilerplate.
2021-11-19 14:10:00 +01:00
Michael Davis
3375db12a5
Adding --reuse to dev-build command. (#27487) 2021-11-19 09:25:45 +01:00
Massimiliano Culpo
57d3b02800
Remove spurious debug print (#27541) 2021-11-19 09:02:22 +01:00
Massimiliano Culpo
e3cd91af53
Refactor bootstrap of "spack style" dependencies (#27485)
Remove the "get_executable" function from the
spack.bootstrap module. Now "flake8", "isort",
"mypy" and "black" will use the same
bootstrapping method as GnuPG.
2021-11-18 15:23:09 +01:00
Massimiliano Culpo
f981682bdc
Allow recent pytest versions to be used with Spack (#25371)
Currently Spack vendors `pytest` at a version which is three major 
versions behind the latest (3.2.5 vs. 6.2.4). We do that since v3.2.5 
is the latest version supporting Python 2.6. Remaining so much 
behind the currently supported versions though might introduce 
some incompatibilities and is surely a technical debt.

This PR modifies Spack to:
- Use the vendored `pytest@3.2.5` only as a fallback solution, 
  if the Python interpreter used for Spack doesn't provide a newer one
- Be able to parse `pytest --collect-only` in all the different output 
  formats from v3.2.5 to v6.2.4 and use it consistently for `spack unit-test --list-*`
- Updating the unit tests in Github Actions to use a more recent `pytest` version
2021-11-18 15:08:59 +01:00
Harmen Stoppels
8f7640dbef
ci: run style unit tests only if we target develop (#27472)
Some tests assume the base branch is develop, but this branch may not
have been checked out.
2021-11-18 13:00:39 +01:00
Harmen Stoppels
cc62689504
Fix overly generic exceptions in log parser (#27413)
This type of error is skipped:

make[1]: *** [Makefile:222: /tmp/user/spack-stage/.../spack-src/usr/lib/julia/libopenblas64_.so.so] Error 1

but it's useful to have it, especially when a package sets a variable
incorrectly in makefiles
2021-11-17 11:24:14 +01:00
Robert Cohn
67cba372e8
Intel mpi: allow use of external libfabric (#27292)
Intel mpi comes with an installation of libfabric (which it needs as a
dependency). It can use other implementations of libfabric at runtime
though, so if you install a package that depends on `mpi` and
`libfabric`, you can specify `intel-mpi+external-libfabric` and ensure
that the Spack-built instance is used (both by `intel-mpi` and the
root).

Apply analogous change to intel-oneapi-mpi.
2021-11-16 12:55:24 -08:00
Harmen Stoppels
21308eb5cc
Turn some verbose messages into debug messages (#27408) 2021-11-15 11:21:37 +01:00
Seth R. Johnson
a04cc4470e
Add PyPI docs and warning in auto-generated package (#27404)
* docs: Add cross-references for pypi setup

* create: add warning for missing pypi
2021-11-12 10:58:44 -06:00
Harmen Stoppels
9637ed05f5
Fix overloaded argparse keys (#27379)
Commands should not reuse option names defined in main.
2021-11-11 23:34:18 -08:00
Massimiliano Culpo
b16bfe4f5f
spack tutorial: fix output to screen (#27316)
Spack was checking out v0.17, the output reported v0.16
2021-11-09 15:10:22 -08:00
Maxim Belkin
0ab5d42bd5
Fix typos (ouptut) (#27317) 2021-11-09 16:11:34 -06:00
Jordan Galby
6e9c0a8155
Fix log-format reporter ignoring install errors (#25961)
When running `spack install --log-format junit|cdash ...`, install
errors were ignored. This made spack continue building dependents of
failed install, ignoring `--fail-fast`, and exit 0 at the end.
2021-11-09 15:47:32 +00:00
Dylan Simon
2b990b400e
make --enable-locks actually enable locks (#24675) 2021-11-09 10:52:08 +00:00
Valentin Volkl
a3dd0e7861
build_environment: clean *_ROOT variables (#26474)
Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
2021-11-09 11:16:42 +01:00
iarspider
f164bae4a3
Python tests: skip importing weirdly-named modules (#27151)
* Python tests: allow importing weirdly-named modules

e.g. with dashes in name

* SIP tests: allow importing weirdly-named modules

* Skip modules with invalid names

* Changes from review

* Update from review

* Update from review

* Cleanup
2021-11-08 19:22:33 +00:00
Todd Gamblin
0e3d0516e6 bump version number to 0.17.0 2021-11-05 17:11:37 -07:00
Massimiliano Culpo
0feb5ec70a
Prevent additional properties to be in the answer set when reusing specs (#27240)
* Prevent additional properties to be in the answer set when reusing specs

fixes #27237

The mechanism to reuse concrete specs relies on imposing
the set of constraints stemming from the concrete spec
being reused.

We also need to prevent that other constraints get added
to this set.
2021-11-05 10:52:44 -06:00
Harmen Stoppels
8bb5ed8464
make version docs reflect reality (#27149)
* make version docs reflect reality

* typo and make things

* 2.6 -> 2.7 in example
2021-11-05 09:39:31 +00:00
Todd Gamblin
e13e697067
commands: spack load --list alias for spack find --loaded (#27184)
See #25249 and https://github.com/spack/spack/pull/27159#issuecomment-958163679.
This adds `spack load --list` as an alias for `spack find --loaded`.  The new command is
not as powerful as `spack find --loaded`, as you can't combine it with all the queries or
formats that `spack find` provides.  However, it is more intuitively located in the command
structure in that it appears in the output of `spack load --help`.

The idea here is that people can use `spack load --list`  for simple stuff but fall back to
`spack find --loaded` if they need more.

- add help to `spack load --list` that references `spack find`
- factor some parts of `spack find` out to be called from `spack load`
- add shell tests
- update docs

Co-authored-by: Peter Josef Scheibel <scheibel1@llnl.gov>
Co-authored-by: Richarda Butler <39577672+RikkiButler20@users.noreply.github.com>
2021-11-05 00:58:29 -07:00
Todd Gamblin
8e76244266 docs for experimental --reuse argument to spack install
Add docs for `--reuse`, along with a warning that it will likely be
removed and refactored.
2021-11-05 00:15:47 -07:00
Gregory Becker
5efa76a033 error message for reusing multiple hashes for package 2021-11-05 00:15:47 -07:00
Todd Gamblin
ac1e05fe1b concretizer: add error messages and simplify asp.py 2021-11-05 00:15:47 -07:00
Massimiliano Culpo
0186f0f955 Fix logic program for multi-valued variants
Reformulate variant rules so that we minimize both

1. The number of non-default values being used
2. The number of default values not-being used

This is crucial for MV variants where we may have
more than one default value
2021-11-05 00:15:47 -07:00
Todd Gamblin
e0c3d074c0 bugfix: handle hashes that only exist in input specs
In our tests, we use concrete specs generated from mock packages,
which *only* occur as inputs to the solver. This fixes two problems:

1. We weren't previously adding facts to encode the necessary
   `depends_on()` relationships, and specs were unsatisfiable on
   reachability.

2. Our hash lookup for reconstructing the DAG does not
   consider that a hash may have come from the inputs.
2021-11-05 00:15:47 -07:00
Todd Gamblin
a4a2ed3c34 concretizer: exempt already-installed specs from compiler and variant rules
Concrete specs that are already installed or that come from a buildcache
may have compilers and variant settings that we do not recognize, but that
shouldn't prevent reuse (at least not until we have a more detailed compiler
model).

- [x] make sure compiler and variant consistency rules only apply to
      built specs
- [x] don't validate concrete specs on input, either -- they're concrete
      and we shouldn't apply today's rules to yesterday's build
2021-11-05 00:15:47 -07:00
Todd Gamblin
49ed41b028 spack diff: more flexible tests, restore transitive diff with spec_clauses
In switching to hash facts for concrete specs, we lost the transitive facts
from dependencies. This was fine for solves, because they were implied by
the imposed constraints from every hash. However, for `spack diff`, we want
to see what the hashes mean, so we need another mode for `spec_clauses()` to
show that.

This adds a `expand_hashes` argument to `spec_clauses()` that allows us to
output *both* the hashes and their implications on dependencies. We use
this mode in `spack diff`.
2021-11-05 00:15:47 -07:00
Massimiliano Culpo
3e3e84ba30 Add a missing definition in the logic program 2021-11-05 00:15:47 -07:00
Massimiliano Culpo
be2cf16b67 Add buildcache to reusable specs 2021-11-05 00:15:47 -07:00
Massimiliano Culpo
31dfad9c16 spack install: add --reuse argument 2021-11-05 00:15:47 -07:00
Massimiliano Culpo
e2744fafa1 spack concretize: add --reuse argument 2021-11-05 00:15:47 -07:00
Massimiliano Culpo
290f57c779 spack spec: add --reuse argument 2021-11-05 00:15:47 -07:00
Todd Gamblin
652fa663b5 concretizer: get rid of last maximize directive in concretize.lp
- [x] Get rid of forgotten maximize directive.
- [x] Simplify variant handling
- [x] Fix bug in treatment of defaults on externals (don't count
      non-default variants on externals against them)
2021-11-05 00:15:47 -07:00
Massimiliano Culpo
0d74a4f46e Trim dependencies on externals 2021-11-05 00:15:47 -07:00
Massimiliano Culpo
0b80035eaa Fix a unit test to match the new OS semantics
CNL, debian6 and Suse are not compatible
2021-11-05 00:15:47 -07:00
Massimiliano Culpo
4d25fc0068 ASP-based solve: if an OS is set, respect the value 2021-11-05 00:15:47 -07:00
Massimiliano Culpo
6e297b9ba1 Fix a type in "variant_not_default" rule 2021-11-05 00:15:47 -07:00
Todd Gamblin
ace4586bf8 concretizer: rework spack solve output to handle reuse better 2021-11-05 00:15:47 -07:00
Todd Gamblin
c537785f6f spec: ensure_valid_variants() should not validate concrete specs
Variants in concrete specs are "always" correct -- or at least we assume
them to be b/c they were concretized before. Their variants need not match
the current version of the package.
2021-11-05 00:15:47 -07:00