Commit graph

23547 commits

Author SHA1 Message Date
Massimiliano Culpo
290f57c779 spack spec: add --reuse argument 2021-11-05 00:15:47 -07:00
Todd Gamblin
652fa663b5 concretizer: get rid of last maximize directive in concretize.lp
- [x] Get rid of forgotten maximize directive.
- [x] Simplify variant handling
- [x] Fix bug in treatment of defaults on externals (don't count
      non-default variants on externals against them)
2021-11-05 00:15:47 -07:00
Massimiliano Culpo
da57b8775f Update command completion 2021-11-05 00:15:47 -07:00
Massimiliano Culpo
0d74a4f46e Trim dependencies on externals 2021-11-05 00:15:47 -07:00
Massimiliano Culpo
0b80035eaa Fix a unit test to match the new OS semantics
CNL, debian6 and Suse are not compatible
2021-11-05 00:15:47 -07:00
Massimiliano Culpo
4d25fc0068 ASP-based solve: if an OS is set, respect the value 2021-11-05 00:15:47 -07:00
Massimiliano Culpo
6e297b9ba1 Fix a type in "variant_not_default" rule 2021-11-05 00:15:47 -07:00
Todd Gamblin
ace4586bf8 concretizer: rework spack solve output to handle reuse better 2021-11-05 00:15:47 -07:00
Todd Gamblin
c537785f6f spec: ensure_valid_variants() should not validate concrete specs
Variants in concrete specs are "always" correct -- or at least we assume
them to be b/c they were concretized before. Their variants need not match
the current version of the package.
2021-11-05 00:15:47 -07:00
Todd Gamblin
b60a95cd5d concretizer: unify handling of single- and multi-valued variants
Multi-valued variants previously maximized default values to handle
cases where the default contained two values, e.g.:

    variant("foo", default="bar,baz")

This is because previously we were minimizing non-default values, and
`foo=bar`, `foo=baz`, and `foo=bar,baz` all had the same score, as
none of them had any "non-default" values.

This commit changes the approach and considers a non-default value
to be either a value set to something not default *or* the absence
of a default value from the set value.  This allows multi- and
single-valued variants to be handled the same way, with the same
minimization criterion.  It alse means that the "best" value for every
optimization criterion is now zero, which allows us to make useful
assumptions about the optimization criteria.
2021-11-05 00:15:47 -07:00
Todd Gamblin
b88da9d73d concretizer: reuse installs, but assign default values for new builds
Minimizing builds is tricky. We want a minimizing criterion because
we want to reuse the avaialble installs, but we also want things that
have to be built to stick to *default preferences* from the package
and from the user. We therefore treat built specs differently and
apply a different set of optimization criteria to them. Spack's *first*
priority is to reuse what it can, but if it builds something, the built
specs will respect defaults and preferences.

This is implemented by bumping the priority of optimization criteria
for built specs -- so that they take precedence over the otherwise
topmost-priority criterion to reuse what is installed.

The scheme relies on all of our optimization criteria being minimizations.
That is, we need the case where all specs are reused to be better than
any built spec could be. Basically, if nothing is built, all the build
criteria are zero (the best possible) and the number of built packages
dominates. If something *has* to be built, it must be strictly worse
than full reuse, because:

  1. it increases the number of built specs
  2. it must have either zero or some positive number for all criteria

Our optimziation criteria effectively sum into two buckets at once to
accomplish this. We use a `build_priority()` number to shift the
priority of optimization criteria for built specs higher.
2021-11-05 00:15:47 -07:00
Todd Gamblin
cfb60ab9e1 tests: make spack diff test more lenient
The constraints in the `spack diff` test were very specific and assumed
a lot about the structure of what was being diffed. Relax them a bit to
make them more resilient to changes.
2021-11-05 00:15:47 -07:00
Todd Gamblin
9eb94be6dd concretizer: only minimize builds when --reuse is enabled.
Make the first minimization conditional on whether `--reuse` is enabled in the solve.
If `--reuse` is not enabled, there will be nothing in the set to minimize and the
objective function (for this criterion) will be 0 for every answer set.
2021-11-05 00:15:47 -07:00
Todd Gamblin
40b914503e concretizer: adjust integrity constraints to only apply to builds.
Many of the integrity constraints in the concretizer are there to restrict how solves are done, but
they ignore that past solves may have had different initial conditions. For example, for things
we're building, we want the allowed variants to be restricted to those currently in Spack packages,
but if we are reusing a concrete spec, we need to be flexible about names that may have existed in
old packages.

Similarly, restrictions around compatibility of OS's, compiler versions, compiler OS support, etc.
are really only about what is supported by the *current* set of compilers/build tools known to
Spack, not about what we may get from concrete specs.

- [x] restrict certain integrity constraints to only apply to packages that we need to build, and
      omit concrete specs from consideration.
2021-11-05 00:15:47 -07:00
Todd Gamblin
2c142f9dd4 concretizer: rework operating system semantics for installed packages
The OS logic in the concretizer is still the way it was in the first version.
Defaults are implemented in a fairly inflexible way using straight logic. Most
of the other sections have been reworked to leave these kinds of decisions to
optimization. This commit does that for OS's as well.

As with targets, we optimize for target matches. We also try to optimize for
OS matches between nodes. Additionally, this commit adds the notion of
"OS compatibility" where we allow for builds to depend on binaries for certain
other OS's. e.g, for macos, a bigsur build can depend on an already installed
(concrete) catalina build. One cool thing about this is that we can declare
additional compatible OS's later, e.g. CentOS and RHEL.
2021-11-05 00:15:47 -07:00
Todd Gamblin
9c70d51a4f concretizer: impose() for concrete specs should use body facts.
The concretizer doesn't get a say in whether constraints from
concrete specs are imposed, so use body facts for them.
2021-11-05 00:15:47 -07:00
Todd Gamblin
3866b3e7d3 include installed hashes in solve and optimize for reuse 2021-11-05 00:15:47 -07:00
Todd Gamblin
7abe4ab309 rename checked_spec_clauses() to spec_clauses() 2021-11-05 00:15:47 -07:00
Todd Gamblin
ad5d632eeb add --reuse option to spack solve 2021-11-05 00:15:47 -07:00
Sinan
288176326c
new package: librttopo (#27182)
Co-authored-by: sbulut <sbulut@3vgeomatics.com>
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
2021-11-04 21:01:42 -06:00
Alec Scott
82b45d112f
[New Package] Add Restic v0.12.1 (#27208)
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
2021-11-04 17:12:34 -07:00
Massimiliano Culpo
839057e98d
Rename the temporary scope for bootstrap buildcache (#27231)
If we don't rename Spack will fail with:
```
ImportError: cannot bootstrap the "clingo" Python module from spec "clingo-bootstrap@spack+python %gcc target=x86_64" due to the following failures:
    'spack-install' raised ValueError: Invalid config scope: 'bootstrap'.  Must be one of odict_keys(['_builtin', 'defaults', 'defaults/cray', 'bootstrap/cray', 'disable_modules', 'overrides-0'])
    Please run `spack -d spec zlib` for more verbose error messages
```
in case bootstrapping from binaries fails and we are
falling back to bootstrapping from sources.
2021-11-04 16:17:00 -07:00
Ben Darwin
f2a42ac4c6
c3d: add new package (#27155) 2021-11-04 15:38:28 -07:00
Tamara Dahlgren
080f1872b8
Add the spack tutorial environment as a cloud pipeline stack (#27137) 2021-11-04 15:14:46 -07:00
Jerome Soumagne
c25a4ecfc2
libfabric: add 1.13.2 (#27202) 2021-11-04 15:04:40 -07:00
Massimiliano Culpo
79f754a968
Sort arguments lexicographically in command's help (#27196) 2021-11-04 12:41:58 -07:00
downloadico
084ce46107
exciting: add "oxygen" version, multiple fixes (#27217)
ensure that none of ^intel-mkl, ^intel-mpi, and ^mkl are used, unless
the compiler is intel.
Fix bad logic in the src/src_xs/m_makespectrum.f90 file in the oxygen version.
Add the -fallow-argument-mismatch for gcc >= 10.
2021-11-04 12:24:27 -07:00
Cameron Smith
7542d8adc8
omega-h: add support for stand-alone testing (#26931)
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
2021-11-04 12:02:02 -07:00
Joe Schoonover
4cc2adcbc3
Add new versions for HOHQMesh and switch to tar-ball releases (#27194)
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
2021-11-04 11:54:02 -07:00
Jon Rood
ed3d459a83
Add RelWithDebInfo to protobuf build_type list. (#27220) 2021-11-04 10:16:44 -07:00
Andre Merzky
f6ed8b6150
Feature/exaworks spack (#27216)
* add exaworks spack meta package

* add maintainer

* switch to `BundlePackage`

* flake8

* another flake8 fix

* remove incorrect dependency
2021-11-04 06:07:45 -06:00
Massimiliano Culpo
1dcabdbc8d
mfem, hpx: fix recipes after conditional variants (#27215) 2021-11-04 03:52:38 -06:00
Seth R. Johnson
503576c017
kokkos: refactor defines to use helper functions (#27189) 2021-11-04 09:31:29 +01:00
Erik Schnetter
6a7b07c60b
ssht: New version 1.5.1 (#27173) 2021-11-03 22:10:56 -06:00
Cameron Stanavige
7102e295b9
scr: 3.0rc2 release, variants and deps updates (#27178)
* scr: 3.0rc2 release, variants and deps updates

This adds 3.0rc2 release for end users to aid in testing scr for
upcoming 3.0 release.

Included in this change:
- Require most recent component versions for this release
- Add a variant for PDSH as it is now an optional dependency with
this release
- Add bbapi and datawarp (dw) variants
- bbapi_fallback variant now requires bbapi variant with latest
release
- Add variants to enable/disable examples and tests
- Add shared variant and current conflicts with ~shared
- Update cmake_args to account for added variants where needed

Additional updates:
- Add maintainers
- Use lists and for loops to clean up repetitive code involving all
components
- Use self.define and self.define_from_variant to clean up cmake_args
- Use consistent quoting throughout package

* Un-deprecate v2 and legacy

* Use new conditional variants
2021-11-03 20:06:02 -07:00
Erik Schnetter
b9cdaa5429
simulationio package: add variants asdf, hdf5, rnpl, silo (#27172)
Define new variants asdf, hdf5, rnpl, silo to allow disabling or
enabling dependencies.
2021-11-03 18:07:35 -07:00
Erik Schnetter
0d30799bef
hwloc: New version 2.6.0 (#27170) 2021-11-03 15:45:38 -07:00
Erik Schnetter
4045ab45de
mpitrampoline: New version 2.2.0 (#27171) 2021-11-03 15:42:14 -07:00
Erik Schnetter
91a6e38404
shtools: disable libtool, add 4.9.1, fix --test=root (#27014)
The Makefile expects the "other" libtool, not the GNU libtool we have in Spack.
Closes https://github.com/spack/spack/issues/26993
Co-authored-by: Bernhard Kaindl <bernhardkaindl7@gmail.com>
2021-11-03 23:33:52 +01:00
Thomas Helfer
c0a81399bf
tfel and mgis: add new versions and fix tests (#27011)
Co-authored-by: Bernhard Kaindl <bernhardkaindl7@gmail.com>
2021-11-03 23:24:28 +01:00
Andrew W Elble
7b3e02471b
new package: telegraf (#27201) 2021-11-03 15:00:33 -07:00
David Boehme
47babb2518
Add Caliper v2.7.0 (#27197) 2021-11-03 13:22:43 -06:00
Seth R. Johnson
12a0738030
trilinos: mark conflicts in @:13.1+tpetra^cuda@11 (#27188)
* trilinos: fix @13.0.1+tpetra^cuda@11
* Mark CUDA conflict with old versions and always define TPL
* trilinos: patch doesn't build so just mark as conflict
2021-11-03 12:38:03 -06:00
Axel Huebl
e5a9beed28
WarpX: 21.11 (#27158)
Update `warpx` & `py-warpx` to the latest release, `21.11`.
2021-11-03 11:18:23 -07:00
Kyle Gerheiser
25f1aad1c8
nemsio package: add version 2.5.4; add option to build without MPI (#27030)
Version 2.5.4 adds an option which allows Nemsio to be built without MPI
2021-11-03 10:18:17 -07:00
Manuela Kuhn
09eb79b571
r-reprex: add 2.0.1 (#27174) 2021-11-03 12:05:28 -05:00
Satish Balay
346f3652c6
petsc, py-petsc4py: add versions 3.16.1 (#27152) 2021-11-03 10:52:35 -05:00
Manuela Kuhn
78b1512966
py-datalad: add 0.15.3 (#27193) 2021-11-03 09:49:46 -06:00
iarspider
dede8c9d6b
New version: py-jupyterlab-pygments 0.1.2 (#27186) 2021-11-03 10:27:55 -05:00
Manuela Kuhn
8e4d5a0922
sip: fix python_include_dir (#26953) 2021-11-03 10:27:04 -05:00