Commit graph

26041 commits

Author SHA1 Message Date
Scott Wittenburg
c202953528 gitlab ci: Docstring methods or make them private 2022-05-13 10:45:12 -07:00
Scott Wittenburg
be0e3f4458 binary_distribution: Refactor index generation into smaller methods 2022-05-13 10:45:12 -07:00
Scott Wittenburg
fd3bb5177b env: Use order of roots to resolve DAG hash conflicts in legacy lockfiles 2022-05-13 10:45:12 -07:00
Scott Wittenburg
9de61c0197 env: enforce predictable ordering when reading lockfile
Without some enforcement of spec ordering, python 2 produced
different results in the affected test than did python 3.  This
change makes the arbitrary but reproducible decision to sort
the specs by their lockfile key alphabetically.
2022-05-13 10:45:12 -07:00
Scott Wittenburg
84cfb3b7fe spec: fix infinite recursion when computing package hash
Issue described in the following PR comment:

https://github.com/spack/spack/pull/28504#issuecomment-1051835568

Solution described in subsequent comment:

https://github.com/spack/spack/pull/28504#issuecomment-1053986132
2022-05-13 10:45:12 -07:00
Scott Wittenburg
cb0d12b9d5 Fix how environments are read from lockfile 2022-05-13 10:45:12 -07:00
Scott Wittenburg
f6e7c0b740 hashes: remove full_hash and build_hash from spack 2022-05-13 10:45:12 -07:00
Scott Wittenburg
512645ff2e environment: key by dag_hash instead of build_hash 2022-05-13 10:45:12 -07:00
Scott Wittenburg
32a2c22b2b tests: fix failing test_hash_change
The full hash appears twice in the spec dict now, replacing just
the value replaces it under "hash" and "full_hash".  Only replace
the one that appears after "full_hash".

I'm actually not sure what purpose this test served, so maybe it
could be removed, as it may be testing some distinction between
full and dag hash which no longer exists.
2022-05-13 10:45:12 -07:00
Todd Gamblin
e02020c80a Include all deps and package content in the dag_hash()
For a long time, Spack has used a coarser hash to identify packages
than it likely should. Packages are identified by `dag_hash()`, which
includes only link and run dependencies. Build dependencies are
stripped before hashing, and we have notincluded hashes of build
artifacts or the `package.py` files used to build.  This means the
DAG hash actually doesn't represent all the things Spack can build,
and it reduces reproducibility.

We did this because, in the early days, users were (rightly) annoyed
when a new version of CMake, autotools, or some other build dependency
would necessitate a rebuild of their entire stack. Coarsening the hash
avoided this issue and enabled a modicum of stability when only reusing
packages by hash match.

Now that we have `--reuse`, we don't need to be so careful. Users can
avoid unnecessary rebuilds much more easily, and we can add more
provenance to the spec without worrying that frequent hash changes
will cause too many rebuilds.

This commit starts the refactor with the following major change:

- [x] Make `Spec.dag_hash()` include build, run, and link
      dependencides and the package hash (it is now equivalent to
      `full_hash()`).

It also adds a couple of bugfixes for problems discovered during
the switch:

- [x] Don't add a `package_hash()` in `to_node_dict()` unless
      the spec is concrete (fixes breaks on abstract specs)

- [x] Don't add source ids to the package hash for packages without
      a known fetch strategy (may mock packages are like this)

- [x] Change how `Spec.patches` is memoized. Using
      `llnl.util.lang.memoized` on `Spec` objects causes specs to
      be stored in a `dict`, which means they need a hash.  But,
      `dag_hash()` now includes patch `sha256`'s via the package
      hash, which can lead to infinite recursion
2022-05-13 10:45:12 -07:00
Massimiliano Culpo
d900ac2003
Reuse concretization by default (#30396)
* Enable reuse by default in Spack
* Update documentation to match new default
* Configure pipelines not to reuse software
2022-05-13 09:11:10 -07:00
Teodor Nikolov
faa277778e
sarus: fix dependency on boost (#30659)
The program_options variant is disabled by default in Boost and
has to be enabled explicitly.
2022-05-13 08:41:41 -06:00
Harmen Stoppels
b60d3dcd29
libtree: add v3.0.4, v3.1.1 (#30658) 2022-05-13 11:35:06 +00:00
Todd Gamblin
e0bed2d6a7
tutorial stack: allow deprecated versions (#30648)
For tutorial builds, we should continue to allow deprecated builds to be installed. We
can update them as needed when we update the tutorial, but we don't need to correct them
immediately on deprecation in CI.

- [x] add `deprecated:true` to tutorial `spack.yaml` config.
2022-05-13 04:09:39 -06:00
Massimiliano Culpo
745c191d73
Gitlab pipelines: add a small legend in the logs to interpret "[x]" (#30643) 2022-05-12 21:46:35 +02:00
Chuck Atkins
42e9430fbc
silo: Cleanup the HDF5 dependency specs (#30593) 2022-05-12 15:43:11 -04:00
Chuck Atkins
f11572166f
vtk: fix hdf5 version specification (#30592) 2022-05-12 21:16:57 +02:00
Adam J. Stewart
aa6665d5ee
py-scikit-learn: add v1.1.0 (#30649) 2022-05-12 12:50:26 -06:00
Simon Pintarelli
3e8f31a068
atompaw: add version 4.2.0.0 (#30650) 2022-05-12 18:09:30 +00:00
Massimiliano Culpo
3625ea4726
Revert "atompaw: add new package (#30619)" (#30647)
This reverts commit 12b0278f08.
2022-05-12 19:47:47 +02:00
snehring
245b95223d
soapdenovo-trans: add v1.0.5 (#30629) 2022-05-12 19:14:49 +02:00
Max Zeyen
e00c8a7d98
code-server: add new versions (#30646) 2022-05-12 10:08:02 -07:00
Adam J. Stewart
97792f04e9
py-netcdf4: add v1.5.8 (#30598) 2022-05-12 09:02:27 -05:00
Frédéric Simonis
ca069f6906
precice: add v2.4.0 (#30633) 2022-05-12 15:57:53 +02:00
Qian Jianhua
35a91bdd72
fakexrandr: remove 'ldconfig' from Makefile (#30632) 2022-05-12 14:55:45 +02:00
Mikael Simberg
c866a50446
Add patch for compiling llvm ~omp_as_runtime (#30583) 2022-05-12 05:49:39 -06:00
Harmen Stoppels
3033abb5bd
Add cuda 11.7 compat bounds for gcc/clang (#30639) 2022-05-12 10:41:30 +00:00
Harmen Stoppels
d37f439557
pigz: add v2.7 (#30640) 2022-05-12 10:39:08 +00:00
dlkuehn
18710936f1
platypus: limit install to Python2 (#30624)
Co-authored-by: David Kuehn <las_dkuehn@iastate.edu>
2022-05-12 09:30:45 +02:00
Simon Pintarelli
12b0278f08
atompaw: add new package (#30619) 2022-05-12 09:24:38 +02:00
Eureka Hwang
34fd6e36ce
cuda: add v11.7 (#30630) 2022-05-12 09:20:42 +02:00
marcosmazz
363536fd92
quantum-espresso: add gipaw variant (#30628)
Co-authored-by: Marcos Mazzini - <mmazzini@serafin.ccad.unc.edu.ar>
2022-05-11 18:31:19 -07:00
Abhik Sarkar
d57d343b6d
This commit removes the Boost.with_default_variants to variants (#28960)
that packages are precisely dependant upon. This is the second batch
of 20 packages with modified boost dependencies.
2022-05-11 17:16:19 -07:00
Tamara Dahlgren
1b254d19c4
Allow read-only access to file cache (when needed) (#29693)
* Allow read-only access to file cache (when needed)
* Tweaked and added unit tests
* Skip test_cache_init_entry_fails for windows
2022-05-11 16:25:06 -07:00
shanedsnyder
82b916be36
add darshan-3.4.0-pre1 release (#30569)
include new dependencies on autotools/m4/etc.
2022-05-11 16:21:56 -06:00
snehring
66d3648200
bamutil: updating to 1.0.15 (#30623)
* bamutil: updating to 1.0.15

* bamutil: switching to github archive
2022-05-11 22:12:32 +00:00
AMD Toolchain Support
d2fc7b9f7d
lammps: updating googletest version to 1.11 to avoid GTEST_DISALLOW_ASSIGN_Error (#30563)
* updating googletest version to 1.11 to avoid GTEST_DISALLOW_ASSIGN_ error

* limiting the version scope

* modified the version limit

Co-authored-by: mohan babu <mohbabul@amd.com>
2022-05-11 13:17:47 -06:00
Wouter Deconinck
1067749371
assimp: add v5.2.3 (#29629) 2022-05-11 20:03:07 +02:00
snehring
8bd893367d
sentieon-genomics: add version 202112.02, remove manual download (#30145) 2022-05-11 19:38:24 +02:00
Mikael Simberg
2ed542b744
Add conflict for ROCm and asio in HPX package (#30620) 2022-05-11 10:33:16 -07:00
Stephen Hudson
5cb7a5db45
libEnsemble: add v0.9.1 (#30621) 2022-05-11 16:41:45 +00:00
Cory Bloor
f84991b5a8
rocfft: add spack build test support (#29957)
* rocfft: add spack build test support

* Style cleanup for variant specs
2022-05-11 09:37:22 -07:00
Hadrien G
928ecd1f4e
[acts] Add version 19 (#30615)
No build system visible changes, so AFAIK this should Just Work
2022-05-11 09:13:27 -07:00
Thomas Dickerson
72e594fb10
Fix default buildcache location (#30230)
Resolve path/URL parsing issues introduced by #27021
2022-05-11 17:50:04 +02:00
Vasileios Karakasis
f9d701f9cf
ReFrame: add v3.11.0 (#30578) 2022-05-11 08:41:40 -06:00
Harmen Stoppels
63f7053fe8
eccodes: add v2.25.0 (#30618) 2022-05-11 12:04:43 +00:00
Harmen Stoppels
1c51d6313b
jasper: avoid --gc-sections / hidden symbols (#30617)
Jasper v3.x changed a default to hide hidden symbols, but apparently
eccodes relies on those symbols and fails to link otherwise.
2022-05-11 11:52:54 +00:00
Harmen Stoppels
c164e6fe03
jasper: update cmake dependency (#30616) 2022-05-11 13:33:46 +02:00
Gregory Lee
2d823dcf90
gobject-introspection: add libffi dependency (#30601) 2022-05-11 13:29:54 +02:00
Tom Scogland
a12c638224
Neovim luajit (#28855)
Upstream neovim builds with luajit-openresty or luajit in almost all
cases.  To support the current usage, a user can specify that they want
lua, but this will allow the use of the normal (faster, better tested
and better maintained) setup.
2022-05-11 05:01:35 -06:00