Adds four new sub-commands to the buildcache command:
1. save-yaml: Takes a root spec and a list of dependent spec names,
along with a directory in which to save yaml files, and writes out
the full spec.yaml for each of the dependent specs. This only needs
to concretize the root spec once, then indexes it with the names of
the dependent specs.
2. check: Checks a spec (via either an abstract spec or via a full
spec.yaml) against remote mirror to see if it needs to be rebuilt.
Comparies full_hash stored on remote mirror with full_hash computed
locally to determine whether spec needs to be rebuilt. Can also
generate list of specs to check against remote mirror by expanding
the set of release specs expressed in etc/spack/defaults/release.yaml.
3. get-buildcache-name: Makes it possible to attempt to read directly
the spec.yaml file on a remote or local mirror by providing the path
where the file should live based on concretizing the spec.
4. download: Downloads all buildcache files associated with a spec
on a remote mirror, including any .spack, .spec, and .cdashid files
that might exist. Puts the files into the local path provided on
the command line, and organizes them in the same hierarchy found on
the remote mirror
This commit also refactors lib/spack/spack/util/web.py to expose
functionality allowing other modules to read data from a url.
- add CombinatorialSpecSet in spack.util.spec_set module.
- class is iterable and encaspulated YAML parsing and validation.
- Adjust YAML format to be more generic
- YAML spec-set format now has a `matrix` section, which can contain
multiple lists of specs, generated different ways. Including:
- specs: a raw list of specs.
- packages: a list of package names and versions
- compilers: a list of compiler names and versions
- All of the elements of `matrix` are dimensions for the build matrix;
we take the cartesian product of these lists of specs to generate a
build matrix. This means we can add things like [^mpich, ^openmpi]
to get builds with different MPI versions. It also means we can
multiply the build matrix out with lots of different parameters.
- Add a schema format for spec-sets
On macOS, the build of libxxhash is performed by a bare `make`
launched by `MakePackage`s default `build` phase. As `prefix`
is not set at this point, the library gets compiled with a hard-coded
install name of `/usr/local/libxxhash.dylib`. Downstream clients,
in the tested case Root, will then fail to build as the install name
is incorrect.
Move setting of `prefix` from `install` to `edit` so that it
propagates to subsequent `build` and `install` phases.
* QE hdf5 is now a multi-valued variant. Update conflicts accordingly.
* Update dependencies for hdf5 multi-valued variant. Bug fix: high-level hdf5 library needs to be a hard dependecy.
* Support for HDF5 serial interface. Note that this is not supported in QE autoconf, but will be fixed in the next official release.
* Support in QMCPACK for CUDA_ARCH flag.
* No need to have explicit CUDA variant anymore.
* Internal QMCPACK CMake assumes a minimum GPU architecture if none is provided.
Fixes#10617Fixes#10624Closes: #10619#8136 dependended entirely on spec.libs to retrieve library directories
from dependencies. By default this function only retrieves libraries if
their name is something like lib<package> (e.g. "libfoo.so" for a
package called "Foo"). This unconditionally adds lib/lib64 directories
for each dependency as link/rpath directories.
This also filters system paths from link/rpaths/include directories and
removes duplicated paths that #8136 could add.
If the -f <specyamlfile> argument to install is used (rather than
providing package specs on the command line), CDash throws an exception
due to missing the installation command (the packages targeted for
install). This fixes that behavior so CDash reporting succeeds in
either case.
fixes#10601
Due to a bug this attribute is wrong for packages that use directories
as namespaces. For instance it will add "<boost-prefix>/include/boost"
instead of "<boost-prefix>/include" to the include path.
As a minor addition a few loops in the compiler wrappers have been
simplified.
* cp2k: archive the Makefile after a successful installation
* cp2k: all checksums are now sha256
* libint: all checksums are now sha256 + added versions 2.4.X
* libxc: all checksums are now sha256 + added version 4.3.2
* cp2k: os.path.join instead of join_path + simplified openmp flags
* cp2k: turned into a MakefilePackage
* cp2k: refactored edit method so that Makefile writing happens last
* icu4c: Add cxxstd variant
For versions supported in Spack, ICU4C defaults to the C++11.
Provide a variant to default to this, plus options for C++14
and 17 if dependees require for ABI compatibility.
* icu4c: cxxstd variant cannot be multi
* Updated Geant4 and dependencies clhep and vecgeom to use cxxstd= variant so that it is applied consistently.
* Make standalone geant4 data packages that are used when data is not installed with geant4. This allows installing the data files are externals that can be common to many compiler version.
* move geant4 data package dependencies to umbrella package geant4-data
Spack shell detection in setup-env.sh was originally based on
examining the executable name of $$ (from "ps"). In some cases this
does not actually give the name of the shell used, for example when
setup-env.sh was invoked from a script using "#!". To make shell
detection more robust, this adds a preliminary check for shell
variables which indicate that the shell is bash or zsh; the
executable name of $$ is retained as a fallback if those variables
are not defined.
Fixes#7855Closes#8070Closes#2645
When searching for library directories (e.g. to add "-L" arguments to
the compiler wrapper) Spack was only trying the "lib/" and "lib64/"
directories for each dependency install prefix; this missed cases
where packages would install libraries to subdirectories and also was
not customizable. This PR makes use of the ".headers" and ".libs"
properties for more-advanced location of header/library directories.
Since packages can override the default behavior of ".headers" and
".libs", it also allows package writers to customize.
The following environment variables which used to be set by Spack
for a package build have been removed:
* Remove SPACK_PREFIX and SPACK_DEPENDENCIES environment variables as
they are no-longer used
* Remove SPACK_INSTALL environment variable: it was not used before
this PR
* Added a package for the MDAnalysis toolkit.
* Updated LBANN, Hydrogen, and Aluminum to not require CUDA-aware MPI
even when using CUDA. Fixed small bugs within both LBANN and Hydrogen
packages. Added requirements for breathe and cereal to LBANN.
* Update use case for doc dependencies.
* Cleaning up some of the LBANN software stack dependencies.
* fix permission setter
Fix a typo in islink test when applied to files.
* os.walk explicitly set not to follow links
The algorithm strongly rely on not following links.
* Note that `none` is the default for lmod autoload
Save a bit of confusion by *explicitly* pointing out that `none` is
the default value for autoload in the lmod module file generator.
* Add a tip re building software externally
Add a tip about using `autoload: all` when building packages outside
of the tree that use artifacts (e.g. libraries, includes) within the
tree.
CMake supports the notion of secondary generators which provide extra
information to (e.g.) IDEs over and above that normally provided by
the primary generator. Spack only supports the 'Unix Makefiles' and
'Ninja' primary generators but was not parsing out the primary
generator when a secondary generator was also included (e.g. for
a generator attribute like 'Codeblocks - Ninja'). This adds a regex
for extracting the primary generator for validation.
Since the secondary generator is irrelevant to a Spack build, it is
passed on to CMake without further validation.
Add variant to support compiling against main C++ standards if needed
for ABI compatibility. Use 'default' value as Xerces-C autoconf system
does not enforce a minimum, or set a default, C++ standard.
