This commit address the case of concretizing a root spec with a
transitive conditional dependency on a virtual package, provided
by an external. Before these modifications default variant values
for the dependency bringing in the virtual package were not
respected, and the external package providing the virtual was added
to the DAG.
The issue stems from two facts:
- Selecting a provider has higher precedence than selecting default variants
- To ensure that an external is preferred, we used a negative weight
To solve it we shift all the providers weight so that:
- External providers have a weight of 0
- Non external provider have a weight of 10 or more
Using a weight of zero for external providers is such that having
an external provider, if present, or not having a provider at all
has the same effect on the higher priority minimization.
Also fixed a few minor bugs in concretize.lp, that were causing
spurious entries in the final answer set.
Cleaned concretize.lp from leftover rules.
This commit introduces a new rule:
real_node(Package) :- not external(Package), node(Package).
that permits to distinguish between an external node and a
real node that shouldn't trim dependency. It solves the
case of concretizing ninja with an external Python.
As reported, conflicts with compiler ranges were not treated
correctly. This commit adds tests to verify the expected behavior
for the new concretizer.
The new rules to enforce a correct behavior involve:
- Adding a rule to prefer the compiler selected for
the root package, if no other preference is set
- Give a strong negative weight to compiler preferences
expressed in packages.yaml
- Maximize on compiler AND compiler version match
As of #13100, Spack installs the dependencies of a _single_ spec in parallel.
Environments, when installed, can only get parallelism from each individual
spec, as they're installed in order. This PR makes entire environments build
in parallel by extending Spack's package installer to accept multiple root
specs. The install command and Environment class have been updated to use
the new parallel install method.
The specs and kwargs for each *uninstalled* package (when not force-replacing
installations) of an environment are collected, passed to the `PackageInstaller`,
and processed using a single build queue.
This introduces a `BuildRequest` class to track install arguments, and it
significantly cleans up the code used to track package ids during installation.
Package ids in the build queue are now just DAG hashes as you would expect,
Other tasks:
- [x] Finish updating the unit tests based on `PackageInstaller`'s use of
`BuildRequest` and the associated changes
- [x] Change `environment.py`'s `install_all` to use the `PackageInstaller` directly
- [x] Change the `install` command to leverage the new installation process for multiple specs
- [x] Change install output messages for external packages, e.g.:
`[+] /usr` -> `[+] /usr (external bzip2-1.0.8-<dag-hash>`
- [x] Fix incomplete environment install's view setup/update and not confirming all
packages are installed (?)
- [x] Ensure externally installed package dependencies are properly accounted for in
remaining build tasks
- [x] Add tests for coverage (if insufficient and can identity the appropriate, uncovered non-comment lines)
- [x] Add documentation
- [x] Resolve multi-compiler environment install issues
- [x] Fix issue with environment installation reporting (restore CDash/JUnit reports)
* add gcc 4.8 conflict
* commit suggestion
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Add WRF 3.9.1.1 and improve recipe robustness
* Include version 3.9.1.1 as common benchmarking workload
* Fix compilation against recent glibc (detect spack installed libtirpc)
* Detect and handle failed compilation (upstream use make -i)
* WRF: PR changes round 1
fix build jobs
fix maintainers
fix pkgconfig dependency
use Executable to run compile stage
repair some overzealous autoformatting by black
* WRF: make recipe py26 compatible
* wrf: recipe review changes round 2
* more python 26 fixes
The unattended install using the pre-compiled binaries (tl-install)
needs a .profile file or it goes in interactive mode blocking the
install process forever
* Added guard for setting CUB_DIR to only when cuda variant is true
* Added support for OpenMP on OSX platforms
* Updated the way that LBANN, Hydrogen, and DiHydrogen handle
apple-clang with OpenMP and Clang installed on OS X via brew.
* Fixed bug in spec resolution
* Fixed merge conflict
* Fixed typo
* Fixed flake8
* AMD - Bumped up version for hip-rocclr, rocm-opencl, rocm-smi-lib
* AMD ROCm - HIP update and bump up version to 3.9.0 for rccl,debug agent, hip-rocclr and atmi
* Update package.py
* Update package.py
* Update package.py
* Update var/spack/repos/builtin/packages/hip/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-json-get: new package at 1.1.1
* py-json-get: new package at 1.1.1
* r-bigalgebra: new package at 0.8.4
* r-bigalgebra: new package at 0.8.4 with corrections
* Added an additional change to tarball and dependencies
* removing accidentally added file
* Added tarball that uses mirror and removed redundant dependencies
* Fixed version and added dep.
* Updated checksum
* Fixed urls
* Added list_url
Co-authored-by: las_djorton <las_djorton@build.las.iastate.edu>
* Add CUDA support to superlu-dist
* Use spec['cuda'].libs.directories[0] iso spec['cuda'].prefix.lib
so it works for both lib and lib64
The suggested:
args.append('-DTPL_CUDA_LIBRARIES=' +
spec['cuda'].libs.ld_flags)
did not work because it does not link with cuBLAS.
* No version of yaml-cpp in spack can build shared AND
static libraries at the same time. So drop the "static"
variant and let "shared" handle that alone.
Or in other words: No version handles the
BUILD_STATIC_LIBS flag.
* The flag for building shared libraries changed from
BUILD_SHARED_LIBS to YAML_BUILD_SHARED_LIBS at some
point. So just pass both flags.
* Use the newer define_from_variant.
* [py-cuml] created template
* [py-cuml] setup phases and added build_directory
* [py-cuml] added dependencies
* [py-cuml] depends on libcumlprims
* [py-cuml] requiring multigpu version
* [py-cuml] figuring out the best way to get concretization to happen cleanly
* [py-cuml] removed singlegpu variat from libcuml
* [py-cuml] depends on py-cudf
* [py-cuml] depends on cupy
* [py-cuml] fixed typoo
* [py-cuml] depends on py-scipy
* [py-cuml] depends on py-treelite
* [py-cuml] py-treelite is now a variant of treelite
* [py-cuml] depends on joblib
* [py-cuml] depends on py-scikit-learn
* [py-cuml] flake8
* [py-cuml] added homepage and description. removed fixmes
* [py-cuml] updated checksum
* Enabling build of v1.9.x development branch.
* v1.8.1 is the preferred (stable) version.
* Fixing code style
Co-authored-by: Filippo Spiga <fspiga@nvidia.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* [podio] put python dir in python path
* Update var/spack/repos/builtin/packages/podio/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* tskit package
* Update var/spack/repos/builtin/packages/tskit/package.py
I can't see any hard requirement for 3.6:
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* fixes following PR review
* Update var/spack/repos/builtin/packages/tskit/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Version 5.32.0 has been out for quite a while and Linux distributions
are shipping it. I have also done a rebuild of some common packages with
the new version. Let's make it the preferred version.
* amrex: new options names for version > 20.11
* amrex: change option name DIM -> AMReX_SPACEDIM
* Update var/spack/repos/builtin/packages/amrex/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Added code to help DiHydrogen find cuDNN and CUB
* Cleaning up dependencies on CUB and adding guards for when newer
versions of CUDA include CUB and it should be excluded.
* Changed Hydrogen to disable half support by default.
* Have LBANN force Hydrogen and DiHydrogen to build without half when the variant is disabled.
* Added explicit variants to enusre that if LBANN is build without Cuda,
Aluminum, or Half support, it enforces those constraints for Hydrogen
and DiHydrogen. Cleaned up the use of Python extend versus append in
LBANN and DiHydrogen recipes.
* Fixed Flake8
* [evtgen] add env var
* Update var/spack/repos/builtin/packages/evtgen/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
See #19784
virtualgl CMake system is looking for a specific libjpeg-turbo include
file, not present in libjpeg (currently the only other jpeg provider)
* cget package
* Update var/spack/repos/builtin/packages/cget/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/cget/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/cget/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Updates in LBANN an Aluminum code now allow working with versions
HWLOC 1.11.x and 2.x and up.
* Updating the minimum CMake version to address a pending PR in LBANN
that will require C++17 support and needs CMake to properly separate
the compiler flags from nvcc.
* Clarified the support for different versions of HWLOC in LBANN
* filtlong package
* Update var/spack/repos/builtin/packages/filtlong/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
