As of #13100, Spack installs the dependencies of a _single_ spec in parallel.
Environments, when installed, can only get parallelism from each individual
spec, as they're installed in order. This PR makes entire environments build
in parallel by extending Spack's package installer to accept multiple root
specs. The install command and Environment class have been updated to use
the new parallel install method.
The specs and kwargs for each *uninstalled* package (when not force-replacing
installations) of an environment are collected, passed to the `PackageInstaller`,
and processed using a single build queue.
This introduces a `BuildRequest` class to track install arguments, and it
significantly cleans up the code used to track package ids during installation.
Package ids in the build queue are now just DAG hashes as you would expect,
Other tasks:
- [x] Finish updating the unit tests based on `PackageInstaller`'s use of
`BuildRequest` and the associated changes
- [x] Change `environment.py`'s `install_all` to use the `PackageInstaller` directly
- [x] Change the `install` command to leverage the new installation process for multiple specs
- [x] Change install output messages for external packages, e.g.:
`[+] /usr` -> `[+] /usr (external bzip2-1.0.8-<dag-hash>`
- [x] Fix incomplete environment install's view setup/update and not confirming all
packages are installed (?)
- [x] Ensure externally installed package dependencies are properly accounted for in
remaining build tasks
- [x] Add tests for coverage (if insufficient and can identity the appropriate, uncovered non-comment lines)
- [x] Add documentation
- [x] Resolve multi-compiler environment install issues
- [x] Fix issue with environment installation reporting (restore CDash/JUnit reports)
* add gcc 4.8 conflict
* commit suggestion
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Add WRF 3.9.1.1 and improve recipe robustness
* Include version 3.9.1.1 as common benchmarking workload
* Fix compilation against recent glibc (detect spack installed libtirpc)
* Detect and handle failed compilation (upstream use make -i)
* WRF: PR changes round 1
fix build jobs
fix maintainers
fix pkgconfig dependency
use Executable to run compile stage
repair some overzealous autoformatting by black
* WRF: make recipe py26 compatible
* wrf: recipe review changes round 2
* more python 26 fixes
The unattended install using the pre-compiled binaries (tl-install)
needs a .profile file or it goes in interactive mode blocking the
install process forever
* Added guard for setting CUB_DIR to only when cuda variant is true
* Added support for OpenMP on OSX platforms
* Updated the way that LBANN, Hydrogen, and DiHydrogen handle
apple-clang with OpenMP and Clang installed on OS X via brew.
* Fixed bug in spec resolution
* Fixed merge conflict
* Fixed typo
* Fixed flake8
* AMD - Bumped up version for hip-rocclr, rocm-opencl, rocm-smi-lib
* AMD ROCm - HIP update and bump up version to 3.9.0 for rccl,debug agent, hip-rocclr and atmi
* Update package.py
* Update package.py
* Update package.py
* Update var/spack/repos/builtin/packages/hip/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
