Currently, environments can either be concretized fully together or fully separately. This works well for users who create environments for interoperable software and can use `concretizer:unify:true`. It does not allow environments with conflicting software to be concretized for maximal interoperability.
The primary use-case for this is facilities providing system software. Facilities provide multiple MPI implementations, but all software built against a given MPI ought to be interoperable.
This PR adds a concretization option `concretizer:unify:when_possible`. When this option is used, Spack will concretize specs in the environment separately, but will optimize for minimal differences in overlapping packages.
* Add a level of indirection to root specs
This commit introduce the "literal" atom, which comes with
a few different "arities". The unary "literal" contains an
integer that id the ID of a spec literal. Other "literals"
contain information on the requests made by literal ID. For
instance zlib@1.2.11 generates the following facts:
literal(0,"root","zlib").
literal(0,"node","zlib").
literal(0,"node_version_satisfies","zlib","1.2.11").
This should help with solving large environments "together
where possible" since later literals can be now solved
together in batches.
* Add a mechanism to relax the number of literals being solved
* Modify spack solve to display the new criteria
Since the new criteria is above all the build criteria,
we need to modify the way we display the output.
Originally done by Greg in #27964 and cherry-picked
to this branch by the co-author of the commit.
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* Inject reusable specs into the solve
Instead of coupling the PyclingoDriver() object with
spack.config, inject the concrete specs that can be
reused.
A method level function takes care of reading from
the store and the buildcache.
* spack solve: show output of multi-rounds
* add tests for best-effort coconcretization
* Enforce having at least a literal being solved
Co-authored-by: Greg Becker <becker33@llnl.gov>
* Py-x21 now works, needs dependencies
Conflicts:
var/spack/repos/rit-rc/packages/py-x21/package.py
* Added dependencies to py-x21
* Making flake style check happy
* [py-x21] flake8
* [py-x21]
- added homepage
- added placeholder description
- added comment about checksums
* [py-x21] added darwin support and fixed issue with python 3.7 wheel name
* [py-x21] adding checksum hash
* [py-x21] removed duplicate py-pynacl
* [py-x21]
- updated description
- updated version listing to have a different version for each version
of python. Also, versions dependent on sys.platform
- updated url_for_version to not require post concretized information so
that spack checksum works
* [py-x21] isort
Co-authored-by: vehrc <vehrc@rit.edu>
Previously the regex was only checking for presence of quotes as a beginning
or end character and not a matching set. This erroneously identified the
following *single* argument as being quoted:
source bashenvfile &> /dev/null && python3 -c "import os, json; print(json.dumps(dict(os.environ)))"
rocm-5.1.0 removed librocrand.so from ROCM_DIR/rocrand/lib location (but includes are still at this location)
/opt/rocm-5.0.2/lib/librocrand.so
/opt/rocm-5.0.2/rocrand/lib/librocrand.so
/opt/rocm-5.1.0/lib/librocrand.so
drwxr-xr-x 2 root root 617 Mar 8 08:20 /opt/rocm-5.0.2/rocrand/include
drwxr-xr-x 2 root root 617 Mar 31 09:48 /opt/rocm-5.1.0/rocrand/include
Add a config option to strip `-Werror*` or `-Werror=*` from compile lines everywhere.
```yaml
config:
keep_werror: false
```
By default, we strip all `-Werror` arguments out of compile lines, to avoid unwanted
failures when upgrading compilers. You can re-enable `-Werror` in your builds if
you really want to, with either:
```yaml
config:
keep_werror: all
```
or to keep *just* specific `-Werror=XXX` args:
```yaml
config:
keep_werror: specific
```
This should make swapping in newer versions of compilers much smoother when
maintainers have decided to enable `-Werror` by default.
Parse error information is kept for specs, but it doesn't seem like we propagate it
to the user when we encounter an error. This fixes that.
e.g., for this error in a package:
```python
depends_on("python@:3.8", when="0.900:")
```
Before, with no context and no clue that it's even from a particular spec:
```
==> Error: Unexpected token: ':'
```
With this PR:
```
==> Error: Unexpected token: ':'
Encountered when parsing spec:
0.900:
^
```
* Added autotools configure flags to ensure that hwloc finds the correct
version of CUDA that it was concretized against, rather than the first
one that package config finds.
* Added support for finding the correct version of ROCm libraries. Fixed Flake8.
* Fixed guard on finding ROCm library
* [py-h2] py-wheel is implied by PythonPackage
* [py-h2] python dependencies should be type=('build', 'run')
* [py-h2] fixed dependencies for py-h2@4.0.0
* [py-h2] added version 3.2.0
* [py-h2] added version 4.1.0
* [py-h2] Older version requires py-enum34 for older versions of python
Add two new cloud pipelines for E4S on Amazon Linux, include arm and x86 (v3 + v4) stacks.
Notes:
- Updated mpark-variant to remove conflict that no longer exists in Amazon Linux
- Which command on Amazon Linux prefixes on all results when padded_length is too high. In this case, padded_length<=503 works as expected. Chose conservative length of 384.
* Introduce concretizer:unify option to replace spack:concretization
* Deprecate concretization
* Make spack:concretization overrule concretize:unify for now
* Add environment update logic to move from spack:concretization to spack:concretizer:reuse
* Migrate spack:concretization to spack:concretize:unify in all locations
* For new environments make concretizer:unify explicit, so that defaults can be changed in 0.19
* Update h5bench maintainers and versions
* Include version 1.1 for h5bench
* Correct release hash and set default version
* Update .tar.gz version
* Include new version and update runtime
* Update year
* Update package.py
* Update package.py
fixes#30700
To avoid clingo adding penalties for not using the
default value for a variant, it's better to model
the variant as conditional where possible.
* This commit removes the Boost.with_default_variants to variants that packages are precisely dependant upon. This is the third batch of 16 packages with modified boost dependencies.
* style fix
* Update var/spack/repos/builtin/packages/sympol/package.py
Co-authored-by: Tim Haines <thaines.astro@gmail.com>
* fix style
* Apply suggestions from code review
Co-authored-by: Tim Haines <thaines.astro@gmail.com>
* Fix Trilinos boost deps
* Fix style
Co-authored-by: Tim Haines <thaines.astro@gmail.com>
Co-authored-by: Tom Scogland <tom.scogland@gmail.com>
Add a `build_type` variant, which allows building optimized compilers,
as well as target libraries (libstdc++ and friends).
The default is `build_type=RelWithDebInfo`, which corresponds to GCC's
default of -O2 -g.
When building with `+bootstrap %gcc`, also add Spack's arch specific
flags using the common denominator between host and new GCC.
It is done by creating a config/spack.mk file in def patch, that looks
as follows:
```
BOOT_CFLAGS := $(filter-out -O% -g%, $(BOOT_CFLAGS)) -O2 -g -march=znver2 -mtune=znver2
CFLAGS_FOR_TARGET := $(filter-out -O% -g%, $(CFLAGS_FOR_TARGET)) -O2 -g -march=znver2 -mtune=znver2
CXXFLAGS_FOR_TARGET := $(filter-out -O% -g%, $(CXXFLAGS_FOR_TARGET)) -O2 -g -march=znver2 -mtune=znver2
```
The oneapi and dpcpp compilers are essentially the same except for which
binary is used foc CXX. Spack will detect them as "mixed toolchain" and
not inject compiler optimization flags. This will be needed once
archspec has entries for the oneapi and dpcpp compilers. This PR detects
when dpcpp and oneapi are in the toolchains list and explicitly sets
`is_mixed_toolchain` to `False`.
* [py-openslide-python] added verion 1.1.2 and set max py-setuptools version for 1.1.1
* [py-openslide-python]
- setuptools required for all possible newer versions
- python is type build run
* [py-openslide-python] use pil provider
* Add version 3.0 and 3.1 and prelim OpenMP support
* Fix flag handler missing spec variable
* Use self.compiler.openmp_flag instead of -fopenmp
* Fix whitespace
Fixes qt configure errors with external openssl on older systems (rhel7)
See
efc02f9cc3/dist/changes-5.15.0 (L346)
This means for now on, `qt ^openssl@1.0` gets you `qt@5.15.4 ~ssl`:
clingo chooses latest qt version **but disables ssl support**.
Error messages for the clingo concretizer have proven challenging. The current messages are incredibly vague and often don't help users at all. Unsat cores in clingo are not guaranteed to be minimal, and lead to cores that are either not useful or need to be post-processed for hours to reach a minimal core.
Following up on an idea from a slack conversation with kwryankrattiger on slack, this PR takes a new approach. We eliminate most integrity constraints and minima/maxima on choice rules in clingo, and instead force invalid states to imply an error predicate. The error predicate can include context on the cause of the error (Package, Version, etc). These error predicates are then heavily optimized against, to ensure that we do not include error facts in the solution when a solution with no error facts could be generated. When post-processing the clingo solution to construct specs, any error facts cause the program to raise an exception. This leads to much more legible error messages. Each error predicate includes a priority and an error message. The error message is formatted by the remaining arguments to produce the error message. The priority is used to ensure that when clingo has a choice of which rules to violate, it chooses the one which will be most informative to the user.
Performance:
"fresh" concretizations appear to suffer a ~20% performance penalty under this branch, while "reuse" concretizations see a speedup of around 33%.
Possible optimizations if users still see unhelpful messages:
There are currently 3 levels of priority of the error messages. Additional priorities are possible, and can allow us finer granularity to ensure more informative error messages are provided in lieu of less informative ones.
Future work:
Improve tests to ensure that every possible rule implying an error message is exercised
