Commit graph

20235 commits

Author SHA1 Message Date
Todd Gamblin
ada6ecc797 concretizer: unify logic for spec conditionals
This builds on #20638 by unifying all the places in the concretizer where
things are conditional on specs. Previously, we duplicated a common spec
conditional pattern for dependencies, virtual providers, conflicts, and
externals. That was introduced in #20423 and refined in #20507, and
roughly looked as follows.

Given some directives in a package like:

```python
depends_on("foo@1.0+bar", when="@2.0+variant")
provides("mpi@2:", when="@1.9:")
```

We handled the `@2.0+variant` and `@1.9:` parts by generating generated
`dependency_condition()`, `required_dependency_condition()`, and
`imposed_dependency_condition()` facts to trigger rules like this:

```prolog
dependency_conditions_hold(ID, Parent, Dependency) :-
  attr(Name, Arg1)             : required_dependency_condition(ID, Name, Arg1);
  attr(Name, Arg1, Arg2)       : required_dependency_condition(ID, Name, Arg1, Arg2);
  attr(Name, Arg1, Arg2, Arg3) : required_dependency_condition(ID, Name, Arg1, Arg2, Arg3);
  dependency_condition(ID, Parent, Dependency);
  node(Parent).
```

And we handled `foo@1.0+bar` and `mpi@2:` parts ("imposed constraints")
like this:

```prolog
attr(Name, Arg1, Arg2) :-
  dependency_conditions_hold(ID, Package, Dependency),
  imposed_dependency_condition(ID, Name, Arg1, Arg2).

attr(Name, Arg1, Arg2, Arg3) :-
  dependency_conditions_hold(ID, Package, Dependency),
  imposed_dependency_condition(ID, Name, Arg1, Arg2, Arg3).
```

These rules were repeated with different input predicates for
requirements (e.g., `required_dependency_condition`) and imposed
constraints (e.g., `imposed_dependency_condition`) throughout
`concretize.lp`. In #20638 it got to be a bit confusing, because we used
the same `dependency_condition_holds` predicate to impose constraints on
conditional dependencies and virtual providers. So, even though the
pattern was repeated, some of the conditional rules were conjoined in a
weird way.

Instead of repeating this pattern everywhere, we now have *one* set of
consolidated rules for conditions:

```prolog
condition_holds(ID) :-
  condition(ID);
  attr(Name, A1)         : condition_requirement(ID, Name, A1);
  attr(Name, A1, A2)     : condition_requirement(ID, Name, A1, A2);
  attr(Name, A1, A2, A3) : condition_requirement(ID, Name, A1, A2, A3).

attr(Name, A1)         :- condition_holds(ID), imposed_constraint(ID, Name, A1).
attr(Name, A1, A2)     :- condition_holds(ID), imposed_constraint(ID, Name, A1, A2).
attr(Name, A1, A2, A3) :- condition_holds(ID), imposed_constraint(ID, Name, A1, A2, A3).
```

this allows us to use `condition(ID)` and `condition_holds(ID)` to
encapsulate the conditional logic on specs in all the scenarios where we
need it. Instead of defining predicates for the requirements and imposed
constraints, we generate the condition inputs with generic facts, and
define predicates to associate the condition ID with a particular
scenario. So, now, the generated facts for a condition look like this:

```prolog
condition(121).
condition_requirement(121,"node","cairo").
condition_requirement(121,"variant_value","cairo","fc","True").
imposed_constraint(121,"version_satisfies","fontconfig","2.10.91:").
dependency_condition(121,"cairo","fontconfig").
dependency_type(121,"build").
dependency_type(121,"link").
```

The requirements and imposed constraints are generic, and we associate
them with their meaning via the id. Here, `dependency_condition(121,
"cairo", "fontconfig")` tells us that condition 121 has to do with the
dependency of `cairo` on `fontconfig`, and the conditional dependency
rules just become:

```prolog
dependency_holds(Package, Dependency, Type) :-
  dependency_condition(ID, Package, Dependency),
  dependency_type(ID, Type),
  condition_holds(ID).
```

Dependencies, virtuals, conflicts, and externals all now use similar
patterns, and the logic for generating condition facts is common to all
of them on the python side, as well. The more specific routines like
`package_dependencies_rules` just call `self.condition(...)` to get an id
and generate requirements and imposed constraints, then they generate
their extra facts with the returned id, like this:

```python
    def package_dependencies_rules(self, pkg, tests):
        """Translate 'depends_on' directives into ASP logic."""
        for _, conditions in sorted(pkg.dependencies.items()):
            for cond, dep in sorted(conditions.items()):
                condition_id = self.condition(cond, dep.spec, pkg.name)  # create a condition and get its id
                self.gen.fact(fn.dependency_condition(  # associate specifics about the dependency w/the id
                    condition_id, pkg.name, dep.spec.name
                ))
        # etc.
```

- [x] unify generation and logic for conditions
- [x] use unified logic for dependencies
- [x] use unified logic for virtuals
- [x] use unified logic for conflicts
- [x] use unified logic for externals

LocalWords:  concretizer mpi attr Arg concretize lp cairo fc fontconfig
LocalWords:  virtuals def pkg cond dep fn refactor github py
2021-03-16 12:50:14 +01:00
Tomoyasu Nojiri
ce7bde24d4
amdlibflame: Fix build error (#22316) 2021-03-16 10:32:51 +00:00
Tomoyasu Nojiri
475c877fc9
libflame: Fix build error (#22315) 2021-03-16 10:03:32 +00:00
Harmen Stoppels
8ffeb96c77
hwloc: add dependency on ncurses (#22300)
Co-authored-by: Michal Sudwoj <msudwoj@student.ethz.ch>
2021-03-16 08:34:41 +01:00
Adam J. Stewart
d7f1ff68fb
OpenCV: add new version, simplify package (#22292) 2021-03-15 14:41:24 -07:00
Harmen Stoppels
195341113e
Expand relative dev paths in environment files (#22045)
* Rewrite relative dev_spec paths internally to absolute paths in case of relocation of the environment file

* Test relative paths for dev_path in environments

* Add a --keep-relative flag to spack env create

This ensures that relative paths of develop paths are not expanded to
absolute paths when initializing the environment in a different location
from the spack.yaml init file.
2021-03-15 15:38:35 -05:00
Harmen Stoppels
4f1d9d6095
Propagate --test= for environments (#22040)
* Propagate --test= for environments

* Improve help comment for spack concretize --test flag

* Add tests for --test with environments
2021-03-15 15:34:18 -05:00
Chuck Atkins
f949ae772d
vtk-h: Fix missing trailing-colon in CMake version (#22302) 2021-03-15 20:16:56 +00:00
Harmen Stoppels
6ba7632d2b
Make py-boto3 installable again with old concretizer (#22298) 2021-03-15 18:21:11 +00:00
Adam J. Stewart
d166ca91f1
dos2unix: gettext needed for linking (#22288) 2021-03-15 12:19:15 -06:00
Adam J. Stewart
70c505a0b8
Fix use of quotes in Python build system (#22279) 2021-03-15 11:11:27 -07:00
Adam J. Stewart
78d1b28b72
py-tqdm: add new version, variants (#22290) 2021-03-15 10:03:52 -07:00
Adam J. Stewart
3c493ae629
PyTorch: add missing tqdm dependency (#22289) 2021-03-15 09:55:31 -07:00
Tomoyasu Nojiri
9c7669ed67
fortrilinos: Fix trilinos depend (#22295) 2021-03-15 10:31:10 -06:00
Christoph Junghans
0c8df39fce
votca-*: add v2021 (#22282) 2021-03-15 11:38:55 +01:00
a-saitoh-fj
3e4a24c878
py-aioitertools: New package (#22256)
* py-aioitertools: New package

* py-aioitertools: Fixed python dependencies, and Fixed style of code

* switch to the source build
2021-03-15 04:58:14 +00:00
Vanessasaurus
6ff717f395
do not validate variants of concrete specs in solver setup (#22272)
Currently, regardless of a spec being concrete or not, we validate its variants in `spec_clauses` (part of `SpackSolverSetup`).  

This PR skips the check if the spec is concrete.

The reason we want to do this is so that the solver setup class (really, `spec_clauses`) can be used for cases when we just want the logic statements / facts (is that what they are called?) and we don't need to re-validate an already concrete spec.  We can't change existing concrete specs, and we have to be able to handle them *even if they violate constraints in the current spack*.  This happens in practice if we are doing the validation for a spec produced by a different spack install.

Signed-off-by: vsoch <vsoch@users.noreply.github.com>
2021-03-14 19:00:15 +00:00
a-saitoh-fj
d22bad13b1
py-botocore: Update version to install py-s3fs (#22257)
* Update version to install py-s3fs

* Update the required version of py-urllib3, and Set the dependent conditions of py-docutils
2021-03-14 06:29:50 +00:00
Seth R. Johnson
5b2a54952c
openblas: prevent microarch flags for generic targets (#22270)
As of OpenBLAS 0.3.13, leaving off `TARGET` by default optimizes most
code for the host system -- adding flags that cause the resulting
library to fail (SIGILL) on older systems.  This change should ensure
that a "x86_64" target for example will work across deployment systems.

https://github.com/xianyi/OpenBLAS/issues/3139
2021-03-13 11:00:15 +00:00
Vanessasaurus
746081e933
adding spack -c to set one off config arguments (#22251)
This pull request will add the ability for a user to add a configuration argument on the fly, on the command line, e.g.,:

```bash
$ spack -c config:install_tree:root:/path/to/config.yaml -c packages:all:compiler:[gcc] list --help
```
The above command doesn't do anything (I'm just getting help for list) but you can imagine having another root of packages, and updating it on the fly for a command (something I'd like to do in the near future!)

I've moved the logic for config_add that used to be in spack/cmd/config.py into spack/config.py proper, and now both the main.py (where spack commands live) and spack/cmd/config.py use these functions. I only needed spack config add, so I didn't move the others. We can move the others if there are also needed in multiple places.
2021-03-13 05:31:26 +00:00
Bryan Herman
839af2bd70
fixed ^python in py-rich, py-dvc (#22264) 2021-03-12 22:53:31 +00:00
Bryan Herman
f213cf8349
py-h5py: offline installation and deps fix (#22262)
* added h5py patch file for offline installs to work

* h5py v3+ deps consistent with setup.py, add patch
2021-03-12 19:44:37 +00:00
Chuck Atkins
474e616a5b
ascent: Relax CMake version constraints (#22253) 2021-03-12 11:37:43 -08:00
Valentin Volkl
cf29ee6b2b
[dd4hep] update env var (#22218) 2021-03-12 11:10:07 -07:00
Tamara Dahlgren
61baa40160
bugfix: ensure spack test list still works (#22203)
Was getting the following error:

```
$ spack test list
==> Error: issubclass() arg 1 must be a class
```

This PR adds a check in `has_test_method` (in case it is re-used elsewhere such as #22097) and ensures a class is passed to the method from `spack test list`.
2021-03-12 09:56:17 -08:00
iarspider
b2ece3abba
Rivet: syntax fixes (#22225) 2021-03-12 09:52:07 -08:00
Desmond Orton
24b7aff837
Version update to 1.9.4 (#22231) 2021-03-12 09:44:10 -08:00
Brian Van Essen
8a8fad8474
LBANN software stack: new versions and dependencies (#22234)
Updated the versions for DiHydrogen and Aluminum. Added new constraints on versions of Aluminum that are used across the software stack.  Cleaned up the dependency on DiHydrogen for LBANN.
2021-03-12 10:43:13 -07:00
Sinan
650f24f4d3
add hints for dependency resolution (#22238)
Co-authored-by: sbulut <sbulut@3vgeomatics.com>
2021-03-12 09:36:11 -08:00
a-saitoh-fj
5790ec5359
py-fsspec: Update version to install py-s3fs (#22259) 2021-03-12 11:31:00 -06:00
Bryan Herman
a2e9a9076f
fixed when python dep in ftfy dep for py benedict (#22261) 2021-03-12 10:28:13 -07:00
Tomoyasu Nojiri
19b163e49d
chai: Add 2.3.0 version (#22254) 2021-03-12 10:16:06 -07:00
a-saitoh-fj
c6440eb23c
py-chainer: Add test method for ChainerMN (continued #21848, #21940) (#22189)
* py-chainer: Add test method for ChainerMN (continued #21848, #21940)

* py-chainer: Fixed the word in the message

* py-chainer: Delete unnecessary imports

* py-chainer: Incorporation of the measures pointed out in #21940 was insufficient.
2021-03-12 11:40:25 +00:00
kuramoto-fj
40147d9955
mptensor: Add test method (#21712)
* mptensor: Add test method

* mptensor: Reverted the receipe on merged style

* mptensor: Changed """ to #
2021-03-12 06:13:38 +00:00
Sergei Shudler
21b2d7109a
superlu-dist: Add e4s testsuite-inspired smoke test (#22237)
* Added a smoke test for superlu-dist recipe

* Fixed small issues following a PR review
2021-03-12 04:22:34 +00:00
Paul Kuberry
e13ce390fe
Add internal package options to Trilinos (#22205)
Adds several EpetraExt_BUILD_* options as well as an Amesos2_ENABLE_Basker option. Adds `none` as an option to `gotype=`, which should be among the options since 'none' is specifically handled later in the package definition.

Adds `stokhos` and `trilinoscouplings` as options in spack which already are available in CMake for Trilinos (e.g. Trilinos_ENABLE_Stokhos:BOOL=)
2021-03-11 18:23:24 -08:00
BerengerBerthoul
ca4a566443
py-pytest-html recipe (#22221)
* py-pytest-html recipe

* added missing deps + copyright

* Update var/spack/repos/builtin/packages/py-pytest-html/package.py

Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>

* Update var/spack/repos/builtin/packages/py-pytest-metadata/package.py

Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>

Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
2021-03-12 02:17:12 +00:00
Thomas Kluyver
bb4ccdfa91
Add newest versions of py-h5py (#22245)
* Add newest versions of py-h5py

* Update dependencies for py-h5py
2021-03-12 00:39:25 +00:00
Tomoyasu Nojiri
952f76bda3
r-rhdf5filters: Fix for aarch64 (#22074) 2021-03-11 17:56:55 +00:00
Tomoyasu Nojiri
ab9580c168
bind9: add missing dependencies (#22212) 2021-03-11 16:32:29 +00:00
Tomoyasu Nojiri
e11f5df7dc
ctffind: update url and version directive (#22039) 2021-03-11 15:23:41 +00:00
Ye Luo
d0610f7a39
elpa: fix build of versions before 2020 with GCC >= 10 (#22182) 2021-03-11 09:43:53 +00:00
Andrew W Elble
31cf0e9c0d
py-imageio: new version 2.9.0 (#22229) 2021-03-11 04:41:13 +00:00
Gabriel Rockefeller
821769c95e
eospac: add patch to fix selection of compiler flags (#21626)
Patch eospac's Makefile.-linux-gnu.hashes to consider only `$(notdir
$(F90))` when constructing a key to look up compiler flags in the
_F90-CPUINFO_COMP_FLAGS associative array.

This patch was accepted into eospac itself after the release of
6.4.2beta, so apply it only to 6.4.2beta and earlier releases.
2021-03-11 01:47:13 +00:00
Benjamin Tovar
f346db83ac
cctools: add v7.2.1 (#22193) 2021-03-10 21:48:29 +00:00
David Pape
f73182fd98
gearshifft: fix patch, add support for mkl and rocm (#22195)
- Fix faulty patch
- Only use GEARSHIFFT_BACKEND_FFTW_PTHREADS if ~openmp
- Explicitly disable float16 support
- Use correct minimum required Boost version
- Add variants for Intel MKL and ROCm rocfft
2021-03-10 21:03:06 +00:00
Scott Wittenburg
7b97fe206b
Pipelines: Fix issue that allowed failing jobs to pass (#22199)
This is a workaround for an issue with how "spack install" is invoked from within "spack ci rebuild".  The fact that we don't get an exception or even the actual returncode when using the object returned by spack.util.executable.which('spack') to install the target spec means we get no indication of failures about the install command itself.  Instead we rely on the subsequent buildcache creation failure to fail the job.
2021-03-10 11:11:30 -08:00
Michael Kuhn
43473995ad
meson: apply rpath patch for current versions (#22216)
meson now only selectively strips rpaths but can still strip required
ones (like glib's pcre dependency).
2021-03-10 18:42:39 +00:00
darmac
d2ce4b565c
vigra: add master branch (#22037) 2021-03-10 17:57:35 +00:00
Danny McClanahan
f0275e84ad
fix setup-env.sh on older linux zsh (#21721) 2021-03-10 09:44:50 -08:00