Commit graph

6431 commits

Author SHA1 Message Date
Massimiliano Culpo
86216cc36e
environment: improve spack remove matching (#39390)
search for equivalent specs, not for equal strings when selecting a spec to remove.
2023-08-31 09:28:52 +00:00
dependabot[bot]
ecb7ad493f
build(deps): bump sphinx from 7.2.4 to 7.2.5 in /lib/spack/docs (#39716)
Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 7.2.4 to 7.2.5.
- [Release notes](https://github.com/sphinx-doc/sphinx/releases)
- [Changelog](https://github.com/sphinx-doc/sphinx/blob/master/CHANGES)
- [Commits](https://github.com/sphinx-doc/sphinx/compare/v7.2.4...v7.2.5)

---
updated-dependencies:
- dependency-name: sphinx
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-31 10:40:33 +02:00
Harmen Stoppels
fb1e81657c
Remove a few local imports in tests (#39719) 2023-08-31 10:40:02 +02:00
Harmen Stoppels
acb02326aa
ASP-based solver: add hidden mode to ignore versions that are moving targets, use that in CI (#39611)
Setting the undocumented variable SPACK_CONCRETIZER_REQUIRE_CHECKSUM
now causes the solver to avoid accounting for versions that are not checksummed.

This feature is used in CI to avoid spurious concretization against e.g. develop branches.
2023-08-31 08:09:37 +00:00
John W. Parent
1ee7c735ec
Windows: oneapi/msvc consistency (#39180)
Currently, OneAPI's setvars scripts effectively disregard any arguments
we're passing to the MSVC vcvars env setup script, and additionally,
completely ignore the requested version of OneAPI, defaulting to whatever
the latest installed on the system is.

This leads to a scenario where we have improperly constructed Windows
native development environments, with potentially multiple versions of
MSVC and OneAPI being loaded or called in the same env. Obviously this is
far from ideal and leads to some fairly inscrutable errors such as
overlapping header files between MSVC and OneAPI and a different version
of OneAPI being called than the env was setup for.

This PR solves this issue by creating a structured invocation of each
relevant script in an order that ensures the correct values are set in
the resultant build env.

The order needs to be:

1. MSVC vcvarsall
2. The compiler specific env.bat script for the relevant version of
   the oneapi compiler we're looking for. The root setvars scripts seems
   to respect this as well, although it is less explicit
3. The root oneapi setvars script, which sets up everything else the
   oneapi env needs and seems to respect previous env invocations.
2023-08-30 23:19:38 +00:00
Todd Gamblin
396f219011 completion: add alias handling
Bash completion is now smarter about handling aliases. In particular, if all completions
for some input command are aliased to the same thing, we'll just complete with that thing.

If you've already *typed* the full alias for a command, we'll complete the alias.

So, for example, here there's more than one real command involved, so all aliases are
shown:

```console
$ spack con
concretise    concretize    config        containerise  containerize
```

Here, there are two possibilities: `concretise` and `concretize`, but both map to
`concretize` so we just complete that:

```console
$ spack conc
concretize
```

And here, the user has already typed `concretis`, so we just go with it as there is only
one option:

```console
 spack concretis
concretise
```
2023-08-30 12:42:31 -07:00
Todd Gamblin
a3ecd7efed Add concretise and containerise aliases for our UK users
From a user:

> Aargh.
> ```
> ==> Error: concretise is not a recognized Spack command or extension command; check with `spack commands`.
> ```

To make things easier for our friends in the UK, this adds `concretise` and
`containerise` aliases for the `spack concretize` and `spack containerize` commands.

- [x] add aliases
- [x] update completions
2023-08-30 12:42:31 -07:00
Harmen Stoppels
5138c71d34
Revert "Add style tool to fix fish file formatting (#39155)" (#39680)
This reverts commit 70c71e8f93.
2023-08-29 18:12:19 +02:00
dependabot[bot]
ffddaabaa0
build(deps): bump sphinx from 7.2.3 to 7.2.4 in /lib/spack/docs (#39668)
Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 7.2.3 to 7.2.4.
- [Release notes](https://github.com/sphinx-doc/sphinx/releases)
- [Changelog](https://github.com/sphinx-doc/sphinx/blob/master/CHANGES)
- [Commits](https://github.com/sphinx-doc/sphinx/compare/v7.2.3...v7.2.4)

---
updated-dependencies:
- dependency-name: sphinx
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-29 14:45:06 +02:00
Adam J. Stewart
70c71e8f93
Add style tool to fix fish file formatting (#39155) 2023-08-28 22:16:44 -05:00
Jonathon Anderson
d9d1eb24f9
modules: copy matched config to prevent bleed (#39421) 2023-08-28 15:03:29 -07:00
Peter Scheibel
95f78440f1
External ROCm: add example configuration (#39602)
* add an example of an external rocm configuration
* include more info
* generalize section to all GPU support
2023-08-26 15:46:25 -07:00
James Smillie
349ba83bc6
Windows symlinking support (#38599)
This reapplies 66f7540, which adds supports for hardlinks/junctions on
Windows systems where developer mode is not enabled.

The commit was reverted on account of multiple issues:

* Checks added to prevent dangling symlinks were interfering with
  existing CI builds on Linux (i.e. builds that otherwise succeed were
  failing for creating dangling symlinks).
* The logic also updated symlinking to perform redirection of relative
  paths, which lead to malformed symlinks.

This commit fixes these issues.
2023-08-25 12:18:19 -07:00
Harmen Stoppels
8914d26867
rebuild-index: fix race condition by avoiding parallelism (#39594) 2023-08-24 08:26:56 +02:00
Massimiliano Culpo
fdea5e7624
Remove leftover attributes from parser (#39574)
#35042 introduced lazy hash parsing, but didn't remove a
few attributes from the parser that were needed only for
concrete specs

This commit removes them, since they are effectively
dead code.
2023-08-24 08:04:43 +02:00
dependabot[bot]
ca1e4d54b5
build(deps): bump sphinx from 7.2.2 to 7.2.3 in /lib/spack/docs (#39603)
Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 7.2.2 to 7.2.3.
- [Release notes](https://github.com/sphinx-doc/sphinx/releases)
- [Changelog](https://github.com/sphinx-doc/sphinx/blob/master/CHANGES)
- [Commits](https://github.com/sphinx-doc/sphinx/compare/v7.2.2...v7.2.3)

---
updated-dependencies:
- dependency-name: sphinx
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-24 07:55:14 +02:00
Massimiliano Culpo
806b8aa966
Uppercase global constants in spec.py (#39573)
* Uppercase global constants in spec.py

Co-authored-by: Bernhard Kaindl <bernhardkaindl7@gmail.com>
2023-08-23 21:26:30 +00:00
Greg Becker
9e5ca525f7
do not warn for invalid package name on repo.yaml if subdirectory: '' (#39439) 2023-08-23 23:23:35 +02:00
Massimiliano Culpo
e0059ef961
ASP-based solver: split heuristic for duplicate nodes (#39593)
The heuristic for duplicate nodes contains a few typos, and
apparently slows down the solve for specs that have a lot of
sub-optimal choices to be taken.

This is likely because with a lot of sub-optimal choices, the
low priority, flawed heuristic is being used by clingo.

Here I split the heuristic, so complex rules that matter only
if we allow multiple nodes from the same package are used
only in that case.
2023-08-23 19:02:20 +00:00
Dom Heinzeller
31d5f56913
Add --fail-fast option for generating build caches (#38496)
Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
2023-08-23 11:20:33 +02:00
Tamara Dahlgren
3f5f4cfe26
docs: API, f-string, and path updates to test section (#39584) 2023-08-23 09:22:25 +02:00
Kamil Iskra
c21e00f504
spack.caches: make fetch_cache_location lowercase (#39575)
fetch_cache_location was erroneously renamed to FETCH_cache_location
as part of #39428, breaking "spack module create".
2023-08-22 16:18:51 -04:00
Massimiliano Culpo
9ae1317e79
ASP-based solver: use edge properties for reused specs (#39508)
Since #34821 we are annotating virtual dependencies on
DAG edges, and reconstructing virtuals in memory when
we read a concrete spec from previous formats.

Therefore, we can remove a TODO in asp.py, and rely on
"virtual_on_edge" facts to be imposed.
2023-08-22 12:07:51 -07:00
dependabot[bot]
afebc11742
Bump sphinx from 6.2.1 to 7.2.2 in /lib/spack/docs (#39502)
Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 6.2.1 to 7.2.2.
- [Release notes](https://github.com/sphinx-doc/sphinx/releases)
- [Changelog](https://github.com/sphinx-doc/sphinx/blob/master/CHANGES)
- [Commits](https://github.com/sphinx-doc/sphinx/compare/v6.2.1...v7.2.2)

---
updated-dependencies:
- dependency-name: sphinx
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-22 12:02:58 +02:00
dependabot[bot]
1d8ff7f742
Bump sphinx-rtd-theme from 1.2.2 to 1.3.0 in /lib/spack/docs (#39562)
Bumps [sphinx-rtd-theme](https://github.com/readthedocs/sphinx_rtd_theme) from 1.2.2 to 1.3.0.
- [Changelog](https://github.com/readthedocs/sphinx_rtd_theme/blob/master/docs/changelog.rst)
- [Commits](https://github.com/readthedocs/sphinx_rtd_theme/compare/1.2.2...1.3.0)

---
updated-dependencies:
- dependency-name: sphinx-rtd-theme
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-22 09:32:39 +02:00
Jordan Galby
f8e0c8caed
Fix Spack freeze on install child process unexpected exit (#39015)
* Fix spack frozen on child process defunct

* Rename parent/child pipe to read/write to emphasize non-duplex mode
2023-08-18 09:41:02 +02:00
Massimiliano Culpo
a022e45866
ASP-based solver: optimize key to intermediate dicts (#39471)
Computing str(spec) is faster than computing hash(spec), and
since all the abstract specs we deal with come from user configuration
they cannot cover DAG structures that are not captured by str() but
are captured by hash()
2023-08-17 14:11:49 +02:00
dependabot[bot]
b19691d503
Bump mypy from 1.5.0 to 1.5.1 in /lib/spack/docs (#39478)
Bumps [mypy](https://github.com/python/mypy) from 1.5.0 to 1.5.1.
- [Commits](https://github.com/python/mypy/compare/v1.5.0...v1.5.1)

---
updated-dependencies:
- dependency-name: mypy
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-17 12:35:49 +02:00
Peter Scheibel
a89f1b1bf4
Add debugging statements to file search (#39121)
Co-authored-by: Scheibel <scheibel1@ml-9983616.the-lab.llnl.gov>
2023-08-17 08:31:06 +02:00
Harmen Stoppels
190a1bf523
Delay abstract hashes lookup (#39251)
Delay lookup for abstract hashes until concretization time, instead of
until Spec comparison. This has a few advantages:

1. `satisfies` / `intersects` etc don't always know where to resolve the
   abstract hash (in some cases it's wrong to look in the current env,
   db, buildcache, ...). Better to let the call site dictate it.
2. Allows search by abstract hash without triggering a database lookup,
   causing quadratic complexity issues (accidental nested loop during
   search)
3. Simplifies queries against the buildcache, they can now use Spec
   instances instead of strings.

The rules are straightforward:

1. a satisfies b when b's hash is prefix of a's hash
2. a intersects b when either a's or b's hash is a prefix of b's or a's
   hash respectively
2023-08-17 08:08:50 +02:00
Massimiliano Culpo
f41d7a89f3 Extract Package from PackageNode for error messages 2023-08-16 06:20:57 -07:00
Harmen Stoppels
4f07205c63
Avoid sort on singleton list during edge insertion (#39458)
The median length of this list of 1. For reasons I don't know, `.sort()`
still like to call the key function.

This saves ~9% of total database read time, and the number of calls
goes from 5305 -> 1715.
2023-08-16 14:33:03 +02:00
Massimiliano Culpo
08f9c7670e
Do not impose provider conditions, if the node is not a provider (#39456)
* Do not impose provider conditions, if the node is not a provider

fixes #39455

When a node can be a provider of a spec, but is not selected as
a provider, we should not be imposing provider conditions on the
virtual.

* Adjust the integrity constraint, by using the correct atom
2023-08-16 05:15:13 -07:00
Harmen Stoppels
b451791336
json: minify by default (#39457) 2023-08-16 11:26:40 +00:00
Massimiliano Culpo
47f176d635
Add new custom markers to unit tests (#33862)
* Add "only_clingo", "only_original" and "not_on_windows" markers

* Modify tests to use the "not_on_windows" marker

* Mark tests that run only with clingo

* Mark tests that run only with the original concretizer
2023-08-16 09:04:10 +02:00
Massimiliano Culpo
9bb5cffc73 Change semantic for providers
If a possible provider is not used to satisfy a vdep,
then it's not a provider of that vdep.
2023-08-15 15:54:37 -07:00
Massimiliano Culpo
135b44ca59 Change "effect_rules" for symmetry with trigger rules
This even though right now we don't have cases where
the effect is on another package.
2023-08-15 15:54:37 -07:00
Massimiliano Culpo
d3aca68e8f Rework conflicts so that "vendors" is not needed anymore 2023-08-15 15:54:37 -07:00
Massimiliano Culpo
fb83f8ef31 Add a description at the top of lp files 2023-08-15 15:54:37 -07:00
Massimiliano Culpo
f69c18a922 Remove commented out code in lp files 2023-08-15 15:54:37 -07:00
Massimiliano Culpo
b95a9d2e47 Reduce line length in lp file 2023-08-15 15:54:37 -07:00
Massimiliano Culpo
def4d19980 Demote warning to debug message 2023-08-15 15:54:37 -07:00
Massimiliano Culpo
1db91e0ccd Rename "main_node" -> "make_node" 2023-08-15 15:54:37 -07:00
Massimiliano Culpo
34ebe7f53c Rename ""*_node" -> "*_dupe" 2023-08-15 15:54:37 -07:00
Massimiliano Culpo
d07d5410f3 Rename "stringify", improve docs 2023-08-15 15:54:37 -07:00
Massimiliano Culpo
1db73eb1f2 Add vendors directive
For the time being this directive prevents the vendored package
to be in the same DAG as the one vendoring it.
2023-08-15 15:54:37 -07:00
Massimiliano Culpo
2da34de519 Add "^" automatically for named conflicts that don't refer to 'this' package
See https://github.com/spack/spack/pull/38447#discussion_r1285291520
2023-08-15 15:54:37 -07:00
Massimiliano Culpo
d237430f47 Inline a few functions that are not needed anymore 2023-08-15 15:54:37 -07:00
Massimiliano Culpo
3f0adae9ef Remove the need for "node_regex" 2023-08-15 15:54:37 -07:00
Massimiliano Culpo
3b4d7bf119 Rename method: "root_node" -> "main_node" 2023-08-15 15:54:37 -07:00