Commit graph

6418 commits

Author SHA1 Message Date
Harmen Stoppels
8914d26867
rebuild-index: fix race condition by avoiding parallelism (#39594) 2023-08-24 08:26:56 +02:00
Massimiliano Culpo
fdea5e7624
Remove leftover attributes from parser (#39574)
#35042 introduced lazy hash parsing, but didn't remove a
few attributes from the parser that were needed only for
concrete specs

This commit removes them, since they are effectively
dead code.
2023-08-24 08:04:43 +02:00
dependabot[bot]
ca1e4d54b5
build(deps): bump sphinx from 7.2.2 to 7.2.3 in /lib/spack/docs (#39603)
Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 7.2.2 to 7.2.3.
- [Release notes](https://github.com/sphinx-doc/sphinx/releases)
- [Changelog](https://github.com/sphinx-doc/sphinx/blob/master/CHANGES)
- [Commits](https://github.com/sphinx-doc/sphinx/compare/v7.2.2...v7.2.3)

---
updated-dependencies:
- dependency-name: sphinx
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-24 07:55:14 +02:00
Massimiliano Culpo
806b8aa966
Uppercase global constants in spec.py (#39573)
* Uppercase global constants in spec.py

Co-authored-by: Bernhard Kaindl <bernhardkaindl7@gmail.com>
2023-08-23 21:26:30 +00:00
Greg Becker
9e5ca525f7
do not warn for invalid package name on repo.yaml if subdirectory: '' (#39439) 2023-08-23 23:23:35 +02:00
Massimiliano Culpo
e0059ef961
ASP-based solver: split heuristic for duplicate nodes (#39593)
The heuristic for duplicate nodes contains a few typos, and
apparently slows down the solve for specs that have a lot of
sub-optimal choices to be taken.

This is likely because with a lot of sub-optimal choices, the
low priority, flawed heuristic is being used by clingo.

Here I split the heuristic, so complex rules that matter only
if we allow multiple nodes from the same package are used
only in that case.
2023-08-23 19:02:20 +00:00
Dom Heinzeller
31d5f56913
Add --fail-fast option for generating build caches (#38496)
Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
2023-08-23 11:20:33 +02:00
Tamara Dahlgren
3f5f4cfe26
docs: API, f-string, and path updates to test section (#39584) 2023-08-23 09:22:25 +02:00
Kamil Iskra
c21e00f504
spack.caches: make fetch_cache_location lowercase (#39575)
fetch_cache_location was erroneously renamed to FETCH_cache_location
as part of #39428, breaking "spack module create".
2023-08-22 16:18:51 -04:00
Massimiliano Culpo
9ae1317e79
ASP-based solver: use edge properties for reused specs (#39508)
Since #34821 we are annotating virtual dependencies on
DAG edges, and reconstructing virtuals in memory when
we read a concrete spec from previous formats.

Therefore, we can remove a TODO in asp.py, and rely on
"virtual_on_edge" facts to be imposed.
2023-08-22 12:07:51 -07:00
dependabot[bot]
afebc11742
Bump sphinx from 6.2.1 to 7.2.2 in /lib/spack/docs (#39502)
Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 6.2.1 to 7.2.2.
- [Release notes](https://github.com/sphinx-doc/sphinx/releases)
- [Changelog](https://github.com/sphinx-doc/sphinx/blob/master/CHANGES)
- [Commits](https://github.com/sphinx-doc/sphinx/compare/v6.2.1...v7.2.2)

---
updated-dependencies:
- dependency-name: sphinx
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-22 12:02:58 +02:00
dependabot[bot]
1d8ff7f742
Bump sphinx-rtd-theme from 1.2.2 to 1.3.0 in /lib/spack/docs (#39562)
Bumps [sphinx-rtd-theme](https://github.com/readthedocs/sphinx_rtd_theme) from 1.2.2 to 1.3.0.
- [Changelog](https://github.com/readthedocs/sphinx_rtd_theme/blob/master/docs/changelog.rst)
- [Commits](https://github.com/readthedocs/sphinx_rtd_theme/compare/1.2.2...1.3.0)

---
updated-dependencies:
- dependency-name: sphinx-rtd-theme
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-22 09:32:39 +02:00
Jordan Galby
f8e0c8caed
Fix Spack freeze on install child process unexpected exit (#39015)
* Fix spack frozen on child process defunct

* Rename parent/child pipe to read/write to emphasize non-duplex mode
2023-08-18 09:41:02 +02:00
Massimiliano Culpo
a022e45866
ASP-based solver: optimize key to intermediate dicts (#39471)
Computing str(spec) is faster than computing hash(spec), and
since all the abstract specs we deal with come from user configuration
they cannot cover DAG structures that are not captured by str() but
are captured by hash()
2023-08-17 14:11:49 +02:00
dependabot[bot]
b19691d503
Bump mypy from 1.5.0 to 1.5.1 in /lib/spack/docs (#39478)
Bumps [mypy](https://github.com/python/mypy) from 1.5.0 to 1.5.1.
- [Commits](https://github.com/python/mypy/compare/v1.5.0...v1.5.1)

---
updated-dependencies:
- dependency-name: mypy
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-17 12:35:49 +02:00
Peter Scheibel
a89f1b1bf4
Add debugging statements to file search (#39121)
Co-authored-by: Scheibel <scheibel1@ml-9983616.the-lab.llnl.gov>
2023-08-17 08:31:06 +02:00
Harmen Stoppels
190a1bf523
Delay abstract hashes lookup (#39251)
Delay lookup for abstract hashes until concretization time, instead of
until Spec comparison. This has a few advantages:

1. `satisfies` / `intersects` etc don't always know where to resolve the
   abstract hash (in some cases it's wrong to look in the current env,
   db, buildcache, ...). Better to let the call site dictate it.
2. Allows search by abstract hash without triggering a database lookup,
   causing quadratic complexity issues (accidental nested loop during
   search)
3. Simplifies queries against the buildcache, they can now use Spec
   instances instead of strings.

The rules are straightforward:

1. a satisfies b when b's hash is prefix of a's hash
2. a intersects b when either a's or b's hash is a prefix of b's or a's
   hash respectively
2023-08-17 08:08:50 +02:00
Massimiliano Culpo
f41d7a89f3 Extract Package from PackageNode for error messages 2023-08-16 06:20:57 -07:00
Harmen Stoppels
4f07205c63
Avoid sort on singleton list during edge insertion (#39458)
The median length of this list of 1. For reasons I don't know, `.sort()`
still like to call the key function.

This saves ~9% of total database read time, and the number of calls
goes from 5305 -> 1715.
2023-08-16 14:33:03 +02:00
Massimiliano Culpo
08f9c7670e
Do not impose provider conditions, if the node is not a provider (#39456)
* Do not impose provider conditions, if the node is not a provider

fixes #39455

When a node can be a provider of a spec, but is not selected as
a provider, we should not be imposing provider conditions on the
virtual.

* Adjust the integrity constraint, by using the correct atom
2023-08-16 05:15:13 -07:00
Harmen Stoppels
b451791336
json: minify by default (#39457) 2023-08-16 11:26:40 +00:00
Massimiliano Culpo
47f176d635
Add new custom markers to unit tests (#33862)
* Add "only_clingo", "only_original" and "not_on_windows" markers

* Modify tests to use the "not_on_windows" marker

* Mark tests that run only with clingo

* Mark tests that run only with the original concretizer
2023-08-16 09:04:10 +02:00
Massimiliano Culpo
9bb5cffc73 Change semantic for providers
If a possible provider is not used to satisfy a vdep,
then it's not a provider of that vdep.
2023-08-15 15:54:37 -07:00
Massimiliano Culpo
135b44ca59 Change "effect_rules" for symmetry with trigger rules
This even though right now we don't have cases where
the effect is on another package.
2023-08-15 15:54:37 -07:00
Massimiliano Culpo
d3aca68e8f Rework conflicts so that "vendors" is not needed anymore 2023-08-15 15:54:37 -07:00
Massimiliano Culpo
fb83f8ef31 Add a description at the top of lp files 2023-08-15 15:54:37 -07:00
Massimiliano Culpo
f69c18a922 Remove commented out code in lp files 2023-08-15 15:54:37 -07:00
Massimiliano Culpo
b95a9d2e47 Reduce line length in lp file 2023-08-15 15:54:37 -07:00
Massimiliano Culpo
def4d19980 Demote warning to debug message 2023-08-15 15:54:37 -07:00
Massimiliano Culpo
1db91e0ccd Rename "main_node" -> "make_node" 2023-08-15 15:54:37 -07:00
Massimiliano Culpo
34ebe7f53c Rename ""*_node" -> "*_dupe" 2023-08-15 15:54:37 -07:00
Massimiliano Culpo
d07d5410f3 Rename "stringify", improve docs 2023-08-15 15:54:37 -07:00
Massimiliano Culpo
1db73eb1f2 Add vendors directive
For the time being this directive prevents the vendored package
to be in the same DAG as the one vendoring it.
2023-08-15 15:54:37 -07:00
Massimiliano Culpo
2da34de519 Add "^" automatically for named conflicts that don't refer to 'this' package
See https://github.com/spack/spack/pull/38447#discussion_r1285291520
2023-08-15 15:54:37 -07:00
Massimiliano Culpo
d237430f47 Inline a few functions that are not needed anymore 2023-08-15 15:54:37 -07:00
Massimiliano Culpo
3f0adae9ef Remove the need for "node_regex" 2023-08-15 15:54:37 -07:00
Massimiliano Culpo
3b4d7bf119 Rename method: "root_node" -> "main_node" 2023-08-15 15:54:37 -07:00
Massimiliano Culpo
b3087b32c6 Rename const: "root_node_id" -> "main_node_id" 2023-08-15 15:54:37 -07:00
Massimiliano Culpo
ad9c90cb2e Rename atom: "special_case" -> "multiple_nodes_attribute" 2023-08-15 15:54:37 -07:00
Massimiliano Culpo
1b0e113a9d Rename atom: "facts" -> "pkg_fact" 2023-08-15 15:54:37 -07:00
Massimiliano Culpo
6df5738482 Simplify "node_has_variant" internal atom. 2023-08-15 15:54:37 -07:00
Massimiliano Culpo
927d831612 Removed leftover TODOs 2023-08-15 15:54:37 -07:00
Massimiliano Culpo
1de5117ef1 Improve handling of cases with cycles
To avoid paying the cost of setup and of a full grounding again,
move cycle detection into a separate program and check first if
the solution has cycles.

If it has, ground only the integrity constraint preventing cycles
and solve again.
2023-08-15 15:54:37 -07:00
Massimiliano Culpo
006e69265e Optimize grounding of "can_inherit_flags" 2023-08-15 15:54:37 -07:00
Massimiliano Culpo
eaec3062a1 Fix computation of max nodes 2023-08-15 15:54:37 -07:00
Massimiliano Culpo
d5eb5106b0 Add unit-tests for use cases requiring separate concretization of build deps 2023-08-15 15:54:37 -07:00
Massimiliano Culpo
9f8edbf6bf Add a new configuration option to select among different concretization modes
The "concretizer" section has been extended with a "duplicates:strategy"
attribute, that can take three values:

- "none": only 1 node per package
- "minimal": allow multiple nodes opf specific packages
- "full": allow full duplication for a build tool
2023-08-15 15:54:37 -07:00
Massimiliano Culpo
a4301badef Fix a few bugs in the encoding when imposing constraints on build deps only
These bugs would show up when we try to split nodes by
imposing different targets or different compilers to all
build dependencies.
2023-08-15 15:54:37 -07:00
Massimiliano Culpo
4565811556 Construct unification sets on demand, improve heuristic 2023-08-15 15:54:37 -07:00
Massimiliano Culpo
b94d54e4d9 Reduce the number of unification sets to only two 2023-08-15 15:54:37 -07:00