Commit graph

6469 commits

Author SHA1 Message Date
Massimiliano Culpo
ba1d295023
Extract prefix locks and failure markers from Database (#39024)
This PR extracts two responsibilities from the `Database` class:
1. Managing locks for prefixes during an installation
2. Marking installation failures

and pushes them into their own class (`SpecLocker` and `FailureMarker`). These responsibilities are also pushed up into the `Store`, leaving to `Database` only the duty to manage `index.json` files.

`SpecLocker` classes no longer share a global list of locks, but locks are per instance. Their identifier is simply `(dag hash, package name)`, and not the spec prefix path, to avoid circular dependencies across Store / Database / Spec.
2023-08-07 06:47:52 -04:00
Harmen Stoppels
27f04b3544
Picklable HTTPError (#39285) 2023-08-07 10:48:35 +02:00
Harmen Stoppels
ef544a3b6d
Add a more detailed HTTPError (#39187) 2023-08-05 11:16:51 +02:00
Adam J. Stewart
4eed832653
py-pyqt6: add new package (#32696) 2023-08-04 18:49:54 -05:00
Harmen Stoppels
6e933ac7df
repo cache: use -inf default instead of 0 (#39214)
FastPackageChecker.modified_since should use a default number < 0

When the repo cache does not exist, Spack uses mtime 0. This causes the repo
cache not to be generated when the repo has mtime 0.

Some popular package managers such as spack use 0 mtime normalization for
reproducible tarballs. So when installing spack with spack from a buildcache, the
repo cache doesn't generate

Also add some typehints
2023-08-03 14:13:13 +02:00
Harmen Stoppels
2069a42ba3
Buildcache commands cleanup, again... (#39203)
* Inform mypy that tty.die is noreturn

* avoid temporary allocation in env

* update spack buildcache save-specfile

* fix spack buildcache check/download/get-buildcache-name

- ensure that required args and mutually exclusive ones are marked as
  such in argparse for better error messages
- deprecate --spec-file everywhere
- use disambiguate for better error messages
2023-08-03 10:44:02 +02:00
Adam J. Stewart
edbf12cfa8
Add qmake virtual provider (#38848) 2023-08-02 13:50:37 -05:00
kwryankrattiger
0b4631a774
CI: Refactor ci reproducer (#37088)
* CI: Refactor ci reproducer

* Autostart container
* Reproducer paths match CI paths
* Generate start scripts for docker and reproducer

* CI: Add interactive and gpg options to reproduce-build

* Interactive will determine if the docker container persists
  after running reproduction.
* GPG path/url allow downloading GPG keys needed for binary
  cache download validation. This is important for running
  reproducer for protected CI jobs.

* Add exit_on_failure option to CI scripts

* CI: Add runtime option for reproducer
2023-08-02 09:51:12 -07:00
Harmen Stoppels
e7fa6d99bf
version: move to module, avoid circular imports (#39077) 2023-08-02 17:47:08 +02:00
Harmen Stoppels
03c0d74139
buildcache extractall: extract directly into spec.prefix (#37441)
- Run `mkdirp` on `spec.prefix`
- Extract directly into `spec.prefix`
  1. No need for `$store/tmp.xxx` where we extract the tarball directly, pray that it has one subdir `<name>-<version>-<hash>`, and then `rm -rf` the package prefix followed by `mv`.
  2. No need to clean up this temp dir in `spack clean`.
  3. Instead figure out package directory prefix from the tarball contents, and strip the tarinfo entries accordingly (kinda like tar --strip-components but more strict)
- Set package dir permissions
- Don't error during error handling when files cannot removed
- No need to "enrich" spec.json with this tarball-toplevel-path

After this PR, we can in fact tarball packages relative to `/` instead of `spec.prefix/..`, which makes it possible to use Spack tarballs as container layers, where relocation is impossible, and rootfs tarballs are expected.
2023-08-02 17:06:13 +02:00
Wouter Deconinck
a14f4b5a02
feat: move -N/--namespace(s) to common args, allow in buildcache list (#36719)
`spack buildcache list` did not have a way to display the namespace of
packages in the buildcache. This PR adds that functionality.

For consistency's sake, it moves the `-N/--namespace` arg definition to
the `common/arguments.py` and modifies `find`, `solve`, `spec` to use
the common definition.

Previously, `find` was using `--namespace` (singular) to control whether
to display the namespace (it doesn't restrict the search to that
namespace). The other commands were using `--namespaces` (plural). For
backwards compatibility and for consistency with `--deps`, `--tags`,
etc, the plural `--namespaces` was chosen. The argument parser ensures
that `find --namespace` will continue to behave as before.

Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
2023-08-02 11:16:14 +00:00
Chris White
8d2a32f66d
remove CMAKE_GENERATOR from the host-config because it cannot be overwritten on the command line (#39044) 2023-08-01 08:57:56 -07:00
Alec Scott
d4f41b51f4
Add spack checksum --verify, fix --add (#38458)
* Add rewrite of spack checksum to include --verify and better add versions to package.py files
* Fix formatting and remove unused import
* Update checksum unit-tests to correctly test multiple versions and add to package
* Remove references to latest in stage.py
* Update bash-completion scripts to fix unit tests failures
* Fix docs generation
* Remove unused url_dict argument from methods
* Reduce chance of redundant remote_versions work
* Add print() before tty.die() to increase error readablity
* Update version regular expression to allow for multi-line versions
* Add a few unit tests to improve test coverage
* Update command completion
* Add type hints to added functions and fix a few py-lint suggestions
* Add @no_type_check to prevent mypy from failing on pkg.versions
* Add type hints to format.py and fix unit test
* Black format lib/spack/spack/package_base.py
* Attempt ignoring type errors
* Add optional dict type hint and declare versions in PackageBase
* Refactor util/format.py to allow for url_dict as an optional parameter
* Directly reference PackageBase class instead of using TypeVar
* Fix comment typo

---------

Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
2023-07-31 21:49:43 +00:00
Adam J. Stewart
9394fa403e
Remove Xcode mock-up (#39020)
* Remove Xcode mock-up

* Remove unused imports
2023-07-31 10:24:04 -07:00
Scott Wittenburg
2c74b433aa
ci: Make signing requirement explicit (#38995)
Instead of inferring whether to sign binaries, make it explicit,
and fail rebuild jobs early if signing is required but cannot be
accomplished.
2023-07-26 09:16:15 -06:00
Jonathon Anderson
3ad65bbfc1
Always set workflow:rules in spack ci pipelines (#38921) 2023-07-24 14:31:39 -05:00
Ashwin Kumar Karnad
399271832b
Fix typo: binary_caches.rst (#39064) 2023-07-24 15:09:34 +02:00
Harmen Stoppels
4bcceddba9
Don't add .spack/binary_distribution twice to the tarball when re-distributing (#39042)
Previously, spack would list the ./spack/binary_distribution file twice when pushing
a package that was installed from a binary tarball itself.
2023-07-24 14:39:37 +02:00
Harmen Stoppels
0fff219aa4
Fix broken sanitize_file_path (#38926)
The sanitization function is completely bogus as it tries to replace /
on unix after ... splitting on it. The way it's implemented is very
questionable: the input is a file name, not a path. It doesn't make
sense to interpret the input as a path and then make the components
valid -- you'll interpret / in a filename as a dir separator.

It also fails to deal with path components that contain just unsupported
characters (resulting in empty component).

The correct way to deal with this is to have a function that takes a
potential file name and replaces unsupported characters.

I'm not going to fix the other issues on Windows, such as reserved file
names, but left a note, and hope that @johnwparent can fix that
separately.

(Obviously we wouldn't have this problem at all if we just fixed the
filename in a safe way instead of trying to derive something from
the url; we could use the content digest when available for example)
2023-07-24 12:30:47 +02:00
百地 希留耶
90ac0ef66e
Implement fish completion (#29549)
* commands: provide more information to Command

* fish: Add script to generate fish completion

* fish: auto prepend `spack` command to avoid duplication

* fish: impove completion generation code readability

* commands: replace match-case with if-else

* fish: fix optspec variable name prefix

* fish: fix return value in get_optspecs

* fish: fix return value in get_optspecs

* format: split long line and trim trailing space

* bugfix: replace f-string with interpolation

* fish: compete more specs and some fixes

* fish: complete hash spec starts with /

* fish: improve compatibility

* style: trim trailing whitespace

* commands: add fish to update args and update tests

* commands: add fish completion file

* style: merge imports

* fish: source completion in setup-env

* fish: caret only completes dependencies

* fish: make sure we always get same order of output

* fish: spack activate
only show installed packages that have extensions

* fish: update completion file

* fish: make dict keys sorted

* Blacken code

* Fix bad merge

* Undo style changes to setup-env.fish

* Fix unit tests

* Style fix

* Compatible with fish_indent

* Use list for stability of order

* Sort one more place

* Sort more things

* Sorting unneeded

* Unsort

* Print difference

* Style fix

* Help messages need quotes

* Arguments to -a must be quoted

* Update types

* Update types

* Update types

* Add type hints

* Change order of positionals

* Always expand help

* Remove shared base class

* Fix type hints

* Remove platform-specific choices

* First line of help only

* Remove unused maps

* Remove suppress

* Remove debugging comments

* Better quoting

* Fish completions have no double dash

* Remove test for deleted class

* Fix grammar in header file

* Use single quotes in most places

* Better support for remainder nargs

* No magic strings

* * and + can also complete multiple

* lower case, no period

---------

Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
2023-07-22 08:55:12 -05:00
Massimiliano Culpo
c0b42151c3
Remove spack.repo.IndexError (#39029)
This exception is never used and
overrides a built-in.
2023-07-21 15:33:30 +02:00
Ashwin Kumar Karnad
405f563909
binary_caches.rst: fix typo (#39030) 2023-07-21 10:39:53 +02:00
Sergey Kosukhin
d2566e3d62
nag: update the versioning scheme (#35457)
* nag: append build number to the version
* nag: add version 7.1.7125
* nag: deprecate unavailable versions
2023-07-20 12:03:26 -07:00
Massimiliano Culpo
50b90e430d
spack.util.lock: add type-hints, remove **kwargs in method signatures (#39011) 2023-07-20 09:41:23 +02:00
Massimiliano Culpo
3a565c66e9
Respect custom user store when bootstrapping (#39001)
The user store is lazily evaluated. The change
in #38975 made it such that the first evaluation
was happening in the middle of swapping to user
configuration.

Ensure we construct the user store before that.
2023-07-19 19:53:33 -04:00
Xavier Delaruelle
d9fbdfbee9
modules: use curly braces to enclose value in Tcl modulefile (#38375)
Use curly braces instead of quotes to enclose value or text in Tcl
modulefile. Within curly braces Tcl special characters like [, ] or $
are treated verbatim whereas they are evaluated within quotes.

Curly braces is Tcl recommended way to enclose verbatim content [1].

Note: if curly braces charaters are used within content, they must be
balanced. This point has been checked against current repository and no
unbalanced curly braces has been spotted.

Fixes #24243

[1] https://wiki.tcl-lang.org/page/Tcl+Minimal+Escaping+Style
2023-07-19 17:57:37 +02:00
Harmen Stoppels
185bccb70f
Fetch & patch: actually acquire stage lock, and many more issues (#38903)
* Fetching patches wouldn't result in acquiring a stage lock during install
* The installer would acquire a stage lock *after* fetching instead of
   before, leading to races
* The name of the stage for patches was random, so on build failure
   (where stage dirs are not removed), these directories would continue
   to exist after a second successful install.
* There was this redundant "composite fetch" object -- there's already
   a composite stage. Remove this.
* For some reason we do *double* shasum validation of patches, before
   and after compression -- that's just too much? I removed it.
2023-07-19 15:06:56 +02:00
Harmen Stoppels
886946395d
drop redundant rpaths post install (#38976)
Spack heuristically adds `<install prefix>/lib` and `<install prefix>/lib64` as rpath entries, as it doesn't know what the install dir is going to be ahead of the build. This PR cleans up non-existing, absolute paths[^1], which

1. avoids redundant stat calls at runtime
2. drops redundant rpaths in `patchelf`, making it relocatable -- you don't need patchelf recursively then.

[^1]: It also removes relative paths not starting with `$` (so, `$ORIGIN/../lib` is retained -- we _could_ interpolate `$ORIGIN`, but that's hard to get right when symlinks have to be taken into account). Relative paths _are_ supported in glibc, but are relative to _the current working directory_, which is madness, and it would be better to drop those paths.
2023-07-19 09:48:31 +00:00
Massimiliano Culpo
f34c93c5f8
llnl.util.lock: add type-hints (#38977)
Also uppercase global variables in the module
2023-07-19 11:23:08 +02:00
Massimiliano Culpo
a7f2abf924
Remove LazyReference from code (#38944)
A LazyReference object is a reference to an attribute of a 
lazily evaluated singleton. Its only purpose is to let developers
use shorter names to refer to such attribute.

This class does more harm than good, as it obfuscates the fact
that we are using the attribute of a global object. Also, it can easily
go out of sync with the singleton it refers to if, for instance, the
singleton is updated but the references are not.

This commit removes the LazyReference class entirely, and access
the attributes explicitly passing through the global value to which
they are attached.
2023-07-19 11:08:51 +02:00
markus-ferrell
416edfa229
Windows testing: enable tests for installer components (#36970)
These tests now work without any changes to core. Furthermore, it is
surprising that they had to be disabled (at least, as long as the
installer.py tests are run on Windows: these tests are more-basic
and their functionality would have been exercised automatically).
2023-07-18 16:19:14 -07:00
Massimiliano Culpo
14f3297cca
Ensure the bootstrap store has a padding length of zero (#38975)
Without this PR, padded length was propagating from user configuration to
bootstrap configuration, and was causing the issue reported in #38963
2023-07-18 23:49:22 +02:00
markus-ferrell
f24f98a1e2
Windows testing: enable bootstrap test (#36972) 2023-07-18 14:42:26 -07:00
markus-ferrell
ffc283ab8b
test_clear_failures_success: run on Windows too (#36792) 2023-07-18 22:42:57 +02:00
Harmen Stoppels
3fef586cfb
binary cache docs: remove redundant flag and comment (#38960) 2023-07-18 22:38:04 +02:00
markus-ferrell
b710778bda
Windows testing: enable architecture test (#36973)
Works out of the box: remove skip.
2023-07-18 12:33:52 -07:00
markus-ferrell
a965fe9354
Windows testing: enable "spack clean" tests (#36840)
They work out out of the box on windows. Simply removing skips.
2023-07-18 12:25:32 -07:00
Harmen Stoppels
5b23c5dcc0
buildcache push: make --allow-root the default and deprecate the option (#38878)
Without --allow-root spack cannot push binaries that contain paths in
binaries. This flag is almost always needed, so there is no point of
requiring users to spell it out. 

Even without --allow-root, rpaths would still have to be patched, so the 
flag is not there to guarantee binaries are not modified on install.

This commit makes --allow-root the default, and drops the code 
required for it. It also deprecates `spack buildcache preview`, since 
the command made sense only with --allow-root.

As a side effect, Spack no longer depends on binutils for relocation
2023-07-18 18:45:14 +02:00
Massimiliano Culpo
ad1fdcdf48
Pin Spack dev dependencies on RtD (#38950) 2023-07-18 18:37:04 +02:00
Massimiliano Culpo
82aa27f5a5
Fix default construction of locks (#38953)
This fixes a typo introduced in a refactor
2023-07-18 14:36:41 +02:00
Xavier Delaruelle
8c7adbf8f3
modules: add support for conflict in lua modulefile (#36701)
Add support for conflict directives in Lua modulefile like done for Tcl
modulefile.

Note that conflicts are correctly honored on Lmod and Environment
Modules <4.2 only if mutually expressed on both modulefiles that
conflict with each other.

Migrate conflict code from Tcl-specific classes to the common part. Add
tests for Lmod and split the conflict test case in two.

Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2023-07-18 10:24:46 +02:00
Peter Scheibel
10165397da
"spack develop": always pull full history for git repos (#38343) 2023-07-18 09:53:33 +02:00
Dan Lipsa
4831d45852
Decompression: fix naming issues (#37749)
* When using system tools to unpack a .gz file, the input file needs a
  different name than the output file. Normally, we generate this new
  name by stripping off the .gz extension off of the file name.
  This was not sufficient if the file name did not have an extension,
  so we temporarily rename the file in that case.
* When using system tar utility to untar on Windows, we were (erroneously)
  skipping the actual untar step if the filename was lacking a .tar
  extension
* For foo.txz, we were not changing the extension of the decompressed file
  (i.e. we would decompress foo.txz to foo.txz). This did not cause any
  problems, but is confusing, so has been updated such that the output
  filename reflects its decompressed state (i.e. foo.tar).
* Added test for strip_compression_extension
* Update test_native_unpacking to test each archive type with and without
  an extension as part of the file name (i.e. we test "foo.tar.gz", but
  also make sure we decompress properly if it is named "foo").
2023-07-17 14:33:18 -07:00
Harmen Stoppels
f05837a480
Fix wrong StageComposite keep override (#38938)
`Stage(keep=True)` was ignored when put in a composite that doesn't
override the value.
2023-07-17 23:20:24 +02:00
Manuela Kuhn
884b4952af
Fix python import tests (#38928)
Running `spack test run <python package>` resulted in the error
```
'str' object is not callable
```
because the python executable was not set correctly.
2023-07-17 13:19:47 -07:00
Massimiliano Culpo
53ae969aa0
Lock, database and store don't need global configuration on construction (#33495)
Lock objects can now be instantiated independently,
without being tied to the global configuration. The
same is true for database and store objects.

The database __init__ method has been simplified to
take a single lock configuration object. Some common
lock configurations (e.g. NO_LOCK or NO_TIMEOUT) have
been named and are provided as globals.

The use_store context manager keeps the configuration
consistent by pushing and popping an internal scope.
It can also be tuned by passing extra data to set up
e.g. upstreams or anything else that might be related
to the store.
2023-07-17 16:51:04 +02:00
Adam J. Stewart
2b5a7bb4d7
Update new PythonPackage template to prefer --config-settings (#38918) 2023-07-17 08:03:05 -05:00
Peter Scheibel
31431f967a
Environment/depfile: fix bug with Git hash versions (attempt #2) (#37560)
Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
2023-07-17 11:17:32 +00:00
Massimiliano Culpo
63b88c4b75
Minimal cleanup of a few tests in test/packaging.py (#38880)
* Minimal cleanup of a few tests in packaging.py

* Use f-strings
2023-07-17 10:36:29 +02:00
Harmen Stoppels
ac5976d17d
Remove unused context manager (#38897) 2023-07-14 18:41:30 +02:00
Rocco Meli
244dfb3a35
Fix issue on cray with super call (#38895) 2023-07-14 10:00:02 +00:00
Tamara Dahlgren
2e9e7ce7c4
Bugfix/spack spec: read and use the environment concretizer:unification option (#38248)
* Bugfix: spack.yaml concretizer:unify needs to be read and used
* Optional: add environment test to ensure configuration scheme is used
* Activate environment in unit tests
  A more proper solution would be to keep
  an environment instance configuration as
  an attribute, but that is a bigger refactor
* Delay evaluation of Environment.unify
* Slightly simplify unit tests

---------

Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2023-07-13 23:43:20 +02:00
Harmen Stoppels
033eb77aa9
spack buildcache push: improve argparse (#38876) 2023-07-13 16:01:09 +02:00
Harmen Stoppels
522d9e260b
mirrors: distinguish between source/binary mirror; simplify schema (#34523)
Allow the following formats:

```yaml
mirrors:
  name: <url>
```

```yaml
mirrors:
  name:
    url: s3://xyz
    access_pair: [x, y]
```

```yaml
mirrors:
  name:
    fetch: http://xyz
    push:
      url: s3://xyz
      access_pair: [x, y]
```

And reserve two new properties to indicate the mirror type (e.g.
mirror.spack.io is a source mirror, not a binary cache)

```yaml
mirrors:
  spack-public:
    source: true
    binary: false
    url: https://mirror.spack.io
```
2023-07-13 11:29:17 +00:00
Massimiliano Culpo
3261889e3a
spack audit: allow skipping version checks from package.py (#28372)
A few packages have version directives evaluated
within if statements, conditional on the value of
`platform.platform()`.

Sometimes there are no cases for e.g. platform=darwin and that
causes a lot of spurious failures with version existence
audits.

This PR allows expressing conditions to skip version
existence checks in audits and avoid these spurious reports.
2023-07-13 06:47:47 -04:00
Harmen Stoppels
161b30a32f
Add type hints to spack.installer (#38872) 2023-07-13 10:41:19 +00:00
Adam J. Stewart
b67f1f395b
Add missing space in error msg (#38863) 2023-07-13 10:24:02 +02:00
Adam J. Stewart
bb7f437bf5
Standardize subcommand help strings (#38804)
### Rationale

While working on #29549, I noticed a lot of inconsistencies in our argparse help messages. This is important for fish where these help messages end up as descriptions in the tab completion menu. See https://github.com/spack/spack/pull/29549#issuecomment-1627596477 for some examples of longer or more stylized help messages.

### Implementation

This PR makes the following changes:

- [x] help messages start with a lowercase letter.
- [x] Help messages do not end with a period
- [x] the first line of a help message is short and simple

    longer text is separated by an empty line
- [x] "help messages do not use triple quotes" 

    """(except docstrings)"""
- [x] Parentheses not needed for string concatenation inside function call
- [x] Remove "..." "..." string concatenation leftover from black reformatting
- [x] Remove Sphinx argument docs from help messages

The first 2 choices aren't very controversial, and are designed to match the syntax of the `--help` flag automatically added by argparse. The 3rd choice is more up for debate, and is designed to match our package/module docstrings. The 4th choice is designed to avoid excessive newline characters and indentation. We may actually want to go even further and disallow docstrings altogether.

### Alternatives

Choice 3 in particular has a lot of alternatives. My goal is solely to ensure that fish tab completion looks reasonable. Alternatives include:

1. Get rid of long help messages, only allow short simple messages
2. Move longer help messages to epilog
3. Separate by 2 newline characters instead of 1
4. Separate by period instead of newline. First sentence goes into tab completion description

The number of commands with long help text is actually rather small, and is mostly relegated to `spack ci` and `spack buildcache`. So 1 isn't actually as ridiculous as it sounds.

Let me know if there are any other standardizations or alternatives you would like to suggest.
2023-07-13 00:18:23 -07:00
Jonathon Anderson
90208da8a5
containers: retain shallow git data (#37734) 2023-07-12 21:03:10 +00:00
Michael Fink
c8026c3c87
Add path to MPI executables to ^mpi dependents (#35758) 2023-07-12 13:58:43 -05:00
Vicente Bolea
37ef31dc22
vtk-m: correct cuda_arch variant behavior (#38697)
Co-authored-by: eugeneswalker <eugenesunsetwalker@gmail.com>
2023-07-12 14:34:50 +02:00
Michael Kuhn
d5d0b8821c
installer: Improve status reporting (#37903)
Refactor `TermTitle` into `InstallStatus` and use it to show progress
information both in the terminal title as well as inline. This also
turns on the terminal title status by default.

The inline output will look like the following after this change:
```
==> Installing m4-1.4.19-w2fxrpuz64zdq63woprqfxxzc3tzu7p3 [4/4]
```
2023-07-12 08:54:45 +02:00
Todd Gamblin
757f8ae59c
find: add --hashes shortcut for piping to other commands (#38663)
People frequently ask us how to pipe `spack find` output to other commands, and we tell
them to do things like this:

```console
$ spack find --format "/{hash}" | spack uninstall -ay
```

Sometimes users don't know about hash references and come up with potentially ambiguous
formulations like this:

```console
spack find --format {name}@{version}%{compiler} | spack uninstall -ay
```

Since this is a common enough thing to want to do, and to make it more obvious how, this
PR adds a `-H` / `--hashes` as a shortcut, so you can now just do:

```console
spack find -H | spack uninstall -ay
```
2023-07-10 09:43:37 +02:00
Jonathon Anderson
db879a5679
ci: Fix broken SPACK_CHECKOUT_VERSION (#38778) 2023-07-09 12:37:36 -07:00
Harmen Stoppels
374fda1063
Don’t call spec.format in Database._get_matching_spec_key (#38792)
`"%s" % spec` formats the spec with deps included, which produces sometimes KBs
of data and is slow to run in pure Python. It can delay otherwise very short-lived
read/write locks on the database.

Discovered in #38762 where profile output showed about 2 seconds is spent in
`spec.format`, which is significant overhead when using multiprocessing to install
from binary cache in parallel (installation often takes <5s for small packages). With
this change, `spec.format` no longer shows up in profile output.

(This line hasn't changed since Spack v0.9 ;p)

* move format() call to custom NoSuchSpecError exception
* add a comment saying why, so we can eventually change `Spec.__str__`
2023-07-09 11:27:38 -04:00
Massimiliano Culpo
6958b49c2f
Remove "node_compiler" from the list of unknown atoms (#38753) 2023-07-07 13:19:53 +02:00
Harmen Stoppels
a1d33e97ec
Fix multiple quadratic complexity issues in environments (#38771)
1. Fix O(n^2) iteration in `_get_overwrite_specs`
2. Early exit `get_by_hash` on full hash
3. Fix O(n^2) double lookup in `all_matching_specs` with hashes
4. Fix some legibility issues
2023-07-07 10:51:58 +00:00
Massimiliano Culpo
ca9b52bbc5
Prevent "spack external find" to error out on wrong permissions (#38755)
fixes #38733
2023-07-07 12:05:32 +02:00
Andrey Parfenov
ae00d7c358
add info about spack env from spack-configs for oneAPI build tools (#38751)
Signed-off-by: Andrey Parfenov <andrey.parfenov@intel.com>
2023-07-07 03:47:54 +00:00
Adam J. Stewart
45838cee0b
Drop Python 2 super syntax (#38718) 2023-07-05 09:04:29 -05:00
Adam J. Stewart
95847a0b37
Drop Python 2 object subclassing (#38720) 2023-07-05 14:37:44 +02:00
Peter Scheibel
f365386447
Installations: don't set group permissions when they match what is desired (#38036)
* When installing a package Spack will attempt to set group permissions on
the install prefix even when the configuration does not specify a group.

Co-authored-by: David Gomez <dvdgomez@users.noreply.github.com>
2023-07-05 09:54:04 +02:00
Adam J. Stewart
2978911520
spack commands: add type hints and docstrings (#38705) 2023-07-04 16:43:02 -04:00
Harmen Stoppels
08f23f4802
macos sip: apply on macos only, dont store LD_LIBRARY_PATH (#38687) 2023-07-04 10:54:13 +02:00
Adam J. Stewart
5bd7a0c563
spack make-installer: deterministic choice order (#38706) 2023-07-04 09:39:38 +02:00
Adam J. Stewart
33c5959e23
Remove from __future__ imports (#38703) 2023-07-04 08:30:29 +02:00
Adam J. Stewart
63576275be
SIPPackage: documentation fix (#38672) 2023-07-02 11:30:08 +02:00
Scott Wittenburg
af5b93bb97
gitlab ci: reorganize when we check for specs on mirrors (#38626)
Move the logic checking which mirrors have the specs we need closer
to where that information is needed.  Also update the staging summary
to contain a brief description of why we scheduled or pruned each
job.  If a spec was found on any mirrors, regardless of whether
we scheduled a job for it, print those mirrors.
2023-06-30 10:18:16 -06:00
Tamara Dahlgren
dc25da1931
tests/pythons: convert to new stand-alone test process (#38340) 2023-06-30 05:52:42 -04:00
John W. Parent
78f33bc002
Windows: Add PowerShell env support (#37951)
PowerShell requires explicit shell and env support in Spack.
This is due to the distinct differences in shell interactions between
cmd and pwsh. Add a doskey in pwsh piping 'spack' commands to a
powershell script similar to the sh function 'spack'. Add
support for PowerShell-specific shell interactions from Spack
(set/unset shell variables).
2023-06-27 18:26:51 -07:00
Scott Wittenburg
b6b33cfe7a
ci: remove compiler bootstrapping code (#38543) 2023-06-27 10:35:19 -05:00
Wileam Y. Phan
e24bf70af4
Adjust cc90 to require at least CUDA 12 (#38572) 2023-06-27 08:24:12 -04:00
Harmen Stoppels
ab10b645c6
Revert "Updates to symlinking for Windows (#34701)" (#38578)
This reverts commit 66f75407d1.
2023-06-27 09:46:08 +02:00
Brian Van Essen
e79f275bc9
CachedCMakePackage: set build type (#38502)
Fixed the cached CMake package so that the build_type field is saved
in the cached configuration file.
2023-06-26 18:25:46 -07:00
David Alexander
66f75407d1
Updates to symlinking for Windows (#34701)
* Support hardlinks/junctions on Windows systems without developer
  mode enabled
  * Generally, use of llnl.util.symlink.symlink is preferred over
    os.symlink since it handles this automatically
* Generally an error is now reported if a user attempts to create a
  symlink to a file that does not exist (this was previously allowed
  on Linux/Mac).
  * One exception to this: when Spack installs files from the source
    into their final prefix, dangling symlinks are allowed (on
    Linux/Mac - Windows does not allow this in any circumstance).
    The intent behind this is to avoid generating failures for
    installations on Linux/Mac that were succeeding before.
* Because Windows is strict about forbidding dangling symlinks,
  `traverse_tree` has been updated to skip creating symlinks if they
  would point to a file that is ignored. This check is not
  transitive (i.e., a symlink to a symlink to an ignored file would
  not be caught appropriately)
* Relocate function: resolve_link_target_relative_to_the_link
  (this is not otherwise modified)

Co-authored-by: jamessmillie <smillie@txcorp.com>
2023-06-26 13:38:14 -07:00
Xavier Delaruelle
1dcc67535a
modules: ignore more Modules variables in from_sourcing_file (#38455)
Update list of excluded variables in `from_sourcing_file` function to
cover all variables specific to Environment Modules or Lmod. Add
specifically variables relative to the definition of `module()`, `ml()`
and `_module_raw()` Bash functions.

Fixes #13504
2023-06-26 20:21:46 +02:00
Xavier Delaruelle
8164712264
Add raw attribute to env.set command (#38465)
Update `env.set` command and underlying `SetEnv` object to add the `raw`
boolean attribute. `raw` is optional and set to False by default. When
set to True, value format is skipped for object when generating
environment modifications.

With this change it is now possible to define environment variable
whose value contains variable reference syntax (like `{foo}` or `{}`)
that should be set as-is.

Fixes #29578
2023-06-26 11:55:57 +02:00
Harmen Stoppels
cd67b2a1a9
spack buildcache sync (--manifest-glob <glob> | <src> <dest>) are mutually exclusive (#38533) 2023-06-23 12:20:48 +02:00
Andrey Parfenov
7dc485d288
cc: Ensure that user-specified flags take precedence over others (#37376)
Spack flags supplied by users should supersede flags from package build systems and
other places in Spack.  However, Spack currently adds user-supplied flags to the 
beginning of the compile line, which means that in some cases build system flags will
supersede user-supplied ones.

The right place to add a flag to ensure it has highest precedence for the compiler really
depends on the type of flag.  For example, search paths like `-L` and `-I` are examined
in order, so adding them first is highest precedence.  Compilers take the *last* occurrence
of optimization flags like `-O2`, so those should be placed *after* other such flags.  Shim
libraries with `-l` should go *before* other libraries on the command line, so we want
user-supplied libs to go first, etc.

`lib/spack/env/cc` already knows how to split arguments into categories like `libs_list`,
`rpath_dirs_list`, etc., so we can leverage that functionality to merge user flags into
the arg list correctly.

The general rules for injected flags are:

1. All `-L`, `-I`, `-isystem`, `-l`, and `*-rpath` flags from `spack_flags_*` to appear
   before their regular counterparts.
2. All other flags ordered with the ones from flags after their regular counterparts,
   i.e. `other_flags` before `spack_flags_other_flags`

- [x] Generalize argument categorization into its own function in the `cc` shell script
- [x] Apply the same splitting logic to injected flags and flags from the original compile line.
- [x] Use the resulting flag lists to merge user- and build-system-supplied flags by category.
- [x] Add tests.

Signed-off-by: Andrey Parfenov <andrey.parfenov@intel.com>

Co-authored-by: iermolae <igor.ermolaev@intel.com>
2023-06-18 14:07:08 -07:00
simonleary-umass-edu
5c6c3b403b
fix oneapi modules (#38400) 2023-06-18 07:17:19 -04:00
Greg Becker
0ac1c52d17
unparser: drop Python 2, fix testing bugs with newer Pythons (#38424)
The `unparser` that Spack uses for package hashing had several tweaks to ensure compatibility
with Python 2.7:
1. Currently, the unparser automatically moves `*` and `**` args to the end to preserve
   compatibility with `python@:3.4`
2. `print a, b, c` statements and single-tuple `print((a, b, c))` function calls were
   remapped to `print(a, b, c)` in the unparsed output for consistency across versions.

(1) is causing issues in our tests because a recent patch to the Python source code  
(https://github.com/python/cpython/pull/102953/files#diff-7972dffec6674d5f09410c71766ac6caacb95b9bccbf032061806ae304519c9bR813-R823) 
has a `**` arg before an named argument, and we round-trip the core python source code
as a test of our unparser.  This isn't actually a break with our consistent unpausing -- it's still
consistent, the python source just doesn't unparse to the same thing anymore. It does makes
it harder to test, so it's not worth maintaining the Python2-specific stuff anymore.

Since we only support `python@3.6:`, this PR removes (1) and (2) from the unparser, but keeps
one last tweak for unicode AST inconsistencies, as it's still needed for Python 3.5-3.7.

This fixes the CI error we've been seeing on `python@3.11.4` and `python@3.10.12`. Again, that
bug exists only in the test system and doesn't affect our canonical hashing of Python code.
2023-06-16 20:52:26 -04:00
Greg Becker
99f3b9f064
show external status as [e] (#33792) 2023-06-16 18:22:28 +02:00
Gurkirat Singh
69a5c55702
docs: add quotes around some values in a YAML example (#38412) 2023-06-16 10:29:09 +02:00
markus-ferrell
1df4afb53f
Enable build system guess tests on windows (#36971) 2023-06-15 11:40:09 -07:00
markus-ferrell
09fd7d68eb
Windows testing: enable graph, mark, and info cmd tests (#36977) 2023-06-15 11:28:52 -07:00
Massimiliano Culpo
f27d012e0c
Add virtual information on DAG edges (#34821)
* DependencySpec: add virtuals attribute on edges

This works for both the new and the old concretizer. Also,
added type hints to involved functions.

* Improve virtual reconstruction from old format

* Reconstruct virtuals when reading from Cray manifest

* Reconstruct virtual information on test dependencies
2023-06-15 07:16:54 -07:00
Xavier Delaruelle
2db09f27af
modules: use depends-on to autoload module with Lmod on Tcl (#38347)
Update Tcl modulefile template to use the `depends-on` command to
autoload modules if Lmod is the current module tool.

Autoloading modules with `module load` command in Tcl modulefile does
not work well for Lmod at some extend. An attempt to unload then load
designated module is performed each time such command is encountered. It
may lead to a load storm that may not end correctly with large number of
module dependencies.

`depends-on` command should be used for Lmod instead of `module load`,
as it checks if module is already loaded, and does not attempt to reload
this module.

Lua modulefile template already uses `depends_on` command to autoload
dependencies. Thus it is already considered that to use Lmod with Spack,
it must support `depends_on` command (version 7.6+).

Environment Modules copes well with `module load` command to autoload
dependencies (version 3.2+). `depends-on` command is supported starting
version 5.1 (as an alias of `prereq-all` command) which was relased last
year.

This change introduces a test to determine if current module tool that
evaluates modulefile is Lmod. If so, autoload dependencies are defined
with `depends-on` command. Otherwise `module load` command is used.

Test is based on `LMOD_VERSION_MAJOR` environment variable, which is set
by Lmod starting version 5.1.

Fixes #36764
2023-06-14 15:35:53 +02:00
dependabot[bot]
6979d6a96f
build(deps): bump python-levenshtein in /lib/spack/docs (#38339)
Bumps [python-levenshtein](https://github.com/maxbachmann/python-Levenshtein) from 0.21.0 to 0.21.1.
- [Release notes](https://github.com/maxbachmann/python-Levenshtein/releases)
- [Changelog](https://github.com/maxbachmann/python-Levenshtein/blob/main/HISTORY.md)
- [Commits](https://github.com/maxbachmann/python-Levenshtein/compare/v0.21.0...v0.21.1)

---
updated-dependencies:
- dependency-name: python-levenshtein
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-14 13:03:15 +02:00
Dan Lipsa
4648939043
Windows bugfix: path-to-URL conversion (#37827)
When interpreting local paths as relative URL endpoints, they were
formatted as Windows paths on Windows (i.e. with '\'). URLs should
always be POSIX-style.
2023-06-13 10:08:09 -07:00
Xavier Delaruelle
746eaaf01a
modules: append trailing delimiter to MANPATH when set (#36678)
Update modulefile templates to append a trailing delimiter to MANPATH
environment variable, if the modulefile sets it.

With a trailing delimiter at ends of MANPATH's value, man will search
the system man pages after searching the specific paths set.

Using append-path/append_path to add this element, the module tool
ensures it is appended only once. When modulefile is unloaded, the
number of append attempt is decreased, thus the trailing delimiter is
removed only if this number equals 0.

Disclaimer: no path element should be appended to MANPATH by generated
modulefiles. It should always be prepended to ensure this variable's
value ends with the trailing delimiter.

Fixes #11355.
2023-06-13 10:29:11 +02:00
Peter Scheibel
45fbb82d1a
pip is a pythonextension not a pythonpackage, and it turns out we werent doing our external surgery on things that inherited pythonextension (#38186) 2023-06-12 17:07:28 +02:00
Harmen Stoppels
da45073ef9
buildcache: remove deprecated api (#37246)
The API was deprecated in v0.20 and is slated for removal in v0.21
2023-06-12 14:33:26 +02:00
Massimiliano Culpo
c8d95512fc
bootstrap: add minimum versions required for flake8 and pytest (#38322) 2023-06-12 12:29:35 +02:00
Christopher Christofi
7fa3c7f0fa
Fix typo in example code block (#38318) 2023-06-11 22:12:36 -04:00
dependabot[bot]
7896625919
build(deps): bump urllib3 from 2.0.2 to 2.0.3 in /lib/spack/docs (#38244)
Bumps [urllib3](https://github.com/urllib3/urllib3) from 2.0.2 to 2.0.3.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/2.0.2...2.0.3)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-11 13:35:22 -05:00
dependabot[bot]
fb43cb8166
build(deps): bump sphinx-rtd-theme in /lib/spack/docs (#38245)
Bumps [sphinx-rtd-theme](https://github.com/readthedocs/sphinx_rtd_theme) from 1.2.1 to 1.2.2.
- [Changelog](https://github.com/readthedocs/sphinx_rtd_theme/blob/master/docs/changelog.rst)
- [Commits](https://github.com/readthedocs/sphinx_rtd_theme/compare/1.2.1...1.2.2)

---
updated-dependencies:
- dependency-name: sphinx-rtd-theme
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-11 13:35:03 -05:00
Greg Becker
3c672905d0
install status arg: update to default true and add inverse (#38252)
* install status arg: update to default true and add inverse

* update bash completion

* fix tests broken by output change
2023-06-09 10:38:05 -07:00
Massimiliano Culpo
ddfc43be96
Forbid using env: as a top level environment attribute (#38199)
* Remove "env" from environment schema

* Remove spack.env.schema.keys

* Remove spack.environment.config_dict
2023-06-08 14:34:17 -04:00
John W. Parent
06817600e4
CMake/Windows bugfix: Make CMAKE_INSTALL_PREFIX a posix path (#36842)
CMake gives off a warning when passed Windows style paths as
install prefixes as the resultant path often causes invalid
escape sequences.
2023-06-08 10:05:38 -07:00
Massimiliano Culpo
f29aab0d03
Fix compiler removal from command line (#38057)
* Improve lib/spack/spack/test/cmd/compiler.py

* Use "tmp_path" in the "mock_executable" fixture

* Return a pathlib.Path from mock_executable

* Fix mock_executable fixture on Windows

"mock_gcc" was very similar to mock_executable, so use the latter to reduce code duplication

* Remove wrong compiler cache, fix compiler removal

fixes #37996

_CACHE_CONFIG_FILES was both unneeded and wrong, if called
subsequently with different scopes.

Here we remove that cache, and we fix an issue with compiler
removal triggered by having the same compiler spec in multiple
scopes.
2023-06-08 09:38:20 -07:00
Christopher Kotfila
785c1a2070
Add documentation about package signing model (#30939)
Co-authored-by: Scott Wittenburg <scott.wittenburg@kitware.com>
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2023-06-07 23:05:44 +02:00
eugeneswalker
5f1bc15e80
E4S Cray CI Stack (#37837)
* e4s cray ci stack

* e4s ci: add cray

* add zen4 tag

* WIP: new defintions just for cray

* updates

* remove ci signing job overrride, not necessary

* echo $PATH and show modules loaded

* add mirror

* add external def for cray-libsci

* comment out quantum-espresso

* use /etc/protected-runner as key path

* cray ci stack: do not remove tags: [spack, public]

* make cray stack composable

* generate job should run on public tagged runner, override default config:install_tree:root

* CI: Use relative path in default script

* CI: Use relative includes paths for shell runners

* Use concrete_env_dir for relpath

* ml-darwin-aarch64-mps: jax has bazel codesign issue

---------

Co-authored-by: Scott Wittenburg <scott.wittenburg@kitware.com>
Co-authored-by: Ryan Krattiger <ryan.krattiger@kitware.com>
2023-06-06 18:41:32 -07:00
Mikael Simberg
fa9fb60df3
CachedCMakePackage: fix bug where CMAKE_CUDA_ARCHITECTURES=none is set (#38169)
#37592 updated cached cmake packages to set CMAKE_CUDA_ARCHITECTURES.
The condition `if archs != "none"` lead to `CMAKE_CUDA_ARCHITECTURES=none`
when cuda_arch=none (incorrect check on the value of a multi-valued
variant), i.e. CMAKE_CUDA_ARCHITECTURES is always set. This PR udpates
the condition to if archs[0] != "none" to ensure CMAKE_CUDA_ARCHITECTURES
is only set if cuda_arch is not none (which seems to be the pattern used
in other packages).

This does the same for HIP (although in general ROCmPackage disallows
amdgpu_target=none when +rocm).
2023-06-06 14:41:02 -07:00
Jonathon Anderson
f41446258a
containers: don't install epel-release on Fedora (#37766) 2023-06-06 21:50:46 +02:00
Massimiliano Culpo
e483762015
Update RtD and Sphinx configuration (#38046) 2023-06-05 17:39:11 +02:00
Massimiliano Culpo
42667fe7fa
Memoize a few hot functions during module file generation (#37739) 2023-06-01 13:36:42 -07:00
Greg Becker
dace0316a2
Spec.format: print false attributes if requested (#37932) 2023-06-01 09:08:45 +02:00
Brian Van Essen
543b697df1
CachedCMakePackage: add CUDA/HIP options and improve independent builds (#37592)
* Add CMake options for building with CUDA/HIP support to
  CachedCMakePackages (intended to reduce duplication across packages
  building with +hip/+cuda and using CachedCMakePackage)
* Define generic variables like CMAKE_PREFIX_PATH for
  CachedCMakePackages (so that a user may invoke "cmake" themselves
  without needing to setthem on the command line).
* Make `lbann` a CachedCMakePackage.

Co-authored-by: Chris White <white238@llnl.gov>
2023-05-31 17:35:11 -07:00
Tiziano Müller
0f84782fcc
Bugfix: cray manifest parsing regression (#37909)
fa7719a changed syntax for specifying exact versions, which are
required for some compiler specs (including those read as part
of parsing a Cray manifest). This fixes that and also makes a
couple other improvements to manifest parsing.

* Instantiate compiler specs with exact versions (fixes #37893)
* fix slingshot network detection (CPE 22.10+ has libcxi.so
  in /usr/lib64)
* "spack external find": add arg to ignore default dir for cray
  manifests
2023-05-30 18:03:44 -07:00
Massimiliano Culpo
8e9efa86c8
Simplify implementation of "get_compiler_config" (#37989) 2023-05-30 15:11:33 -07:00
Harmen Stoppels
8790efbcfe
Remove patchelf self-relocation (#33834) 2023-05-29 13:14:24 +02:00
Harmen Stoppels
3b59c95323
fix InternalConcretizerError msg (#37791) 2023-05-29 11:24:43 +02:00
Tamara Dahlgren
77ce4701b9
Bugfix/tests: add slash to test log message (#37874) 2023-05-29 10:36:36 +02:00
Tamara Dahlgren
3a5864bcdb
tests/sip: convert to new stand-alone test process (#35693) 2023-05-29 10:16:35 +02:00
Xavier Delaruelle
6c42d2b7f7
modules: improve default naming scheme (#37808)
Change default naming scheme for tcl modules for a more user-friendly
experience. 

Change from flat projection to "per software name" projection.

Flat naming scheme restrains module selection capabilities. The
`{name}/{version}...` scheme make possible to use user-friendly
mechanisms:

* implicit defaults (`module load git`)
* extended default (`module load git/2`)
* advanced version specifiers (`module load git@2:`)
2023-05-28 10:06:30 +02:00
kwryankrattiger
afb3bef7af
CI: Use relative path in default script (#36649) 2023-05-26 14:28:48 -06:00
John W. Parent
0b12a480eb
Windows MSVC: do not set sdk version if installing sdk (#37930)
Note the win-sdk package is not installable and reports an error
which instructs the user how to add it. Without this fix, a
(more confusing) error occurs before this message can be generated.
2023-05-25 21:38:04 -04:00
John W. Parent
d147ef231f
Windows: fix "spack build-env" (#37923)
"spack build-env" was not generating proper environment variable
definitions on Windows; this commit updates the generated commands
to succeed with batch/PowerShell.
2023-05-25 17:08:15 -07:00
Tamara Dahlgren
55561405b8
Bugfix/tests: write not append stand-alone test status (#37841) 2023-05-25 12:36:24 -07:00
Greg Becker
033599c4cd
bugfix: env concretize after remove (#37877) 2023-05-24 15:41:57 +02:00
Harmen Stoppels
8096ed4b22
spack remove: fix traversal when user specs intersect (#37882)
drop unnecessary double loop over the matching user specs.
2023-05-24 09:23:46 -04:00
Glenn Johnson
69e99f0c16
Remove myself as maintainer of R packages (#37859)
* Remove myself as maintainer of R packages
  I will no longer have the time to properly maintain these packages.
* fix flake8 test for import
2023-05-23 15:35:32 -05:00
simonleary-umass-edu
ab3f705019
deleted package.py better error message (#37814)
adds the namespace to the exception object's string representation
2023-05-22 09:59:07 -07:00
Harmen Stoppels
bf45a2b6d3
spack env create: generate a view when newly created env has concrete specs (#37799) 2023-05-19 18:44:54 +02:00
Massimiliano Culpo
e66888511f
archspec: fix entry in the JSON file (#37793) 2023-05-19 09:57:57 -04:00
Massimiliano Culpo
bfadd5c9a5
lmod: allow core compiler to be specified with a version range (#37789)
Use CompilerSpec with satisfies instead of string equality tests

Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
2023-05-19 13:21:40 +02:00
Greg Becker
16e9279420
compiler specs: do not print '@=' when clear from context (#37787)
Ensure that spack compiler add/find/list and lists of concrete specs
print the compiler effectively as {compiler.name}{@compiler.version}.

Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
2023-05-19 11:31:27 +02:00
Dom Heinzeller
e733b87865
Remove references to gmake executable, only use make (#37280) 2023-05-18 19:03:03 +00:00
Massimiliano Culpo
c2ce9a6d93
Bump Spack version on develop to 0.21.0.dev0 (#37760) 2023-05-18 12:47:55 +02:00
Peter Scheibel
4e3ed56dfa
Bugfix: allow preferred new versions from externals (#37747) 2023-05-18 09:40:26 +02:00
Tamara Dahlgren
dcfcc03497
maintainers: switch from list to directive (#37752) 2023-05-17 22:25:57 +00:00
Massimiliano Culpo
d45818ccff
Limit deepcopy to just the initial "all" section (#37718)
Modifications:
- [x] Limit the scope of the deepcopy when initializing module file writers
2023-05-17 10:17:41 -07:00
Scott Wittenburg
bcb7af6eb3
gitlab ci: no copy-only pipelines w/ deprecated config (#37720)
Make it clear that copy-only pipelines are not supported while still
using the deprecated ci config format. Also ensure that the deprecated
stack does not fail on spack pipelines for tags.
2023-05-17 09:46:30 -06:00
Tamara Dahlgren
86b9ce1c88
spack test: fix stand-alone test suite status reporting (#37602)
* Fix reporting of packageless specs as having no tests

* Add test_test_output_multiple_specs with update to simple-standalone-test (and tests)

* Refactored test status summary; added more tests or checks
2023-05-17 16:03:21 +02:00
Yoshiaki Senda
d96406a161
Add recently added Spack Docker Images to documentation (#37732)
Signed-off-by: Yoshiaki Senda <yoshiaki@live.it>
2023-05-17 08:48:27 +02:00
Massimiliano Culpo
67e74da3ba
Fix spack find not able to display version ranges in compilers (#37715) 2023-05-17 00:24:38 -04:00
John W. Parent
79e4a13eee
Windows: fix MSVC version handling (#37711)
MSVC compiler logic was using string parsing to extract version
from compiler spec, which was fragile. This broke in #37572, so has
been fixed and made more robust by using attribute access.
2023-05-16 11:00:55 -07:00
Peter Scheibel
7bc5b26c52
Requirements and preferences should not define (non-git) versions (#37687)
Ensure that requirements `packages:*:require:@x` and preferences `packages:*:version:[x]`
fail concretization when no version defined in the package satisfies `x`. This always holds
except for git versions -- they are defined on the fly.
2023-05-16 15:45:11 +02:00
Harmen Stoppels
e56c90d839
check_modules_set_name: do not check for "enable" key (#37701) 2023-05-16 11:51:52 +02:00
Greg Becker
3765a5f7f8
unify: when_possible and unify: true -- Bugfix for error in 37438 (#37681)
Two bugs came in from #37438

1. `unify: when_possible` was broken, because of an incorrect assertion. abstract/concrete
   spec pairs were compared against the results that were in the process of being computed,
   rather than against the previous results.
2. `unify: true` had an ordering bug that could mix the association between abstract and
   concrete specs

- [x] 1 is resolved by creating a lookup from old concrete specs to old abstract specs,
      and we use that to associate the "new" concrete specs that happen to be the old
      ones with their abstract specs (since those are stripped out for concretization
- [x] 2 is resolved by combining the new and old abstract as lists instead of combining
      them as sets. This is important because `set() | set()` does not make any ordering
      promises, even though set ordering is otherwise guaranteed in `python@3.7:`
2023-05-16 01:08:34 -04:00
Carson Woods
33752eabb8
Improve package source code context display on error (#37655)
Spack displays package code context when it shouldn't (e.g., on `FetchError`s)
and doesn't display it when it should (e.g., when errors occur in builder classes.
The line attribution can sometimes be off by one, as well.

- [x] Display package context when errors occur in a subclass of `PackageBase`
- [x] Display package context when errors occur in a subclass of `BaseBuilder`
- [x] Do not display package context when errors occur in `PackageBase`,
      `BaseBuilder` or other core code that is not in a `package.py` file.
- [x] Fix off-by-one error for core code (don't subtract one from the line number *unless*
      it's in an actual `package.py` file.

---------

Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
2023-05-15 13:38:11 -07:00
Todd Gamblin
e79a911bac bugfix: allow reuse of packages from foreign namespaces
We currently throw a nasty error if you try to reuse packages from some other namespace
(e.g., OLCF), but we should be able to reuse patched local versions of builtin packages.

Right now the only obstacle to that is that we try to look up virtual info for unknown
namespaces, and we can't get the package from the repo to do that. We *can* assume that
a package with a known namespace is similar, and that its virtual provider information
is reasonably accurate, so we now do that. This isn't 100% accurate, but neither is
relying on the package itself, as it may have gone out of date.

The real solution here is virtual edge information, but this is a stopgap until we have
that.
2023-05-15 09:15:49 -07:00
Todd Gamblin
fd3efc71fd bugfix: don't look up virtual information for unknown packages
`spec_clauses()` attempts to look up package information for concrete specs in order to
determine which virtuals they may provide. This fails for renamed/deleted dependencies
of buildcaches and installed packages.

This will eventually be fixed by #35258, which adds virtual information on edges, but we
need a workaround to make older buildcaches usable.

- [x] make an exception for renamed packages and omit their virtual constraints
- [x] add a note that this will be solved by adding virtuals to edges
2023-05-15 09:15:49 -07:00
Todd Gamblin
0458de18de bugfix: don't look up patches from packages for concrete specs
The concretizer can fail with `reuse:true` if a buildcache or installation contains a
package with a dependency that has been renamed or deleted in the main repo (e.g.,
`netcdf` was refactored to `netcdf-c`, `netcdf-fortran`, etc., but there are still
binary packages with dependencies called `netcdf`).

We should still be able to install things for which we are missing `package.py` files.

`Spec.inject_patches_variant()` was failing this requirement by attempting to look up
the package class for concrete specs.  This isn't needed -- we can skip it.

- [x] swap two conditions in `Spec.inject_patches_variant()`
2023-05-15 09:15:49 -07:00
Harmen Stoppels
e33eafd34f
Bump tutorial command (#37674) 2023-05-15 13:54:52 +02:00
Todd Gamblin
cf9dc3fc81
spack find: get rid of @= in arch/compiler headers (#37672)
The @= in `spack find` output adds a bit of noise. Remove it as we
did for `spack spec` and `spack concretize`.

This modifies display_specs so it actually covers other places we use that routine, as
well, e.g., `spack buildcache list`.

before:

```
-- linux-ubuntu20.04-aarch64 / gcc@=11.1.0 -----------------------
ofdlcpi libpressio@0.88.0
```

after:

```
-- linux-ubuntu20.04-aarch64 / gcc@11.1.0 -----------------------
ofdlcpi libpressio@0.88.0
```
2023-05-15 09:08:50 +02:00
Greg Becker
a2a6e65e27
concretizer: don't change concrete environments without --force (#37438)
If a user does not explicitly `--force` the concretization of an entire environment,
Spack will try to reuse the concrete specs that are already in the lockfile.

---------

Co-authored-by: becker33 <becker33@users.noreply.github.com>
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
2023-05-14 13:36:03 +02:00
Todd Gamblin
c5a24675a1
spack spec: remove noisy @= from output (#37663)
@= is accurate, but noisy. Other UI commands tend not to
print the redundant `@=` for known concrete versions;
make `spack spec` consistent with them.
2023-05-13 11:34:15 -07:00
Scott Wittenburg
c08be95d5e
gitlab ci: release fixes and improvements (#37601)
* gitlab ci: release fixes and improvements

  - use rules to reduce boilerplate in .gitlab-ci.yml
  - support copy-only pipeline jobs
  - make pipelines for release branches rebuild everything
  - make pipelines for protected tags copy-only

* gitlab ci: remove url changes used in testing

* gitlab ci: tag mirrors need public key

Make sure that mirrors associated with release branches and tags
contain the public key needed to verify the signed binaries.  This
also ensures that when stack-specific mirror contents are copied
to the root, the root mirror has the public key as well.

* review: be more specific about tags, curl flags

* Make the check in ci.yaml consistent with the .gitlab-ci.yml

---------

Co-authored-by: Ryan Krattiger <ryan.krattiger@kitware.com>
2023-05-12 15:22:42 -05:00
Nathan Hanford
eef2536055
Allow buildcache specs to be referenced by hash (#35042)
Currently, specs on buildcache mirrors must be referenced by their full description. This PR allows buildcache specs to be referenced by their hashes, rather than their full description.

### How it works

Hash resolution has been moved from `SpecParser` into `Spec`, and now includes the ability to execute a `BinaryCacheQuery` after checking the local store, but before concluding that the hash doesn't exist.

### Side-effects of Proposed Changes

Failures will take longer when nonexistent hashes are parsed, as mirrors will now be scanned.

### Other Changes

- `BinaryCacheIndex.update` has been modified to fail appropriately only when mirrors have been configured.
- Tests of hash failures have been updated to use `mutable_empty_config` so they don't needlessly search mirrors.
- Documentation has been clarified for `BinaryCacheQuery`, and more documentation has been added to the hash resolution functions added to `Spec`.
2023-05-12 10:27:42 -07:00
Massimiliano Culpo
e2ae60a3b0
Update archspec to v0.2.1 (#37633) 2023-05-12 18:59:58 +02:00
Robert Cohn
fe8734cd52
Fix logic in setting oneapi microarchitecture flags (#37634) 2023-05-12 10:58:08 -04:00
Massimiliano Culpo
fd45839c04
Improve error message for buildcaches (#37626) 2023-05-12 11:55:13 +02:00
Massimiliano Culpo
ebfc706c8c
Improve error messages when Spack finds a too new DB / lockfile (#37614)
This PR ensures that we'll get a comprehensible error message whenever an old
version of Spack tries to use a DB or a lockfile that is "too new".

* Fix error message when using a too new DB
* Add a unit-test to ensure we have a comprehensible error message
2023-05-12 08:13:10 +00:00
Tamara Dahlgren
8e18297cf2
Environments: store spack version/commit in spack.lock (#32801)
Add a section to the lock file to track the Spack version/commit that produced
an environment. This should (eventually) enhance reproducibility, though we
do not currently do anything with the information. It just adds to provenance
at the moment.

Changes include:
- [x] adding the version/commit to `spack.lock`
- [x] refactor `spack.main.get_version()
- [x] fix a couple of environment lock file-related typos
2023-05-11 23:13:36 -04:00
Massimiliano Culpo
5c7dda7e14
Allow using -j to control the parallelism of concretization (#37608)
fixes #29464

This PR allows to use
```
$ spack concretize -j X
```
to set a cap on the parallelism of concretization from the command line
2023-05-11 13:29:17 -07:00
Harmen Stoppels
4fa7880b19
lmod: fix CompilerSpec concrete version / range (#37604) 2023-05-11 12:00:07 +02:00
Massimiliano Culpo
8da29d1231
Improve the message for errors in package recipes (#37589)
fixes #30355
2023-05-11 10:34:39 +02:00
Massimiliano Culpo
297329f4b5
Improve error message for missing "command" entry in containerize (#37590)
fixes #21242
2023-05-11 10:33:51 +02:00
Peter Scheibel
bfa54da292
Allow clingo to enforce flags when they appear in requirements (#37584)
Flags are encoded differently from other variants, and they need a choice rule to
ensure clingo has a choice to impose (or not) a constraint.
2023-05-11 09:17:16 +02:00
Harmen Stoppels
2c17c4e632
ci: remove --mirror-url flag (#37457)
The flags --mirror-name / --mirror-url / --directory were deprecated in 
favor of just passing a positional name, url or directory, and letting spack
figure it out.

---------

Co-authored-by: Scott Wittenburg <scott.wittenburg@kitware.com>
2023-05-10 16:34:29 -06:00
John W. Parent
ec800cccbb
Windows: Fix external detection for service accounts (#37293)
Prior to this PR, the HOMEDRIVE environment variable was used to
detect what drive we are operating in. This variable is not available
for service account logins (like what is used for CI), so switch to
extracting the drive from PROGRAMFILES (which is more-widely defined).
2023-05-10 18:12:58 -04:00
John W. Parent
85cc9097cb
Windows: prefer Python decompression support (#36507)
On Windows, several commonly available system tools for decompression
are unreliable (gz/bz2/xz). This commit refactors `decompressor_for`
to call out to a Windows or Unix-specific method:

* The decompressor_for_nix method behaves the same as before and
  generally treats the Python/system support options for decompression
  as interchangeable (although avoids using Python's built-in tar
  support since that has had issues with permissions).
* The decompressor_for_win method can only use Python support for
  gz/bz2/xz, although for a tar.gz it does use system support for
  untar (after the decompression step). .zip uses the system tar
  utility, and .Z depends on external support (i.e. that the user
  has installed 7zip).

A naming scheme has been introduced for the various _decompression
methods:

* _system_gunzip means to use a system tool (and fail if it's not
    available)
* _py_gunzip means to use Python's built-in support for decompressing
    .gzip files (and fail if it's not available)
* _gunzip is a method that can do either
2023-05-10 18:07:56 -04:00
Harmen Stoppels
1d96fdc74a
Fix compiler version issues (concrete vs range) (#37572) 2023-05-10 17:26:22 +02:00
matteo-chesi
e70755f692
cuda: add versions 12.0.1, 12.1.0 and 12.1.1 (#37083) 2023-05-10 15:31:07 +02:00
G-Ragghianti
ebb40ee0d1
New option "--first" for "spack location" (#36283) 2023-05-10 12:26:29 +02:00
Robert Cohn
a2ea30aceb
Create include/lib in prefix for oneapi packages (#37552) 2023-05-10 06:25:00 -04:00
Tamara Dahlgren
9a37c8fcb1
Stand-alone testing: make recipe support and processing spack-/pytest-like (#34236)
This is a refactor of Spack's stand-alone test process to be more spack- and pytest-like. 

It is more spack-like in that test parts are no longer "hidden" in a package's run_test()
method and pytest-like in that any package method whose name starts test_ 
(i.e., a "test" method) is a test part. We also support the ability to embed test parts in a
test method when that makes sense.

Test methods are now implicit test parts. The docstring is the purpose for the test part. 
The name of the method is the name of the test part. The working directory is the active
spec's test stage directory. You can embed test parts using the test_part context manager.

Functionality added by this commit:
* Adds support for multiple test_* stand-alone package test methods, each of which is 
   an implicit test_part for execution and reporting purposes;
* Deprecates package use of run_test();
* Exposes some functionality from run_test() as optional helper methods;
* Adds a SkipTest exception that can be used to flag stand-alone tests as being skipped;
* Updates the packaging guide section on stand-alone tests to provide more examples;
* Restores the ability to run tests "inherited" from provided virtual packages;
* Prints the test log path (like we currently do for build log paths);
* Times and reports the post-install process (since it can include post-install tests);
* Corrects context-related error message to distinguish test recipes from build recipes.
2023-05-10 11:34:54 +02:00
kwryankrattiger
45e1d3498c
CI: Backwards compatibility requires script override behavior (#37015) 2023-05-09 10:42:06 -06:00
Massimiliano Culpo
89520467e0
Use single quotes to inline manifest in Dockerfiles (#37571)
fixes #22341

Using double quotes creates issues with shell variable substitutions,
in particular when the manifest has "definitions:" in it. Use single
quotes instead.
2023-05-09 13:20:25 +02:00
Harmen Stoppels
9e1440ec7b
spack view copy: relocate symlinks (#32306) 2023-05-09 12:17:16 +02:00
Massimiliano Culpo
0139288ced
Add a "requires" directive, extend functionality of package requirements (#36286)
Add a "require" directive to packages, which functions exactly like
requirements specified in packages.yaml (uses the same fact-generation
logic); update both to allow making the requirement conditional.

* Packages may now use "require" to add constraints. This can be useful
  for something like "require(%gcc)" (where before we had to add a
  conflict for every compiler except gcc).
* Requirements (in packages.yaml or in a "require" directive) can be
  conditional on a spec, e.g. "require(%gcc, when=@1.0.0)" (version
  1.0.0 can only build with gcc).
* Requirements may include a message which clarifies why they are needed.
  The concretizer assigns a high priority to errors which generate these
  messages (in particular over errors for unsatisfied requirements that
  do not produce messages, but also over a number of more-generic
  errors).
2023-05-08 10:12:26 -07:00
Tamara Dahlgren
374264f610
Packaging Guide: build-time test updates: option and test logs (#37093)
* Packaging Guide: build-time test updates: option and test logs
* Fix a couple of typos
2023-05-05 22:19:06 -06:00
Harmen Stoppels
fa7719a031
Improve version, version range, and version list syntax and behavior (#36273)
## Version types, parsing and printing

- The version classes have changed: `VersionBase` is removed, there is now a
  `ConcreteVersion` base class. `StandardVersion` and `GitVersion` both inherit
  from this.

- The public api (`Version`, `VersionRange`, `ver`) has changed a bit:
  1. `Version` produces either `StandardVersion` or `GitVersion` instances.
  2. `VersionRange` produces a `ClosedOpenRange`, but this shouldn't affect the user.
  3. `ver` produces any of `VersionList`, `ClosedOpenRange`, `StandardVersion`
     or `GitVersion`.

- No unexpected type promotion, so that the following is no longer an identity:
  `Version(x) != VersionRange(x, x)`.

- `VersionList.concrete` now returns a version if it contains only a single element
  subtyping `ConcreteVersion` (i.e. `StandardVersion(...)` or `GitVersion(...)`)

- In version lists, the parser turns `@x` into `VersionRange(x, x)` instead
  of `Version(x)`.

- The above also means that `ver("x")` produces a range, whereas
  `ver("=x")` produces a `StandardVersion`. The `=` is part of _VersionList_
  syntax.

- `VersionList.__str__` now outputs `=x.y.z` for specific version entries,
  and `x.y.z` as a short-hand for ranges `x.y.z:x.y.z`.

- `Spec.format` no longer aliases `{version}` to `{versions}`, but pulls the
  concrete version out of the list and prints that -- except when the list is
  is not concrete, then is falls back to `{versions}` to avoid a pedantic error.
  For projections of concrete specs, `{version}` should be used to render
  `1.2.3` instead of `=1.2.3` (which you would get with `{versions}`).
  The default `Spec` format string used in `Spec.__str__` now uses
  `{versions}` so that `str(Spec(string)) == string` holds.

## Changes to `GitVersion`

- `GitVersion` is a small wrapper around `StandardVersion` which enriches it
   with a git ref. It no longer inherits from it.

- `GitVersion` _always_ needs to be able to look up an associated Spack version
  if it was not assigned (yet). It throws a `VersionLookupError` whenever `ref_version`
  is accessed but it has no means to look up the ref; in the past Spack would
  not error and use the commit sha as a literal version, which was incorrect.
   
- `GitVersion` is never equal to `StandardVersion`, nor is satisfied by it. This
  is such that we don't lose transitivity. This fixes the following bug on `develop`
  where `git_version_a == standard_version == git_version_b` does not imply
  `git_version_a == git_version_b`. It also ensures equality always implies equal
  hash, which is also currently broken on develop; inclusion tests of a set of
  versions + git versions would behave differently from inclusion tests of a
  list of the same objects.

- The above means `ver("ref=1.2.3) != ver("=1.2.3")` could break packages that branch
  on specific versions, but that was brittle already, since the same happens with
  externals: `pkg@1.2.3-external` suffixes wouldn't be exactly equal either. Instead,
  those checks should be `x.satisfies("@1.2.3")` which works both for git versions and
  custom version suffixes.

- `GitVersion` from commit will now print as `<hash>=<version>` once the
  git ref is resolved to a spack version. This is for reliability -- version is frozen
  when added to the database and queried later. It also improves performance
  since there is no need to clone all repos of all git versions after `spack clean -m`
  is run and something queries the database, triggering version comparison, such
  as potentially reuse concretization.

- The "empty VerstionStrComponent trick" for `GitVerison` is dropped since it wasn't
  representable as a version string (by design). Instead, it's replaced by `git`,
  so you get `1.2.3.git.4` (which reads 4 commits after a tag 1.2.3). This means
  that there's an edge case for version schemes `1.1.1`, `1.1.1a`, since the
  generated git version `1.1.1.git.1` (1 commit after `1.1.1`) compares larger
  than `1.1.1a`, since `a < git` are compared as strings. This is currently a
  wont-fix edge case, but if really required, could be fixed by special casing
  the `git` string.

- Saved, concrete specs (database, lock file, ...) that only had a git sha as their
  version, but have no means to look the effective Spack version anymore, will
  now see their version mapped to `hash=develop`. Previously these specs
  would always have their sha literally interpreted as a version string (even when
  it _could_ be looked up). This only applies to databases, lock files and spec.json
  files created before Spack 0.20; after this PR, we always have a Spack version
  associated to the relevant GitVersion).

- Fixes a bug where previously `to_dict` / `from_dict` (de)serialization would not
  reattach the repo to the GitVersion, causing the git hash to be used as a literal
  (bogus) version instead of the resolved version. This was in particularly breaking
  version comparison in the build process on macOS/Windows.


## Installing or matching specific versions

- In the past, `spack install pkg@3.2` would install `pkg@=3.2` if it was a
  known specific version defined in the package, even when newer patch releases
  `3.2.1`, `3.2.2`, `...` were available. This behavior was only there because
  there was no syntax to distinguish between `3.2` and `3.2.1`. Since there is
  syntax for this now through `pkg@=3.2`, the old exact matching behavior is
  removed. This means that `spack install pkg@3.2` constrains the `pkg` version
  to the range `3.2`, and `spack install pkg@=3.2` constrains it to the specific
  version `3.2`.

- Also in directives such as `depends_on("pkg@2.3")` and their when
  conditions `conflicts("...", when="@2.3")` ranges are ranges, and specific
  version matches require `@=2.3.`.

- No matching version: in the case `pkg@3.2` matches nothing, concretization
  errors. However, if you run `spack install pkg@=3.2` and this version
  doesn't exist, Spack will define it; this allows you to install non-registered
  versions.

- For consistency, you can now do `%gcc@10` and let it match a configured
  `10.x.y` compiler. It errors when there is no matching compiler.
  In the past it was interpreted like a specific `gcc@=10` version, which
  would get bootstrapped.

- When compiler _bootstrapping_ is enabled, `%gcc@=10.2.0` can be used to
  bootstrap a specific compiler version.

## Other changes

- Externals, compilers, and develop spec definitions are backwards compatible.
  They are typically defined as `pkg@3.2.1` even though they should be
  saying `pkg@=3.2.1`. Spack now transforms `pkg@3` into `pkg@=3` in those cases.

- Finally, fix strictness of `version(...)` directive/declaration. It just does a simple
  type check, and now requires strings/integers. Floats are not allowed because
  they are ambiguous `str(3.10) == "3.1"`.
2023-05-05 22:04:41 -06:00
Eric Berquist
b2a8e8734e
Fix typos in packaging guide (#37460) 2023-05-05 22:08:58 +00:00
Harmen Stoppels
9ef062fcca
Add spack buildcache push (alias to buildcache create) (#34861)
`spack buildcache create` is a misnomer cause it's the only way to push to
an existing buildcache (and it in fact calls binary_distribution.push).

Also we have `spack buildcache update-index` but for create the flag is
`--rebuild-index`, which is confusing (and also... why "rebuild"
something if the command is "create" in the first place, that implies it
wasn't there to begin with).

So, after this PR, you can use either

```
spack buildcache create --rebuild-index
```

or

```
spack buildcache push --update-index
```

Also, alias `spack buildcache rebuild-index` to `spack buildcache
update-index`.
2023-05-05 19:54:26 +02:00
Chris Green
d600aef4f4
Relax environment manifest filename requirements and lockfile identification criteria (#37413)
* Relax filename requirements and lockfile identification criteria

* Tests

* Update function docs and help text

* Update function documentation

* Update Sphinx documentation

* Adjustments per https://github.com/spack/spack/pull/37413#pullrequestreview-1413540132

* Further tweaks per https://github.com/spack/spack/pull/37413#pullrequestreview-1413971254

* Doc fixes per https://github.com/spack/spack/pull/37413#issuecomment-1535976068
2023-05-05 07:40:49 -05:00
Harmen Stoppels
bbc779f3f0
cc: deal with -Wl,-rpath= without value, deal with NAG (#37215)
Spack never parsed `nagfor` linker arguments put on the compiler line: 
```
nagfor -Wl,-Wl,,-rpath,,/path
````
so, let's continue not attempting to parse that.
2023-05-05 12:16:31 +02:00
Michael Kuhn
b2c3973d4a
meson: change default build type to "release" (#37436)
The same was done for CMake in #36679.
2023-05-05 10:35:40 +02:00
Harmen Stoppels
35e1dc8eba
spack uninstall: reduce verbosity with named environments (#34001) 2023-05-05 10:23:08 +02:00
Harmen Stoppels
bf71b78094
deprecate buildcache create --rel, buildcache install --allow-root (#37285)
`buildcache create --rel`: deprecate this because there is no point in
making things relative before tarballing; on install you need to expand
`$ORIGIN` / `@loader_path` / relative symlinks anyways because some
dependencies may actually be in an upstream, or have different
projections.

`buildcache install --allow-root`: this flag was propagated through a
lot of functions but was ultimately unused.
2023-05-05 09:51:53 +02:00
Massimiliano Culpo
0c5a5e2ce0
Remove "blacklist" and "whitelist" from module configuration (#37432)
The sections were deprecated in v0.19
2023-05-05 00:28:34 -04:00
Greg Becker
c3593e5b48
Allow choosing the name of the packages subdirectory in repositories (#36643)
Co-authored-by: becker33 <becker33@users.noreply.github.com>
2023-05-04 23:36:21 +02:00
Massimiliano Culpo
16613408e4
Place an upper bound on urllib3 to build docs (#37433) 2023-05-04 19:40:43 +02:00
Robert Cohn
420e093e42
detect ifx 2023.1, add test (#37377) 2023-05-04 10:27:19 -07:00
Massimiliano Culpo
86d3bad1e0
cmake build system: change default build type to Release (#36679)
This switches the default Make build type to `build_type=Release`.

This offers:
- higher optimization level, including loop vectorization on older GCC
- adds NDEBUG define, which disables assertions, which could cause speedups if assertions are in loops etc
- no `-g` means smaller install size

Downsides are:
- worse backtraces (though this does NOT strip symbols)
- perf reports may be useless
- no function arguments / local variables in debugger (could be of course)
- no file path / line numbers in debugger

The downsides can be mitigated by overriding to `build_type=RelWithDebInfo` in `packages.yaml`,
if needed.  The upside is that builds will be MUCH smaller (and faster) with this change.

---------

Co-authored-by: Gregory Becker <becker33@llnl.gov>
2023-05-04 11:33:35 -04:00
Massimiliano Culpo
600955edd4
Update vendored ruamel.yaml to v0.17.21 (#37008)
* Vendor ruamel.yaml v0.17.21

* Add unit test for whitespace regression

* Add an abstraction layer in Spack to wrap ruamel.yaml

All YAML operations are routed through spack.util.spack_yaml

The custom classes have been adapted to the new ruamel.yaml
class hierarchy.

Fixed line annotation issue in "spack config blame"
2023-05-04 08:00:38 -07:00
Massimiliano Culpo
95e61f2fdf
Remove the old spec format in configuration (#37425)
The format was deprecated in v0.15
2023-05-04 07:59:11 -07:00
Massimiliano Culpo
cf5daff6f5
Deprecate env: as top level environment key (#37424) 2023-05-04 07:08:29 -04:00
Annop Wongwathanarat
e5dcaebd43
acfl: add compiler-package mapping and fix version number (#36768) 2023-05-04 03:59:15 -05:00