Commit graph

5652 commits

Author SHA1 Message Date
Massimiliano Culpo
1913dc2da3
Fix performance regression with spack mirror create --all (#32005)
This PR fixes the performance regression reported in #31985 and a few
other issues found while refactoring the spack mirror create command.

Modifications:

* (Primary) Do not require concretization for
  `spack mirror create --all`
* Forbid using --versions-per-spec together with --all
* Fixed a few issues when reading specs from input file (specs were
  not concretized, command would fail when trying to mirror
  dependencies)
* Fix issue with default directory for spack mirror create not being
  canonicalized
* Add more unit tests to poke spack mirror create
* Skip externals also when mirroring environments
* Changed slightly the wording for reporting (it was mentioning
  "Successfully created" even in presence of errors)
* Fix issue with colify (was not called properly during error
  reporting)
2022-08-11 16:51:01 -07:00
Chris White
8c23545a3a
CachedCMakePackage: Port over logic from CMake build system for ldlibs and ldflags (#31750)
* port over logic from cmake build system for ldlibs and ldflags

* follow new style standards

* escape quotes

* escape quotes

* nevermind going back
2022-08-11 14:55:43 -07:00
Harmen Stoppels
ceda5fb46c
Don't set LD_LIBRARY_PATH by default on Linux (#28354)
`LD_LIBRARY_PATH` can break system executables (e.g., when an enviornment is loaded) and isn't necessary thanks to `RPATH`s.  Packages that require `LD_LIBRARY_PATH` can set this in `setup_run_environment`.

- [x] Prefix inspections no longer set `LD_LIBRARY_PATH` by default
- [x] Document changes and workarounds for people who want `LD_LIBRARY_PATH`
2022-08-11 09:33:08 -05:00
Chris Green
46c8962e91
Add missing info to diagnostic for circular provides (#32027) 2022-08-11 14:20:17 +02:00
Tamara Dahlgren
f96b54b0a6
info: properly report default when for overridden variants (#32040) 2022-08-11 14:09:53 +02:00
Richarda Butler
ef818339db
Update tests so /tmp/store can be accessed (#32003) 2022-08-10 20:53:48 +00:00
Mikael Simberg
f237e669f2
Patch /usr/bin/file to file when on nixos (#30718)
These changes make many packages build on nixos where nearly nothing
comes from /bin or /usr/bin (the only things in "system locations" are
/bin/sh and /usr/bin/env, all the rest is found through PATH).

Many configuration scripts hardcode /usr/bin/file instead of using the
one from PATH. This patches them to use file from PATH.
2022-08-10 17:54:09 +02:00
Seth R. Johnson
5d6ab9a76c
core: add a helpful long message for UnknownPackageError (#31996)
```
Error: Package 'armpl' not found.
```
is pretty useless after an upgrade unless you're a spack pro. I've
recently hit this on multiple machines. See
https://github.com/spack/spack/issues/31453 ,
https://github.com/spack/spack/issues/31489  .
2022-08-10 11:01:07 +02:00
Harmen Stoppels
857a798aed
modules: add missing Python imports (#32012) 2022-08-10 10:00:33 +02:00
Tamara Dahlgren
3320c983a7
Reduce installation noise: filter path padding and make some details debug level 2 (#31998) 2022-08-09 19:22:09 +02:00
Adam J. Stewart
d29d5462c6
PythonPackage: add --config-settings support (#31823) 2022-08-09 10:09:51 -07:00
Harmen Stoppels
f53b522572
Add base class for directory visitor (#32008) 2022-08-09 15:43:30 +02:00
Massimiliano Culpo
b61187455a
Update release procedure, keep CHANGELOG up-to-date (#31969)
Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
2022-08-09 12:10:49 +00:00
Massimiliano Culpo
65a4125dd0
spack mirror: skip non-concretizable specs (#31897)
fixes #31736

Catch errors when concretizing specs and report them as
debug messages. The corresponding spec is skipped.

Co-authored-by: Greg Becker <becker33@llnl.gov>
2022-08-05 19:19:51 +02:00
Massimiliano Culpo
5bf6b7e6a5
Refactor cmd/install.py for better readability (#31936)
* Extracted two functions in cmd/install.py

* Extracted a function to perform installation from the active environment

* Rename a few functions, remove args from their arguments

* Rework conditional in install_from_active_environment to reduce nesting in the function

* Extract functions to parsespecs from cli and files

* Extract functions to getuser confirmation for overwrite

* Extract functions to install specs inside and outside environments

* Rename a couple of functions

* Fix outdated comment

* Add missing imports

* Split conditional to dedent one level

* Invert check and exit early to dedent one level when requiring user confirmation
2022-08-05 12:36:33 +02:00
psakievich
ddc373b4e1
Fix doc format for code blocks: GitRef versions (#31937)
Rendering was not pretty using `console` from #31914
2022-08-05 02:01:03 +00:00
Chris Green
20de47a1e2
Move type hint to satisfy Python 3.6, 3.7 interpreters (#31932) 2022-08-04 20:48:05 +00:00
psakievich
d67ead5978
Add documentation for git refs as versions (#31914)
* document git commit versions

Include documentation for manually specifying associated known version

* document spack develop command

Co-authorerd-by: psakievich <psakiev@sandia.gov>
Co-authored-by: Gregory Becker <becker33@llnl.gov>
Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
2022-08-04 13:20:33 -07:00
psakievich
3b1401f292
Git Ref versions can be paired to defined versions in the spec (#30998)
The current use of git ref's as a version requires a search algorithm to pick the right matching version based on the tags in the git history of the package.

This is less than ideal for the use case where users already know the specific version they want the git ref to be associated with. This PR makes a new version syntax [package]@[ref]=[version] to allow the users to specify the exact hash they wish to use.
2022-08-04 13:17:34 -07:00
Massimiliano Culpo
0e40804cd0
Deprecate support for Python 3.5 (by removing it from CI) (#31908)
* Deprecate support for Python 3.5 (by removing it from CI)

* Restore Python 3.9
2022-08-04 15:26:56 +02:00
Massimiliano Culpo
9a48035e49 asp: refactor low level API to permit the injection of configuration
This allows writing extension commands that can benchmark
different configurations in clingo, or try different
configurations for a single test.
2022-08-03 18:01:08 -07:00
dunatotatos
ab87cac63f
Match documentation of spack create with actual behavior. (#31892) 2022-08-03 10:38:56 +00:00
Jonathon Anderson
8c50ea9dc0
spack ci rebuild: Don't install in a login shell (#31771)
On some systems the shell in login mode wipes important parts of the
environment, such as PATH. This causes the build to fail since it can't
find `spack`.

For better robustness, don't use a login shell.

In a full CI job the final spack install is run in an environment formed by scripts running in this order:

export AWS_SECRET=...                  # 1. Load environment from GitLab project variables
source spack/share/spack/setup-env.sh  # 2. Load Spack into the environment (PATH)
spack env activate -V concrete_env     # 3. Activate the concrete environment
source /etc/profile                    # 4. Bash login shell (from -l)
spack install ...
Whereas when a user launches their own container with (docker|podman) run -it, they end up running spack install in an environment formed in this order:

source /etc/bash.bashrc                # (not 4). Bash interactive shell (default with TTY)
export AWS_SECRET=...                  #~1. Manually load environment from GitLab project variables
source spack/share/spack/setup-env.sh  # 2. Load Spack into the environment (PATH)
spack env activate -V concrete_env     # 3. Activate the concrete environment
spack install ...
The big problem being that (4) has a completely different position and content (on Leap 15 and possibly other containers).

So in context, this PR removes (4) from the CI job case, leaving us with the simpler:

export AWS_SECRET=...                  # 1. Load environment from GitLab project variables
source spack/share/spack/setup-env.sh  # 2. Load Spack into the environment (PATH)
spack env activate -V concrete_env     # 3. Activate the concrete environment
spack install ...
2022-08-02 11:45:50 -07:00
Massimiliano Culpo
6e0b7959a4
Optimize reuse from buildcaches (#30806)
* database: don't sort on return from query_local

* ASP-based solver: don't build the hash-lookup dictionary twice

Building this dictionary twice and traversing all the specs
might be time-consuming for large buildcaches.
2022-08-02 11:43:14 -07:00
Massimiliano Culpo
aeac72e1e3 Style fixes 2022-08-02 10:52:52 -07:00
Massimiliano Culpo
0b832b2929 Remove line that sets non-existing attribute? 2022-08-02 10:52:52 -07:00
Massimiliano Culpo
2b3f350071 Use __slots__ for fast attribute access 2022-08-02 10:52:52 -07:00
Massimiliano Culpo
b6c8779772 Use __slots__ for fast attribute access during parsing 2022-08-02 10:52:52 -07:00
Harmen Stoppels
57e6452831
Document that reuse is higher priority than preferences in packages.yaml (#31864)
Resurrect Known issues, since users ask frequently about that.
2022-08-02 18:54:41 +02:00
Todd Gamblin
415ac29665
docs: Fix outdated pre-0.18 text on build dependencies (#31860) 2022-08-02 11:34:58 +00:00
Adam J. Stewart
22d4612d94
Add meson Python build backend (#31809) 2022-08-02 07:55:41 +02:00
Adam J. Stewart
1a61252c64
spack checksum: black compliance (#31851) 2022-08-02 07:29:30 +02:00
Peter Scheibel
c0ede0e0fa
Add c++14 support flag for xl compiler (#31826) 2022-08-01 22:51:02 +00:00
Adam J. Stewart
f018b869e6
Add support for macOS Ventura (#31812) 2022-08-01 16:45:56 +00:00
Adam J. Stewart
c98d367a91
CUDA: add new GCC conflict (#31820) 2022-08-01 14:31:07 +00:00
Todd Gamblin
ab04d697c7 tests: remove old changed_files() test
This test relied on an old version of the `flake8_package` fixture that modified
the spack repository, but it doesn't do that anymore. There are other tests for
`changed_files()` that do a better job of mocking up a git repository with
changes, so we can just delete this one.
2022-07-31 21:01:02 -07:00
Todd Gamblin
143f3f830c style: simplify arguments with --tool TOOL and --skip TOOL
`spack style` tests were annoyingly brittle because we could not easily be
specific about which tools to run (we had to use `--no-black`, `--no-isort`,
`--no-flake8`, and `--no-mypy`). We should be able to specify what to run OR
what to skip.

Now you can run, e.g.:

    spack style --tool black,flake8

or:

    spack style --skip black,isort

- [x] Remove  `--no-black`, `--no-isort`, `--no-flake8`, and `--no-mypy` args.
- [x] Add `--tool TOOL` argument.
- [x] Add `--skip TOOL` argument.
- [x] Allow either `--tool black --tool flake8` or `--tool black,flake8` syntax.
2022-07-31 13:29:20 -07:00
Todd Gamblin
76b190a624 black: ensure that spack create templates are black-compliant
- [x] remove alignment spaces from tempaltes
- [x] replace single with double quotes
- [x] Makefile template now generates parsable code
      (function body is `pass` instead of just a comment)
- [x] template checks now run black to check output
2022-07-31 13:29:20 -07:00
Todd Gamblin
b87ca6da27 black: do not align sha56's with spaces in spack checksum output 2022-07-31 13:29:20 -07:00
Todd Gamblin
357968469e black: bootstrap if black in PATH is too new
Previously we'd accept any version for bootstrapping black, but we need <= 21.

- [x] modify bootstrapping code to check black version before accepting an
      executable from `PATH`.
2022-07-31 13:29:20 -07:00
Todd Gamblin
f27eeaa2e8 black: add .git-blame-ignore-revs toignore black reformatting in blame
- [x] add `.git-blame-ignore-revs` to ignore black reformatting
- [x] make `spack blame` respect `.git-blame-ignore-revs`
      (even if the user hasn't configured git to do so)
2022-07-31 13:29:20 -07:00
Todd Gamblin
c661ca248b black: fix format-sensitive tests
Some of our tests rely on single vs. double quotes, and others rely on specific
line numbers in the source. These needed fixing after the switch to Black.
2022-07-31 13:29:20 -07:00
Todd Gamblin
f52f6e99db black: reformat entire repository with black 2022-07-31 13:29:20 -07:00
Todd Gamblin
549ba1ed32 black: fix style check package and flake8 formatting for black
Black will automatically fix a lot of the exceptions we previously allowed for
directives, so we don't need them in our custom `flake8_formatter` anymore.

- [x] remove `E501` (long line) exceptions for directives from `flake8_formatter`,
      as they won't help us now.
- [x] Refine exceptions for long URLs in the `flake8_formatter`.
- [x] Adjust the mock `flake8-package` to exhibit the exceptions we still allow.
- [x] Update style tests for new `flake8-package`.
- [x] Blacken style test.
2022-07-31 13:29:20 -07:00
Todd Gamblin
156af2a60a black: clean up noqa comments from most of the code
Many noqa's in the code are no longer necessary now that the column limit is 99
characters. Others can easily be eliminated, and still more can just be made more
specific if they do not have to do with line length.

The only E501's still in the code are in the tests for `spack.util.path` and the tests
for `spack style`.
2022-07-31 13:29:20 -07:00
Todd Gamblin
3fa090f490 black: break up long strings that black cannot fix 2022-07-31 13:29:20 -07:00
Todd Gamblin
67d27841ae black: configuration
This adds necessary configuration for flake8 and black to work together.

This also sets the line length to 99, per the data here:

* https://github.com/spack/spack/pull/24718#issuecomment-876933636

Given the data and the spirit of black's 88-character limit, we set the limit to 99
characters for all of Spack, because:

* 99 is one less than 100, a nice round number, and all lines will fit in a
  100-character wide terminal (even when the text editor puts a \ at EOL).
* 99 is just past the knee the file size curve for packages, and it means that packages
  remain readable and not significantly longer than they are now.
* It doesn't seem to hurt core -- files in core might change length by a few percent but
  seem like they'll be mostly the same as before -- just a bit more roomy.

- [x] set line length to 99
- [x] remove most exceptions from `.flake8` and add the ones black cares about
- [x] add `[tool.black]` to `pyproject.toml`
- [x] make `black` run if available in `spack style --fix`

Co-Authored-By: Tom Scogland <tscogland@llnl.gov>
2022-07-31 13:29:20 -07:00
Adam J. Stewart
ec87924039
MACOSX_DEPLOYMENT_TARGET: MAJOR.MINOR (#31811) 2022-07-30 02:12:02 +00:00
Harmen Stoppels
e4f0f3582e
environment.py: only install root specs explicitly (#31645) 2022-07-28 12:16:35 -07:00
Stephen Sachs
584cc47fb3
Fix parsing of clean_url (#31783)
In #31618 the idea was to determine the file extension heuristically by dropping query params etc from a url and then consider it as a file path. That broke for URLs that only have query params like http://example.com/?patch=x as it would result in empty string as basename. This PR reverts to the old behavior of saving files as ?patch=x in that case.

Co-authored-by: Stephen Sachs <stesachs@amazon.com>
2022-07-28 09:24:19 +00:00
Harmen Stoppels
ce039e4fa5
environment.py: reduce # of locks further (#31643)
* environment.py: reduce # of locks further
2022-07-26 09:00:27 -07:00
Harmen Stoppels
e2056377d0
containerize: fix concretization -> concretizer (#31594)
* containerize: fix concretization -> concretizer

* fix test
2022-07-26 08:56:24 -07:00
Chuck Atkins
f38a2ebb25
llvm: Variant and dependency cleanup (#31331)
* llvm: Use variant when clauses for many of the expressed conflicts

* llvm: Remove the shared variant as it wasn't really used

* llvm: Remove unnecessary deps and make explicit the ones that are

* llvm: Cleanup patch conditions

* pocl: Update for llvm cleanup

* unit-test: update unparse package hash with the updated llvm package

* llvm: Fix ppc long double patching and add clarifying comments
2022-07-26 10:45:20 -05:00
Stephen Sachs
6a240c9b39
Fix self.extension for symlinks (#31743)
`self.archive_file` is (among others) a symlink to a tarball. `extension()` on a
symlink will result in no extension. This patch fixes the behavior introduced in
https://github.com/spack/spack/pull/31618.

Co-authored-by: Stephen Sachs <stesachs@amazon.com>
2022-07-26 13:05:41 +00:00
Harmen Stoppels
b56871dd9c
Fix stage with uri params (#31618)
* don't determine extension of a url

* parse url to drop query params from filename
2022-07-25 10:31:13 +02:00
Adam J. Stewart
5850afc9dc
Sphinx 5.1.0 breaks doc builds (#31709) 2022-07-24 21:45:01 +00:00
Harmen Stoppels
fce861d2ac
Mark external as explicit only when installing explicitly (#31665) 2022-07-22 15:20:17 +02:00
Harmen Stoppels
799553d302
autotools: add -I flag when non-standard libtool (#31677)
When
1. Spack installs libtool,
2. system libtool is installed too, and
3. system automake is used

Spack passes system automake's `-I <prefix>` flag to itself, even though
it's a default search path. This takes precedence over spack's libtool
prefix dir. This causes the wrong `libtool.m4` file to be used (since
system libtool is in the same prefix as system automake).

And that leads to error messages about incompatible libtool, something
something LT_INIT.
2022-07-22 15:11:38 +02:00
Massimiliano Culpo
5cf7bf3770
Use pkg_cls in spack.mirror.get_all_versions (#31636)
fixes #31627

spack.mirror.get_all_versions now uses the package class
instead of the package object in its implementation.

Ensure spec is concrete before staging for mirrors
2022-07-22 10:04:17 +02:00
psakievich
22798d7540
Don't restage develop specs when a patch fails (#31593)
* make develop specs not restage when a patch fails
* add a unit test
2022-07-20 13:55:16 -07:00
Harmen Stoppels
e7be8dbbcf
spack stage: add missing --fresh and --reuse (#31626) 2022-07-20 13:50:56 +02:00
Harmen Stoppels
e3aca44601
installer.py: require "explicit: True" in the install arguments to mark a package "explicit" (#31646) 2022-07-20 12:09:45 +02:00
Jonathon Anderson
117c7cc3db
Only hack botocore when needed (#31610)
Newer versions of botocore (>=1.23.47) support the full IOBase
interface, so the hacks added to supplement the missing attributes are
no longer needed. Conditionally disable the hacks if they appear to be
unnecessary based on the class hierarchy found at runtime.
2022-07-18 08:40:13 +02:00
Harmen Stoppels
6f8d7390f1
Use lexists instead of exists during fetch (#31509) 2022-07-15 18:43:14 +02:00
Joseph Snyder
64b41b012c
Bug/fix credentials s3 buildcache update (#31391)
* Add connection information to buildcache update command

Ensure that the s3 connection made when attempting to update the content
of a buildcache attempts to use the extra connection information
from the mirror creation.

* Add unique help for endpoint URL argument

Fix copy/paste error for endpoint URL help which was the same as
the access token

* Re-work URL checking for S3 mirrors

Due to the fact that nested bucket URLs would never match the string used
for checking that the mirror is the same, switch the check used.

Sort all mirror URLs by length to have the most specific cases first
and see if the desired URL "starts with" the mirror URL.

* Long line style fixes

Add execptions for long lines and fix other style errors

* Use format() function to rebuild URL

Use the format command to rebuild the url instead of crafing a
formatted string out of known values

* Add early exit for URL checking

When a valid mirror is found, break from the loop
2022-07-14 20:49:46 +00:00
Todd Gamblin
3d0347ddd3
Deprecate blacklist/whitelist in favor of include/exclude (#31569)
For a long time the module configuration has had a few settings that use
`blacklist`/`whitelist` terminology. We've been asked by some of our users to replace
this with more inclusive language. In addition to being non-inclusive, `blacklist` and
`whitelist` are inconsistent with the rest of Spack, which uses `include` and `exclude`
for the same concepts.

- [x] Deprecate `blacklist`, `whitelist`, `blacklist_implicits` and `environment_blacklist`
      in favor of `exclude`, `include`, `exclude_implicits` and `exclude_env_vars` in module
      configuration, to be removed in Spack v0.20.
- [x] Print deprecation warnings if any of the deprecated names are in module config.
- [x] Update tests to test old and new names.
- [x] Update docs.
- [x] Update `spack config update` to fix this automatically, and include a note in the error
      that you can use this command.
2022-07-14 20:42:33 +00:00
Peter Scheibel
7741bfb7d1
Decompression: use tar exe vs. built-in Python tarfile support (#31563)
Python's built-in tarfile support doesn't address some general
cases of malformed tarfiles that are already handled by the system
'tar' utility; until these can be addressed, use that exclusively.
2022-07-13 14:12:21 -07:00
Massimiliano Culpo
7f2b5e8e57
spack.repo.get() can only be called on concrete specs (#31411)
The goal of this PR is to make clearer where we need a package object in Spack as opposed to a package class.

We currently instantiate a lot of package objects when we could make do with a class.  We should use the class
when we only need metadata, and we should only instantiate and us an instance of `PackageBase` at build time.

Modifications:
- [x] Remove the `spack.repo.get` convenience function (which was used in many places, and not really needed)
- [x] Use `spack.repo.path.get_pkg_class` wherever possible
- [x] Try to route most of the need for `spack.repo.path.get` through `Spec.package`
- [x] Introduce a non-data descriptor, that can be used as a decorator, to have "class level properties"
- [x] Refactor unit tests that had to be modified to reduce code duplication
- [x] `Spec.package` and `Repo.get` now require a concrete spec as input
- [x] Remove `RepoPath.all_packages` and `Repo.all_packages`
2022-07-12 19:45:24 -04:00
Harmen Stoppels
ea91e8f4ca
file_cache.py: idempotent remove without races (#31477)
There's a race condition in `remove()` as the lockfile is removed after
releasing the lock, which is a problem when another process acquires a
write lock during deletion.

Also simplify life a bit in multiprocessing when a file is possibly
removed multiple times, which currently is an error on the second
deletion, so the proposed fix is to make remove(...) idempotent and not
error when deleting non-existing cache entries.

Don't tests for existence of lockfile, cause windows/linux behavior is different
2022-07-12 15:03:48 +00:00
Harmen Stoppels
74e2625dcf
compression.py: buffered copy (#31533) 2022-07-12 16:49:59 +02:00
Harmen Stoppels
b2c519470b
environment.py: only acquire write lock when necessary (#31493) 2022-07-12 16:12:43 +02:00
Harmen Stoppels
d904a789d5
env depfile: add missing touch (#31494)
Oversight in #31433, the non-phony `env` target was missing a file being
created for it, which can cause make to infinitely loop when including
multiple generated makefiles.
2022-07-12 14:16:31 +02:00
Massimiliano Culpo
89b8d33d05
Remove os.getcwd from function signature (#31480)
fixes #29730

This is also wrong since it binds the directory at the time
of module import, rather than the one at the time of function
call.
2022-07-12 10:08:37 +02:00
Jonathon Anderson
25f198aa91
Sanitize ownership when extracting tarfiles (#31524) 2022-07-12 09:28:24 +02:00
Adam J. Stewart
5bd1074afb
py-hatchling: add new packages, document new build backend (#31512)
* py-hatchling: add new package, document new build backend

* Minor doc changes

Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>

Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
2022-07-11 16:23:28 -07:00
Cory Bloor
bb92ea59a2
Fix spack edit message when no editor installed (#31412)
When no default editor is installed and no environment variable is set,
which_string would return None and this would be passed to os.execv
resulting in a TypeError. The message presented to the user would be:

    Error: execv: path should be string, bytes or os.PathLike,
    not NoneType

This change checks that which_string has returned successfully before
attempting to execute the result, resulting in a new error message:

    Error: No text editor found! Please set the VISUAL and/or EDITOR
    environment variable(s) to your preferred text editor.

It's not strictly necessary, but I've also changed try_exec to catch
all errors rather than just OSErrors. This would have provided slightly
more context for the original error message.
2022-07-07 17:08:15 +02:00
Tamara Dahlgren
1b919d306f
Documentation: Add SourceforgePackage to the build systems docs (#31473) 2022-07-07 14:30:22 +02:00
Vanessasaurus
6b1e86aecc
removing feature bloat: monitor and analyzers (#31130)
Signed-off-by: vsoch <vsoch@users.noreply.github.com>

Co-authored-by: vsoch <vsoch@users.noreply.github.com>
2022-07-07 00:49:40 -06:00
Tamara Dahlgren
e68ed3268b
Testing Bugfix: refactor clean --python-cache to support all (#31449)
There were two choices: 1) remove '-p' from '-a' or 2) allow monkeypatching
the cleaning of the python cache since clean's test_function_calls isn't
supposed to be actually cleaning anything.

This commit supports the latter and adds a test case for `-p`.
2022-07-06 11:44:41 -07:00
Harmen Stoppels
dac31ef3c4
Remove fetch from depfile (#31433) 2022-07-05 14:48:32 +02:00
Alberto Invernizzi
bb3a663392
ROCmPackage: unify amdgpu_targets (#30582)
* unify amdgpu_targets for rocsolver and rocblas

* add more archs

* adapt all roc packages to unified amdgpu_targets
2022-07-05 14:33:30 +02:00
Scott Wittenburg
87b8321fa9
gitlab: Fix pipeline generation for non-develop protected pipelines (#31378)
Release branches and tags run protected pipelines, and we noticed
that those pipelines were generating all jobs in the stack, even
when the mirror contained all the built specs and an up to date
index.  The problem was caused because the override mirror was
not present in spacks mirror configuration at the time when the
binary_distribution.update() method was called.  This fixes that
by always adding the mirror override, if present, to the list of
configured mirrors.
2022-07-01 11:56:36 -07:00
Peter Scheibel
abeee9e1fd
Cray manifest: compiler duplicates (#31173)
* remove unhelpful comment
* Filter compiler duplicates while reading manifest
* more-specific version matching edited to use module-specific version (to avoid an issue where a user might add a compiler with the same version to the initial test configuration
2022-06-29 17:05:29 -07:00
Adam J. Stewart
93c16f1ee3
PythonPackage: add default libs/headers attributes (#28527)
* PythonPackage: add default libs/headers attributes

* Style fix

* libs and headers should be properties

* Check both platlib and include

* Fix variable reference
2022-06-29 11:55:24 -07:00
Massimiliano Culpo
557abe04ec
Update containerize templates to account for view indirection (#31321)
fixes #30965
2022-06-29 15:39:18 +02:00
Wouter Deconinck
8f7f96ae2b
oneAPI supports cxx17_flag and cxx20_flag (#31274) 2022-06-28 10:33:19 +02:00
Greg Becker
df44045fdb
Feature: use git branches/tags as versions (#31200)
Building on #24639, this allows versions to be prefixed by `git.`. If a version begins `git.`, it is treated as a git ref, and handled as git commits are starting in the referenced PR.

An exception is made for versions that are `git.develop`, `git.main`, `git.master`, `git.head`, or `git.trunk`. Those are assumed to be greater than all other versions, as those prefixed strings are in other contexts.
2022-06-28 01:54:41 +00:00
Sergey Kosukhin
7fc68240fe
oneapi-* packages: improve use with modules (#30981)
This commit adds some changes which improve use of Spack-installed
oneAPI packages with Spack-generated modules, but not within Spack
(e.g. if you install some of these packages with Spack, then load
their associated modules to build other packages outside of Spack).

The majority of the PR diff is refactoring. The functional changes
are:

* intel-oneapi-mkl:
  * setup_run_environment: update Intel compiler flags to RPATH the
    mkl libs
* intel-oneapi-compilers: update the compiler configuration to RPATH
  libraries installed by this package (note that Spack already handled
  this when installing dependent packages with Spack, but this is
  specifically to use the intel-oneapi-compilers package outside
  of Spack). Specifically:
  * inject_rpaths: this modifies the binaries installed as part of
    the intel-oneapi-compilers package to RPATH libraries provided
    by this package
  * extend_config_flags: this instructs the compiler executables
    provided by this package to RPATH those same libraries

Refactoring includes:

* intel-oneapi-compilers: in addition to the functional changes,
  a large portion of this diff is reorganizing the creation of
  resources/archives downloaded for the install
* The base oneAPI package renamed component_path, so several packages
  changed as a result:
  * intel-oneapi-dpl
  * intel-oneapi-dnn
  * extrae
* intel-oneapi-mpi:
  * Perform file filtering in one pass rather than two
2022-06-27 16:34:15 -07:00
Peter Scheibel
f7ef064bdc
spack external find: handle manifest with bad permissions (#31201)
Allow `spack external find` (with no extra args) to proceed if the manifest file exists but
without sufficient permissions; in that case, print a warning. Also add a test for that behavior.

TODOs:

- [x] continue past any exception raised during manifest parsing as part of `spack external find`, 
      except for CTRL-C (and other errors that indicate immediate program termination)
- [x] Semi-unrelated but came up when discussing this with the user who reported this issue to
      me: the manifest parser now accepts older schemas 

See: https://github.com/spack/spack/issues/31191
2022-06-27 19:26:12 +00:00
Massimiliano Culpo
11d71ca85e
Rename os_facts.lp to os_compatibility.lp (#31206)
Follow-up from #31170
2022-06-27 09:15:25 -07:00
Cody Balos
147f39d7aa
Fix typo in documentation note about concretizer:unify (#31246) 2022-06-24 11:31:43 +02:00
Massimiliano Culpo
70650cacbd
ASP-based solver: rescale target weights so that 0 is always the best score (#31226)
fixes #30997

Instead of giving a penalty of 30 to all nodes when preferences
are not package specific, give a penalty of 100 to all targets
of a node where we have package specific preferences, if the target
is not explicitly preferred.
2022-06-23 23:33:36 -07:00
eugeneswalker
7ef52acfce
relocation: x-pie-executable needs relocation (#31248) 2022-06-22 23:55:28 +00:00
Adam J. Stewart
ef278c601c
spack create: fix for no URL (#31239) 2022-06-22 18:45:03 +00:00
Massimiliano Culpo
55cac3b098
Remove update functions used to ease the transition in v0.16 (#31216) 2022-06-22 11:15:57 +02:00
Massimiliano Culpo
649760dc1a
Canonicalize positional argument to spack bootstrap mirror (#31180)
fixes #31139
2022-06-21 19:25:01 +02:00
Massimiliano Culpo
aebb601b70
Stricter compatibility rules for OS and compiler when reusing specs (#31170)
* Stricter compatibility rules for OS and compiler when reusing specs
* Add unit test
2022-06-20 17:32:13 -07:00
Christian Goll
0fe57962b3
OpenSUSE Tumbleweed: use GLIBC version as distro version (#19895)
Tumbleweed is a rolling release that would have used a date 
as a version instead.
2022-06-20 22:20:31 +02:00
Sam Broderick
760a12c440
Fix request for bzip2, since bzip was pulled due to patent issues (#31198) 2022-06-20 08:13:20 +00:00
Brian Van Essen
15c35a3cff
Bugfix external find --all for libraries (#31186)
* Fixed a bug in the 'external find --all' command where the call failed
to find packages by both executable and library. The bug was that the
call `path.all_packages()` incorrectly turned the variable
`packages_to_check` into a generator rather than keeping it a list.
Thus the second call to `detection.by_library` had no work to do.

* Fixed the help message for the find external and compiler commands as
well as others that used the `scopes_metavar` field to define where
the results should be stored in configuration space.  Specifically,
the fact that configuration could be added to the environment was not
mentioned in the help message.
2022-06-17 21:52:06 +00:00