Commit graph

5840 commits

Author SHA1 Message Date
John W. Parent
793a7bc6a9
Windows: add registry query and SDK/WDK packages (#33021)
* Add a WindowsRegistryView class, which can query for existing
  package installations on Windows. This is particularly important
  because some Windows packages (including those added here)
  do not allow two simultaneous installs, and this can be
  queried in order to provide a clear error message.
* Consolidate external path detection logic for Windows into
  WindowsKitExternalPaths and WindowsCompilerExternalPaths objects.
* Add external-only packages win-sdk and wgl
* Add win-wdk (including external detection) which depends on
  win-sdk
* Replace prior msmpi implementation with a source-based install
  (depends on win-wdk). This install can control the install
  destination (unlike the binary installation).
* Update MSVC compiler to choose vcvars based on win-sdk dependency
* Provide "msbuild" module-level variable to packages during build
* When creating symlinks on Windows, need to explicitly specify when
  a symlink target is a directory
* executables_in_path no-longer defaults to using PATH (this is
  now expected to be taken care of by the caller)
2022-11-22 00:27:42 -08:00
Harmen Stoppels
44c22a54c9
Spec traversal: add option for topological ordering (#33817)
Spec traversals can now specify a topological ordering. A topologically-
ordered traversal with input specs X1, X2... will

* include all of X1, X2... and their children
* be ordered such that a given node is guaranteed to appear before any
  of its children in the traversal

Other notes:

* Input specs can be children of other input specs (this is useful if
  a user specifies a set of specs to uninstall: some of those specs
  might be children of others)
* `direction="parents"` will produce a reversed topological order
  (children always come before parents).
* `cover="edges"` will generate a list of edges L such that (a) input
  edges will always appear before output edges and (b) if you create
  a list with the destination of each edge in L the result is
  topologically ordered
2022-11-21 18:33:35 -08:00
Massimiliano Culpo
0e4ee3d352
Speed-up a few unit-tests (#34044)
* test_suite.py: speed up slow test by using mock packages

* Don't resolve the sha during unit-tests

* Skip long-running test that fails, instead of executing it
2022-11-21 23:50:55 +01:00
Scott Wittenburg
6411cbd803
ci: restore ability to reproduce gitlab job failures (#33953) 2022-11-21 10:39:03 -06:00
Harmen Stoppels
8ea366b33f
uninstall: fix accidental cubic complexity (#34005)
* uninstall: fix accidental cubic complexity

Currently spack uninstall runs in worst case cubic time complexity
thanks to traversal during traversal during traversal while collecting
the specs to be uninstalled.

Also brings down the number of error messages printed to something
linear in the amount of matching specs instead of quadratic.
2022-11-21 16:44:48 +01:00
Massimiliano Culpo
45b4cedb7e
spack find: remove deprecated "--bootstrap" option (#34015) 2022-11-19 16:09:34 +01:00
Chris Green
409cf185ce
package_base.py: Fix #34006: test msg needs to be a string (#34007) 2022-11-19 13:02:51 +01:00
Chris Green
e715901cb2
PackageBase should not define builder legacy attributes (#33942)
* Add a regression test for 33928

* PackageBase should not set `(build|install)_time_test_callbacks`

* Fix audits by preserving the current semantic

Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2022-11-18 22:22:51 +01:00
Massimiliano Culpo
839a14c0ba
Improve error message for requirements (#33988)
refers #33985
2022-11-18 15:49:46 +01:00
Jared Popelar
381bedf369
Hdf5 package: build on Windows (#31141)
* Enable hdf5 build (including +mpi) on Windows
* This includes updates to hdf5 dependencies openssl (minor edit) and
  bzip2 (more-extensive edits)
* Add binary-based installation of msmpi (this is currently the only
  supported MPI implementation in Spack for Windows). Note that this
  does not install to the Spack-specified prefix. This implementation
  will be replaced with a source-based implementation

Co-authored-by: John Parent <john.parent@kitware.com>
2022-11-17 10:40:53 -08:00
Chris Green
22aada0e20
Waf build system: fix typo in legacy_attributes (#33958)
Fix erroneous duplication of `build_time_test_callbacks` in
`legacy_attributes`: one of the duplicates should be
`install_time_test_callbacks`
2022-11-17 16:54:46 +01:00
Harmen Stoppels
cc2d0eade6
docs: fix typo in multiple build systems (#33965) 2022-11-17 15:20:10 +01:00
Harmen Stoppels
f00e411287
relocate.py: small refactor for file_is_relocatable (#33967) 2022-11-17 13:21:27 +01:00
Massimiliano Culpo
da0a6280ac
Remove deprecated subcommands from "spack bootstrap" (#33964)
These commands are slated for removal in v0.20
2022-11-17 12:42:57 +01:00
Michael Kuhn
93eecae0c3
Add sgid notice when running on AFS (#30247) 2022-11-17 09:17:41 +01:00
Brian Vanderwende
c721aab006
lib/spack/spack/store.py: Fix #28170 for padding relocation (#33122) 2022-11-17 03:56:00 +01:00
Greg Becker
1263b5c444
initial implementation of slingshot detection (#33793) 2022-11-16 13:01:37 -08:00
psakievich
90f0a8eacc
Upstreams: add canonicalize path (#33946) 2022-11-16 14:37:32 -06:00
John W. Parent
61a7420c94
Windows bootstrapping: remove unneeded call to add dll to PATH (#33622)
#32942 fixed bootstrapping on Windows by having the core Spack
code explicitly add the Clingo package bin/ directory as a
DLL path.

Since then, #33400 has been merged, which ensures that the Python
module installed by the Spack `clingo` package can find the DLLs
in bin/.

Note that this only works for Spack instances which have been
bootstrapped after #33400: for installations bootstrapped before
then, you will need to run `spack clean -b` (this would only
be needed for Spack instances running on Windows).
2022-11-16 12:04:57 -08:00
Stephen Sachs
39a1f1462b
SIP build system: fix "python not defined in builder" (#33906)
Co-authored-by: Stephen Sachs <stesachs@amazon.com>
2022-11-16 13:36:25 -06:00
Harmen Stoppels
0a952f8b7b
docs updates for spack env depfile (#33937) 2022-11-16 15:47:31 +01:00
Harmen Stoppels
d18cccf7c5
spack env depfile in Gitlab CI should use install-deps/pkg-version-hash target (#33936) 2022-11-16 14:02:56 +00:00
Kevin Broch
fbe6b4b486
Change code suggestions to output black formatter compliant code (#33931) 2022-11-16 12:08:20 +01:00
Adam J. Stewart
93430496e2
Getting Started: Python 2 is no longer supported (#33927) 2022-11-16 08:36:49 +01:00
Michael Kuhn
901b31a7aa
docs: fix typo (#33926) 2022-11-15 16:06:12 -08:00
Harmen Stoppels
a0182c069f
Show time per phase (#33874) 2022-11-15 21:10:07 +01:00
Harmen Stoppels
af74680405
depfile: improve tab completion (#33773)
This PR allows you to do:

```
spack env create -d .
spack -e . add python
spack -e . concretize
spack -e . env depfile -o Makefile

make in<tab>              # -> install
make install-<tab>        # -> install-deps/
make install-deps/py<tab> # -> install-deps/python-x.y.z-hash
make install/zl<tab>      # -> install/zlib-x.y.z-hash

make SP<tab>              # -> make SPACK
make SPACK_<tab>          # -> make SPACK_INSTALL_FLAGS=
```
2022-11-15 18:03:17 +01:00
Harmen Stoppels
d1715c5fdf
Fixup: start the timer before the phase (#33917) 2022-11-15 16:52:43 +01:00
Harmen Stoppels
b245f1ece1
Fix incorrect timer (#33900)
Revamp the timer so we always have a designated begin and end.

Fix a bug where the phase timer was stopped before the phase started,
resulting in incorrect timing reports in timers.json.
2022-11-15 16:33:47 +01:00
Brian Van Essen
fd4f905ce5
External find now searches all dynamic linker paths (#33800)
Add spack.ld_so_conf.host_dynamic_linker_search_paths

Retrieve the current host runtime search paths for shared libraries;
for GNU and musl Linux we try to retrieve the dynamic linker from the
current Python interpreter and then find the corresponding config file
(e.g. ld.so.conf or ld-musl-<arch>.path). Similar can be done for
BSD and others, but this is not implemented yet. The default paths
are always returned. We don't check if the listed directories exist.

Use this in spack external find for libraries.

Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
2022-11-15 14:48:15 +01:00
Harmen Stoppels
2948248d7a
Remove exit(0) (#33896)
Since they cause --backtrace to report backtraces even with exit code 0
2022-11-15 14:35:44 +01:00
Massimiliano Culpo
b3124bff7c
Stop using six in Spack (#33905)
Since we dropped support for Python 2.7, there's no need
so use `six` anymore. We still need to vendor it until
we update our vendored dependencies.
2022-11-15 10:07:54 +01:00
Scott Wittenburg
5c4137baf1
gitlab: Add shared PR mirror to places pipelines look for binaries. (#33746)
While binaries built for PRs that get merged must still be rebuilt
in develop pipelines, they can be used by other PRs that find they
would otherwise need to rebuild them.  Now that spackbot is
managing copying PR binaries from merged PRs into a shared location,
keeping it pruned to a reasonable size, and making sure the indices
are up to date, spack can use these mirrors as a potential source
of binaries.
2022-11-14 19:37:23 -07:00
psakievich
2460c4fc28
Add $date option to the list of config variables (#33875)
I'm finding I often want the date in my paths and it would be nice if spack had a config variable for this.

Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
2022-11-14 10:13:30 -08:00
Cory Bloor
6e39efbb9a
rocm: add all GFX9, GFX10 and GFX11 amdgpu_targets (#33871)
This change adds all documented AMDGPU processors from GFX9 through GFX11 and sorts the list.
2022-11-14 10:11:22 -08:00
Todd Gamblin
6ecb57e91f Bump version to v0.20.0.dev0 2022-11-14 08:22:29 -06:00
Harmen Stoppels
a75af62fe3
Get rid of context for exceptions outside PackageBase (#33887) 2022-11-14 14:11:22 +01:00
Massimiliano Culpo
e4e02dbeae
Fix a bug/typo in a config_values.py fixture (#33886) 2022-11-14 05:26:14 -07:00
Massimiliano Culpo
3efa4ee26f
Remove support for running with Python 2.7 (#33063)
* Remove CI jobs related to Python 2.7

* Remove Python 2.7 specific code from Spack core

* Remove externals for Python 2 only

* Remove llnl.util.compat
2022-11-14 13:11:28 +01:00
Harmen Stoppels
a4cec82841
Speed up traverse unit tests (#33840) 2022-11-14 09:34:14 +01:00
Todd Gamblin
3d811617e6
hotfix: ensure that schema is compatible with tutorial VM config
We added a hotfix to releases/v0.19 with a feature flag, but the flag
is incompatible with the config schema on `develop`.

- [x] Ensure schema is compatible on develop even though config option is unused.
2022-11-13 09:14:00 -06:00
Massimiliano Culpo
03224e52d2
Speed-up bootstrap and architecture unit tests (#33865)
* Speed-up bootstrap mirror unit test

The unit test doesn't need to concretize, since it checks
only metadata for the mirror.

* architecture.py: use "default_mock_concretization" for slow test
2022-11-13 13:09:22 +01:00
Greg Becker
4ebe57cd64
update tutorial command to newest release branch (#33867) 2022-11-12 13:29:38 -08:00
Greg Becker
343cd04a54
use spack.version_info as source of version truth for spack tutorial command (#33860)
* Use spack.spack_version_info as source of truth

Co-authored-by: Todd Gamblin <gamblin2@llnl.gov>
2022-11-12 11:42:59 -08:00
Massimiliano Culpo
022a2d2eaf
Speed-up unit tests by caching default mock concretization (#33755) 2022-11-11 21:32:40 +01:00
Greg Becker
2d2c591633
ensure view projections for extensions always point to extendee (#33848) 2022-11-11 10:59:23 -07:00
Harmen Stoppels
c14dc2f56a
docs: updates related to extensions (#33837) 2022-11-11 13:24:17 +01:00
Harmen Stoppels
0f54a63dfd
remove activate/deactivate support in favor of environments (#29317)
Environments and environment views have taken over the role of `spack activate/deactivate`, and we should deprecate these commands for several reasons:

- Global activation is a really poor idea:
   - Install prefixes should be immutable; since they can have multiple, unrelated dependents; see below
   - Added complexity elsewhere: verification of installations, tarballs for build caches, creation of environment views of packages with unrelated extensions "globally activated"... by removing the feature, it gets easier for people to contribute, and we'd end up with fewer bugs due to edge cases.
- Environment accomplish the same thing for non-global "activation" i.e. `spack view`, but better.

Also we write in the docs:

```
However, Spack global activations have two potential drawbacks:

#. Activated packages that involve compiled C extensions may still
   need their dependencies to be loaded manually.  For example,
   ``spack load openblas`` might be required to make ``py-numpy``
   work.

#. Global activations "break" a core feature of Spack, which is that
   multiple versions of a package can co-exist side-by-side.  For example,
   suppose you wish to run a Python package in two different
   environments but the same basic Python --- one with
   ``py-numpy@1.7`` and one with ``py-numpy@1.8``.  Spack extensions
   will not support this potential debugging use case.
```

Now that environments are established and views can take over the role of activation
non-destructively, we can remove global activation/deactivation.
2022-11-11 00:50:07 -08:00
Greg Becker
f11778bb02
remove deprecated top-level module config (#33828)
* remove deprecated top-level module config per deprecation in 0.18
2022-11-10 23:35:33 -08:00
Greg Becker
277234c044
remove deprecated concretization environment key (#33774) 2022-11-11 00:11:52 +00:00