Commit graph

4056 commits

Author SHA1 Message Date
Peter Scheibel
ae82650174
Update fetch order to match iteration order of MirrorReference (#17572) 2020-07-23 10:58:59 -07:00
Greg Becker
e8aa737b09
util.executable.which: handle path separators like /bin/which (#17668)
* util.executable.which: handle path separators like /bin/which

Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2020-07-23 10:54:25 -07:00
Tamara Dahlgren
605c1a76e0
Reduce output verbosity with debug levels (#17546)
* switch from bool to int debug levels

* Added debug options and changed lock logging to use more detailed values

* Limit installer and timestamp PIDs to standard debug output

* Reduced verbosity of fetch/stage/install output, changing most to debug level 1

* Combine lock log methods; change build process install to debug

* Changed binary cache install messages to extraction messages
2020-07-23 00:49:57 -07:00
eugeneswalker
f1eec05d0e
bugfix: use getattr for variation.prefix/suffix (#17669) 2020-07-22 16:15:25 -07:00
Todd Gamblin
0c44a9a504
bugfix: make compiler preferences slightly saner (#17590)
* bugfix: make compiler preferences slightly saner

This fixes two issues with the way we currently select compilers.

If multiple compilers have the same "id" (os/arch/compiler/version), we
currently prefer them by picking this one with the most supported
languages.  This can have some surprising effects:

* If you have no `gfortran` but you have `gfortran-8`, you can detect
  `clang` that has no configured C compiler -- just `f77` and `f90`. This
  happens frequently on macOS with homebrew. The bug is due to some
  kludginess about the way we detect mixed `clang`/`gfortran`.

* We can prefer suffixed versions of compilers to non-suffixed versions,
  which means we may select `clang-gpu` over `clang` at LLNL. But,
  `clang-gpu` is not actually clang, and it can break builds. We should
  prefer `clang` if it's available.

- [x] prefer compilers that have C compilers and prefer no name variation
  to variation.

* tests: add test for which()
2020-07-21 18:48:37 -07:00
Harmen Stoppels
6c69b8a4d4
ci pipelines: activate environment without view (#17440) 2020-07-21 10:15:43 -07:00
Todd Gamblin
897e80e596
bugfix: ignore Apple's "gcc" by default (#17589)
Apple's gcc is really clang. We previously ignored it by default but
there was a regression in #17110.

Originally we checked for all clang versions with this, but I know of
none other than `gcc` on macos that actually do this, so limiting to
`apple-clang` should be ok.

- [x] Fix check for `apple-clang` in `gcc.py` to use version detection
  from `spack.compilers.apple_clang`
2020-07-20 18:24:18 -07:00
Dr. Christian Tacke
bd236918dd
Configuration: allow usage of command-line scopes with environments (#14608)
Spack did not support usage of the `--config-scope` option in
combination with an environment: In `lib/spack/spack/main.py`,
`spack.config.command_line_scopes` is set equal to any config scopes
passed by the `--config-scope` option. However, this is done after
activating an environment. In the process of activating an environment,
the `spack.config.config` singleton is instantiated, so later setting of
`spack.config.command_line_scopes` is ignored.

This commit sets command line scopes before activating an environment to
ensure that they are included in the configuration.

Co-authored-by: Tim Fuller <tjfulle@sandia.gov>
2020-07-20 13:58:06 -07:00
eugeneswalker
96fa6f0c1b
allow GNUPGHOME to come from SPACK_GNUPGHOME in env, if set (#17139) 2020-07-17 13:29:30 -07:00
Scott Wittenburg
b5f82696e2
Bugfix/install missing compiler from buildcache (#17536)
Ensure compilers installed from buildcache are registered.
2020-07-17 11:13:36 -07:00
Scott Wittenburg
ae03782032 buildcache: list all mirrors even if one fails 2020-07-17 10:04:05 -06:00
Harmen Stoppels
1fcc00df96
Fix security issue in CI (#17545)
The `spack-build-env.txt` file may contains many secrets, but the obvious one is the private signing key in `SPACK_SIGNING_KEY`. This file is nonetheless uploaded as a build artifact to gitlab. For anyone running CI on a public version of Gitlab this is a major security problem. Even for private Gitlab instances it can be very problematic.

Co-authored-by: Scott Wittenburg <scott.wittenburg@kitware.com>
2020-07-16 17:27:37 -07:00
Axel Huebl
a6abd530bd
CUDA 11.0.2 (#17423)
- [x] wait for general release candidate
- [x] compute capability support
- [x] compiler conflicts
  - [x] ppc64le
- [x] new download links
2020-07-13 18:32:28 -05:00
Harmen Stoppels
2b809a5374
Add -o flag to tar decompressor (#17427)
For normal users, `-o` or `--no-same-owner` (GNU extension) is
the default behavior, but for the root user, `tar` attempts to preserve
the ownership from the tarball.

This makes `tar` use `-o` all the time.  This should improve untarring
files owned by users not available in rootless Docker builds.
2020-07-13 15:19:04 -07:00
Omar Padron
573489db71
Add libglvnd packages/Add EGL support (#14572)
* add new package: "libglvnd-frontend"

* add +glvnd variant to opengl package

* add +glvnd variant to mesa package

* add +egl variant to paraview package

* add libglvnd-frontend entries to default packages config

* fix style

* add default providers for glvnd virtuals

add default providers for glvnd-gl, glvnd-glx, and glvnd-egl

* WIP: rough start to external OpenGL documentation

* rename libglvnd-frontend package and backend virtual dependencies

* update documentation

* fix ligvnd-be-* typos

* fix libglvnd-fe package class name

* fix doc parse error
2020-07-13 11:32:36 -04:00
Greg Becker
406596af70
update docs on point releases (#17463) 2020-07-11 14:35:25 -07:00
Peter Josef Scheibel
5e50dc5acb Merge branch 'releases/v0.15' into develop 2020-07-10 23:14:36 -07:00
Peter Josef Scheibel
1741279f16 Bump version to 0.15.1; update CHANGELOG and version references 2020-07-10 16:54:49 -07:00
Greg Becker
c2393fe566 spack install: improve error message with no args (#17454)
The error message was not updated when the behavior of Spack environments
was changed to not automatically activate the local environment in #17258.
The previous error message no longer makes sense.
2020-07-10 13:05:49 -07:00
Greg Becker
afbb4a5cba installation: skip repository metadata for externals (#16954)
When Spack installs a package, it stores repository package.py files
for it and all of its dependencies - any package with a Spack metadata
directory in its installation prefix.

It turns out this was too broad: this ends up including external
packages installed by Spack (e.g. installed by another Spack instance).
Currently Spack doesn't store the namespace properly for such packages,
so even though the package file could be fetched from the external,
Spack is unable to locate it.

This commit avoids the issue by skipping any attempt to locate and copy
from the package repository of externals, regardless of whether they
have a Spack repo directory.
2020-07-10 13:05:49 -07:00
Michael Kuhn
4e4de51f0d autotools bugfix: handle missing config.guess (#17356)
Spack was attempting to calculate abspath on the located config.guess
path even when it was not found (None); this commit skips the abspath
calculation when config.guess is not found.
2020-07-10 13:05:49 -07:00
Michael Kuhn
44681dbca5 autotools: Fix config.guess detection, take two (#17333)
The previous fix from #17149 contained a thinko that produced errors for
packages that overwrite configure_directory.
2020-07-10 13:05:49 -07:00
Greg Becker
f2889e698a
spack install: improve error message with no args (#17454)
The error message was not updated when the behavior of Spack environments
was changed to not automatically activate the local environment in #17258.
The previous error message no longer makes sense.
2020-07-10 10:45:11 -07:00
Patrick Gartung
e72e2568dd
Relocation of sbang needs to be done when the spack prefix changes even if the install tree has not changed. (#17455) 2020-07-09 22:28:51 -05:00
Harmen Stoppels
84eae97f91
aspirin for buildaches (#17437) 2020-07-09 22:00:38 -05:00
Sajid Ali
12099ed55e
clear mpicc and friends before each build (#17450)
* clear mpi env vars
2020-07-09 16:14:49 -05:00
Greg Becker
d0f5b69a19
installation: skip repository metadata for externals (#16954)
When Spack installs a package, it stores repository package.py files
for it and all of its dependencies - any package with a Spack metadata
directory in its installation prefix.

It turns out this was too broad: this ends up including external
packages installed by Spack (e.g. installed by another Spack instance).
Currently Spack doesn't store the namespace properly for such packages,
so even though the package file could be fetched from the external,
Spack is unable to locate it.

This commit avoids the issue by skipping any attempt to locate and copy
from the package repository of externals, regardless of whether they
have a Spack repo directory.
2020-07-09 11:08:51 -07:00
Patrick Gartung
8c41173678
Buildcache: bindist test without invoking spack compiler wrappers. (#15687)
* Buildcache:
   * Try mocking an install of quux, corge and garply using prebuilt binaries
   * Put patchelf install after ccache restore
   * Add script to install patchelf from source so it can be used on Ubuntu:Trusty which does not have a patchelf pat package. The script will skip building on macOS
   * Remove mirror at end of bindist test
   * Add patchelf to Ubuntu build env
   * Revert mock patchelf package to allow other tests to run.
   * Remove depends_on('patchelf', type='build') relying instead on
   * Test fixture to ensure patchelf is available.

* Call g++ command to build libraries directly during test build

* Flake8

* Install patchelf in before_install stage using apt unless on Trusty where a build is done.

* Add some symbolic links between packages

* Flake8

* Flake8:

* Update mock packages to write their own source files

* Create the stage because spec search does not create it any longer

* updates after change of list command arguments

* cleanup after merge

* flake8
2020-07-08 15:05:58 -05:00
Adam J. Stewart
207e496162
spack create: ask how many to download (#17373) 2020-07-08 09:38:42 +02:00
Simon Byrne
f85da868ac
Improve Travis sample in the docs (#17420)
- printf is better than echo for multiline strings
- ** should be &&
- use line continuation
- Use multiline block
2020-07-08 07:25:37 +02:00
Patrick Gartung
7f8e827db8
Relocate rpaths for all binaries, then do text bin replacement if the rpaths still exist after running patchelf/otool (#17418) 2020-07-07 16:46:39 -05:00
Massimiliano Culpo
f4ac3770b4
CudaPackage: maintainers are listed in the docstring (#17409)
fixes #17396

This prevents the class attribute to be inherited and
saves current maintainers from becoming the default
maintainers of every Cuda package.
2020-07-07 20:45:41 +02:00
Todd Gamblin
b0506a722e
releases: document releases/latest tag (#17402)
We got rid of `master` after #17377, but users still want a way to get
the latest stable release without knowing its number.

We've added a `releases/latest` tag to replace what was once `master`.

Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2020-07-07 11:44:15 -07:00
Peter Scheibel
650ab563f4
Uninstall: tolerate hook failures when force=true (#16513)
Fixes #16478

This allows an uninstall to proceed even when encountering pre-uninstall
hook failures if the user chooses the --force option for the uninstall.

This also prevents post-uninstall hook failures from raising an exception,
which would terminate a sequence of uninstalls. This isn't likely essential
for #16478, but I think overall it will improve the user experience: if
the post-uninstall hook fails, there isn't much point in terminating a
sequence of spec uninstalls because at the point where the post-uninstall
hook is run, the spec has already been removed from the database (so it
will never have another chance to run).

Notes:

* When doing spack uninstall -a, certain pre/post-uninstall hooks aren't
  important to run, but this isn't easy to track with the current model.
  For example: if you are uninstalling a package and its extension, you
  do not have to do the activation check for the extension.
* This doesn't handle the uninstallation of specs that are not in the DB,
  so it may leave "dangling" specs in the installation prefix
2020-07-07 11:37:36 -07:00
Todd Gamblin
9ec9327f5a docs: document releases and branches in Spack
- [x] Remove references to `master` branch
- [x] Document how release branches are structured
- [x] Document how to make a major release
- [x] Document how to make a point release
- [x] Document how to do work in our release projects
2020-07-06 11:39:19 -07:00
Todd Gamblin
11088df402 Remove references to master from CI
- [x] remove master from github actions
- [x] remove master from .travis.yml
- [x] make `develop` the default branch for `spack ci`
2020-07-06 11:39:19 -07:00
Todd Gamblin
4ea76dc95c change master/child to controller/minion in pty docstrings
PTY support used the concept of 'master' and 'child' processes. 'master'
has been renamed to 'controller' and 'child' to 'minion'.
2020-07-06 11:39:19 -07:00
Michael Kuhn
516c3e659f
autotools bugfix: handle missing config.guess (#17356)
Spack was attempting to calculate abspath on the located config.guess
path even when it was not found (None); this commit skips the abspath
calculation when config.guess is not found.
2020-07-06 10:53:02 -07:00
Shahzeb Siddiqui
04f3000646
Pipelines doc: fixed two broken links (#17355) 2020-07-03 12:29:45 +02:00
Michael Kuhn
10016a34e0
autotools: Fix config.guess detection, take two (#17333)
The previous fix from #17149 contained a thinko that produced errors for
packages that overwrite configure_directory.
2020-07-02 00:45:41 -07:00
Massimiliano Culpo
a5eabfad91
Moved flake8, shell and documentation tests to Github Action (#17328)
* Move flake8 tests on Github Actions

* Move shell test to Github Actions

* Moved documentation build to Github Action

* Don't run coverage on Python 2.6

Since we get connection errors consistently on Travis
when trying to upload coverage results for Python 2.6,
avoid computing coverage entirely to speed-up tests.
2020-07-01 11:58:53 -05:00
Adam J. Stewart
6a77f1ff45
Fix hashlib function capitalization (#17323) 2020-07-01 09:46:20 -05:00
Glenn Johnson
60283775b3
Documentation update for container example (#17321)
This updates the documentation to reflect #17316.
2020-07-01 08:40:36 +02:00
Gregory Becker
49aebb44b1 bump version number 2020-06-30 18:21:32 -05:00
Massimiliano Culpo
486b4671b6
Don't detect "classic" on Cray to avoid a compiler bug (#17314)
* Don't detect "classic" on Cray to avoid a compiler bug

* add tests

Co-authored-by: Gregory Becker <becker33@llnl.gov>
2020-06-30 15:45:29 -05:00
Johannes Blaschke
1d55adfd2b
Add fish shell support (#9279)
* share/spack/setup-env.fish file to setup environment in fish shell

* setup-env.fish testing script

* Update share/spack/setup-env.fish

Co-Authored-By: Elsa Gonsiorowski, PhD <gonsie@me.com>

* Update share/spack/qa/setup-env-test.fish

Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>

* updates completions using `spack commands --update-completion`

* added stderr-nocaret warning

* added fish shell tests to CI system


Co-authored-by: becker33 <becker33@llnl.gov>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Elsa Gonsiorowski, PhD <gonsie@me.com>
2020-06-30 14:26:27 -05:00
Greg Becker
212299a021
Cray compiler: fix implicit rpaths for classic versions (#17310)
* check link dirs for existence
2020-06-30 12:46:20 -05:00
Greg Becker
b52390dacc
cray compilers: fix bug with verifying cray compilers (#17303)
* fix bug with verifying cray compilers
2020-06-29 20:26:46 -05:00
Greg Becker
d71fdc9719
remove three commands that have been deprecated since v0.13.0 (#17291)
* remove three commands that have been deprecated since v0.13.0
2020-06-29 11:15:56 -05:00
Michael Kuhn
b07d38b3be
autotools: Fix config.guess detection (#17149)
The config.guess detection used a relative path that did not work in
combination with `check_call`. Use an absolute path instead.
2020-06-29 10:08:22 -05:00