Commit graph

3812 commits

Author SHA1 Message Date
Michael Kuhn
545fae8407
spack checksum: Use package's fetch_options (#15481)
This makes sure that a package's fetch_options are used when fetching
new versions to checksum. This allows working around problems with
slow servers or those requiring a cookie to be set.
2020-03-20 12:11:42 -07:00
Dr. Christian Tacke
03945c2541
Show the packaga name for a missing patch sha256 (#15441)
When a patches sha256 is missing, also show the
packagename, so that analyzing is easier.
2020-03-20 11:39:04 -05:00
Greg Becker
e5f8b093a9
Cray bugfix: TERM missing while reading default target (#15381)
Bug: Spack hangs on some Cray machines

Reason: The TERM environment variable is necessary to run bash -lc "echo $CRAY_CPU_TARGET", but we run that command within env -i, which wipes the environment.

Fix: Manually forward the TERM environment variable to env -i /bin/bash -lc "echo $CRAY_CPU_TARGET"
2020-03-19 15:11:50 -07:00
Todd Gamblin
3255eaba0d refactor: spack.package.possible_dependencies() handles virtuals
- [x] move some logic for handling virtual packages from the `spack
  dependencies` command into `spack.package.possible_dependencies()`

- [x] rework possible dependencies tests so that expected and actual
  output are on the left/right respectively
2020-03-19 09:00:40 -07:00
Todd Gamblin
7cfa497912 bugfix: spack.package.possible_dependencies() should return virtuals
`spack.package.possible_dependencies()` was forgetting to pass the
`visited` dict to recursive calls; this fixes it
2020-03-19 09:00:40 -07:00
Patrick Gartung
8bffa57ac4
Buildcache: fix relocation of binaries with relative rpaths (#15558) 2020-03-18 22:37:09 -05:00
Kai Germaschewski
3df712a385
Upstreams: don't write metadata directory to upstream DB (#15526)
When trying to use an upstream Spack repository, as of f2aca86 Spack
was attempting to write to the upstream DB based on a new metadata
directory added in that commit. Upstream DBs are read-only, so this
should not occur.

This adds a check to prevent Spack from writing to the upstream DB
2020-03-18 16:41:18 -07:00
Sinan
056fc99676
Use python extend_path as pyqt sip fix (#15297)
* try extend path to solve PyQt5.sip not found issue

* disable private sip installation in sippackage class

* undo manual PyQt5 dir creation in py-sip site-packages dir

* fix typo

* fix typo

* also apply fix to PyQt4

* tidy up

* flake8 and tidy up

* tidy and undo hardcoding of python_include_dir

* replace hardcoded python inc dir

* fix minor issues

* rethink include dir variable name

* improve style

* add new versions

* implement new sip setup to qsci installation

* set sip-incdir correctly for the new setup

* setup extend_path thing before qsci python bindings

* take care of conflict

* flake8

* also extend for PyQt4

* improve style

* improve style

* SipPackage build sys should depend on py-sip

* consolidate extend_path fixes into SipPackage

* fix typo

* fix bugs

* flake8

* revert sip doc to pre-resource setup

* import os module

* flake8

Co-authored-by: Sinan81 <sbulut@3vgeomatics.com>
2020-03-18 13:19:27 -05:00
Patrick Gartung
be8841cbc5
Remove PatchelfError and change patchelf error message to a warning. (#15518) 2020-03-17 10:49:24 -05:00
Patrick Gartung
e48d24ee45
Warn only if link target is not relative and outside of the install prefix (#15512) 2020-03-16 18:43:11 -05:00
Patrick Gartung
3029dde7ec
When replacing path text in ELF binaries do not include null byte padding in search string (#15508) 2020-03-16 14:49:55 -05:00
Seth R. Johnson
a8706cc89b
CMakePackage: convert variants to CMake arguments (#14376)
Add a 'define_from_variant` helper function to CMake-based Spack
packages to convert package variants into CMake arguments. For
example:

  args.append('-DFOO=%s' % ('ON' if '+foo' in self.spec else 'OFF'))

can be replaced with:

  args.append(self.define_from_variant('foo'))

The following conversions are handled automatically:

* Flag variants will be converted to CMake booleans
* Multivalued variants will be converted to semicolon-separated strings
* Other variant values are converted to CMake string arguments

This also adds a 'define' helper method to convert any variable to
a CMake argument. It has the same conversion rules as
'define_from_variant' (but operates directly on values rather than
requiring the user to supply the name of a package variant).
2020-03-16 11:41:19 -07:00
Patrick Gartung
17e4df1e41
Buildcache: Install into non-default directory layouts (#13797)
* Buildcache: Install into non-default directory layouts

Store a dictionary mapping of original dependency prefixes to dependency hashes

Use the loaded spec to grab the new dependency prefixes in the new directory layout.

Map the original dependency prefixes to the new dependency prefixes using the dependency hashes.

Use the dependency prefixes map to replace original rpaths with new rpaths preserving the order.
For mach-o binaries, use the dependency prefixes map to replace the dependency library entires for libraries and executables and the replace the library id for libraries.

On Linux, patchelf is used to replace the rpaths of elf binaries.
On macOS, install_name_tool is used to replace the rpaths and  dependency libraries  of mach-o binaries and the id of mach-o libraries.
On Linux, macholib is used to replace the dependency libraries of mach-o binaries and the id of mach-o libraries.

Binary text with padding replacement is attempted for all binaries for the following paths:
spack layout root
spack prefix
sbang script location
dependency prefixes
package prefix
 Text replacement is attempted for all text files using the paths above.

Symbolic links to the absolute path of the package install prefix are replaced, all others produce warnings.
2020-03-16 08:42:23 -05:00
Michael Kuhn
14441e00dc
package: Add fetch_options variable (#15317)
PR #15212 added a new connect_timeout option that can be overridden
using fetch_options but had to specified per-version. This adds a new
per-package variable that can be used to override fetch_options for
all versions in the package. This includes connect_timeout as well
as 'cookie' (e.g. for the jdk package).

Packages can combine package-level fetch_options with per-version
fetch_options, in which case the version fetch_options completely
override the package-level fetch_options.

This commit includes tests for the added behavior.
2020-03-13 11:41:19 -07:00
Tamara Dahlgren
f2a13af43e
testing: increase installer coverage (#15237) 2020-03-12 21:20:20 +01:00
Massimiliano Culpo
59b6822a47
Creating versions from urls doesn't modify class attributes (#15452)
fixes #15449

Before this PR a call to pkg.url_for_version was modifying
class attributes determining different results for subsequents
calls and an error when the urls was empty.
2020-03-12 09:21:19 +01:00
Tamara Dahlgren
20c77cade5
bugfix: resolve undefined source_pkg_dir failure (#15339) 2020-03-11 11:39:23 +01:00
Greg Becker
299fa6b5ca
fetch_strategy: remove vestigial code (#15431) 2020-03-11 11:24:38 +01:00
Greg Becker
bbbee6700b
bugfix: fix install_missing_compilers option bug from v0.14.0 (#15416)
* bugfix: ensure bootstrapped compilers built before packages using the compiler
2020-03-10 10:35:20 -07:00
Christoph Junghans
577860761b
decompressor: add support for bz2 (#15424) 2020-03-10 10:46:45 -06:00
Patrick Gartung
c7dda38826
Revert to gzip as the default buildcache compression as bzip2 is not always available. (#15412)
* Revert to gzip as the default buildcache compression as bzip2 is not always available.

* Flake8
2020-03-09 16:46:54 -05:00
Todd Gamblin
49ec260a9a
bugfix: installer.py shouldn't be executable (#15386)
This is a minor permission fix on the new installer.py introduced in #13100.
2020-03-07 23:48:18 -08:00
Massimiliano Culpo
b444fd25bb
ArchSpec: fix semantics of satisfies when not concrete and strict is true (#15319) 2020-03-07 13:58:33 +01:00
Peter Scheibel
697719c181
Only use stable versions for public mirror (#15100)
* add --skip-unstable-versions option to 'spack mirror create' which skips sources/resource for packages if their version is not stable (i.e. if they are the head of a git branch rather than a fixed commit)

* '--skip-unstable-versions' should skip all VCS sources/resources, not just those which are not cachable
2020-03-07 13:38:08 +01:00
Andrew W Elble
5b44a65881
Fix for being able to 'spack load' packages that have been renamed. (#14348)
* Fix for being able to 'spack load' packages that have been renamed.

* tests: add test for 'spack load' of a installed, but renamed/deleted package
2020-03-06 16:29:01 -08:00
Chris Green
da5144793e
Allow overrides for spack.config set() and override(). (#14432)
Allows spack.config InternalConfigScope and Configuration.set() to
handle keys with trailing ':' to indicate replacement vs merge
behavior with respect to lower priority scopes.

Lists may now be replaced rather than merged (this behavior was
previously only available for dictionaries).

This commit adds tests for the new behavior.
2020-03-05 19:25:01 -08:00
Tamara Dahlgren
b2e7e7edaa
Recover coverage from subprocesses during unit tests (#15354)
* Recover coverage from subprocesses during unit tests
2020-03-05 16:54:29 -08:00
Patrick Gartung
820d147e08
Allow buildcaches keys to find public keys on the e4s mirror (#15361) 2020-03-05 15:15:40 -06:00
Massimiliano Culpo
752092f46a
Correct pytest.raises matches to match (#15346) 2020-03-05 11:16:02 +01:00
Massimiliano Culpo
d9626cca50
Bugfix: resolve StopIteration message attribute failure (#15341)
Testing the install StopIteration exception resulted in an attribute error:

AttributeError: 'StopIteration' object has no attribute 'message'

This PR adds a unit test and resolves that error.
2020-03-05 10:26:50 +01:00
Patrick Gartung
e2b3d52999
Add function replace_prefix_nullterm for use on mach-o rpaths. (#15347)
This recovers the old behavior of replace_prefix_bin that was
modified to work with elf binaries by prefixing os.sep to new prefix
until length is the same as old prefix.
2020-03-04 19:21:15 -06:00
Peter Scheibel
cf7dbbf65c
Remove DB conversion of old index.yaml (#15298)
Removed the code that was converting the old index.yaml format into
index.json. Since the change happened in #2189 it should be
considered safe to drop this (untested) code.
2020-03-04 16:39:10 -08:00
Patrick Gartung
3ea0e91533
use os.sep instead of null byte to pad replacement paths in binaries (#15338)
* use os.sep instead of null byte to pad replacement paths in binaries

* Prefix the pading os.sep's
2020-03-04 17:15:09 -06:00
Sergey Kosukhin
b472c13803
Fix FCFLAGS handling for Autotools packages (#14788)
Spack's fflags are meant for both f77 and fc. Therefore, they must
be passed as FFLAGS and FCFLAGS to the configure scripts of
Autotools-based packages.
2020-03-03 13:13:38 -08:00
Tamara Dahlgren
73f7301ec4
bugfix: ensure proper dependency handling for package-only installs (#15197)
The distributed build PR (#13100) -- did not check the install status of dependencies when using the `--only package` option so would refuse to install a package with the claim that it had uninstalled dependencies whether that was the case or not.

- [x] add install status checks for the `--only package` case.
- [x] add initial set of tests
2020-03-02 11:16:20 -08:00
Adam J. Stewart
513f9235c3
Fix detection of redhat enterprise compute node (#15253)
* Fix detection of redhat enterprise compute node

* Add comma

* Remove space
2020-02-28 11:27:15 -06:00
Michael Kuhn
f580b340f8 Add new timeout fetch_option
This allows packages to override the global connect_timeout.
2020-02-27 10:52:52 -08:00
Michael Kuhn
b7cfd05ef7 Generalize curl_options into fetch_options
This allows us to support higher-level concepts such as 'cookie' and
'timeout' without users having to specify curl options.
2020-02-27 10:52:52 -08:00
Michael Kuhn
7325c20794 config: Add a new option connect_timeout
connect_timeout can be used to increase the time Spack waits for the
server to answer. This can be used to work around slow connections or
servers.

Fixes #14700
2020-02-27 10:52:52 -08:00
Adam J. Stewart
6153eea2c2
CudaPackage: add support for Tesla K80 and older CUDA (#15031)
* CudaPackage: add support for Tesla K80 and older CUDA

* Flake8 fixes

* Fix cuda_arch when no arch is set

* Fine-tune cuda_arch=37,50 supported CUDA versions

* CUDA 6.5+ supports SM_37

* Add @svenevs as a maintainer
2020-02-27 12:16:20 -06:00
TZ
d6d095a868
Docs: "spack env" is in the meantime called "spack build-env" (#15233) 2020-02-27 09:13:09 +01:00
Tamara Dahlgren
ed15adbb9e
bugfix: Add dependents when initializing spec from yaml (#15220)
The new build process, introduced in #13100 , relies on a spec's dependents in addition to their dependencies. Loading a spec from a yaml file was not initializing the dependents.

- [x] populate dependents when loading from yaml
2020-02-26 18:49:29 -08:00
Seth R. Johnson
6c445a2c0f
Uniquify suffixes added to module names (#14920) 2020-02-26 18:03:28 +01:00
Glenn Johnson
a568db00b8
A few edits for the Basic Usage Doc page (#15215)
This PR corrects a few minor things and adds a note about colorized
output.
2020-02-26 10:03:03 +01:00
Patrick Gartung
ae87828520
buildcache cmd: add explicit message with default output dir for buildcaches. (#15090)
* Make -d directory a required option. Print messages about where buildcaches will be written.

* Add mutually exclusive required options

* spack commands --update-completion

* Apply @opadron's patch

* Update share/spack/spack-completion.bash

* Incorporate @opadron's suggestions
2020-02-25 17:49:25 -06:00
Omar Padron
00090f8f97
add --only option to buildcache create cmd (#14921)
* add --only option to buildcache create cmd

replaces the --no-deps option
2020-02-25 17:32:20 -05:00
Omar Padron
8d750db9de
remove catch-all exception handling in buildcache command (#15185)
* remove catch-all exception handling in buildcache command

* fix test
2020-02-25 14:10:50 -06:00
Patrick Gartung
676eb56ab2
Buildcache cmd: add install -o/--otherarch option for installing macOS buildcaches on linux (#15192)
* Buildcache command: add install option -o/--otherarch
This will allow matching specs from other archs, for example
installing macOS buildcaches on linux hosts.

* spack commands --update-completion
2020-02-25 11:01:59 -06:00
Michael Kuhn
45306206c7
docs: Fix tiny typo (#15181) 2020-02-24 10:58:21 -06:00
Michael Kuhn
94acd8f235
cmd/load: Fix usage output (#15180)
args.specs is a list, which results in output like this:
```
eval `spack load --sh ['libxml2', 'xz']`
```

We want this instead:
```
eval `spack load --sh libxml2 xz`
```
2020-02-24 09:46:41 -06:00