Commit graph

3554 commits

Author SHA1 Message Date
Massimiliano Culpo
bca59f8d83 Speedup environment activation (#13557)
* Add a transaction around repeated calls to `spec.prefix` in the activation process
* cache the computation of home in the python package to speed up setting deps
* ensure that module-scope variables are only set *once* per module
2019-12-02 14:05:02 -08:00
Adam J. Stewart
71eaffa46b Version bump: 1.13.1 (#13942) 2019-12-02 06:58:30 -08:00
Chris Green
c5da94eb58 Armor test script against shebang character limit if appropriate. (#12447) 2019-11-30 13:00:08 -06:00
Adam J. Stewart
ab98ef1765
Mass conversion of setup_(dependent_)?environment (#13924)
* Mass conversion of setup_(dependent_)?environment

* prefix -> self.prefix
2019-11-29 15:00:44 -06:00
Greg Becker
1450c30656 use semicolons instead of newlines in module/python command (#13904) 2019-11-26 17:09:24 -08:00
Adam J. Stewart
ad1ad83664
IntelPackage: setup_env -> setup_build_env (#13888) 2019-11-26 16:48:32 -06:00
Massimiliano Culpo
f6781d65f0 buildcache list: do not display duplicate specs (#13758)
fixes #13757
2019-11-26 13:24:15 -07:00
Massimiliano Culpo
77c206185b Remove warning from the "Custom Extensions" section of the docs (#13804)
Extensions have been available for a while and the overall design
seems solid enough to be feasible for extensions without losing
backward compatibility.
2019-11-26 09:30:49 -08:00
René Widera
9c10b80591 add package cuda@10.2.89 (#13819)
* add package cuda@10.2.89

add CUDA 10.2 support

* CudaPackage: Add 10.2 Conflicts

* CUDA: Modernize Run Environment

See #11115
2019-11-25 13:23:48 -06:00
Greg Becker
6c55a7c85f
cmd/install: remove unused install_status option (#13751)
* cmd/install: remove unused install_status option

* update bash completions for spack install
2019-11-22 11:17:37 -07:00
Massimiliano Culpo
0684a58d16 Fixed detection for cascadelake microarchitecture (#13820)
fixes #13803
2019-11-21 13:09:48 -07:00
Paul Ferrell
fb3a3ba95e mirror bug fixes: symlinks, duplicate patch names, and exception handling (#13789)
* Some packages (e.g. mpfr at the time of this patch) can have patches
  with the same name but different contents (which apply to different
  versions of the package). This appends part of the patch hash to the
  cache file name to avoid conflicts.
* Some exceptions which occur during fetching are not a subclass of
  SpackError and therefore do not have a 'message' attribute. This
  updates the logic for mirroring a single spec (add_single_spec)
  to produce an appropriate error message in that case (where before
  it failed with an AttributeError)
* In various circumstances, a mirror can contain the universal storage
  path but not a cosmetic symlink; in this case it would not generate
  a symlink. Now "spack mirror create" will create a symlink for any
  package that doesn't have one.
2019-11-20 17:00:44 -07:00
t-karatsu
513fe55fc3 Features/expand microarch for aarch64 (#13780)
* Add process to determine aarch64 microarchitecture

* add microarchitectures for thunderx2 and a64fx

* Add optimize flags for gcc on aarch64 family processors thunderx2 and a64fx.

* Add optimize flags for clang on aarch64 family processors thunderx2 and a64fx

* Add testing for thunderx2 and a64fx microarchitectures
2019-11-20 00:01:12 -07:00
Greg Becker
99dfff4475
binary distribution: relocate text files properly in relative binaries (#13578)
* Make relative binaries relocate text files properly
* rb strings aren't valid in python 2
* move perl to new interface for setup_environment family methods
2019-11-18 14:13:42 -07:00
Tim Fuller
3dbafb5ee7 Fix for issue #13754 (#13755)
* remove reference to `spack.store` in method definition

Referencing `spack.store` in method definition will cache the `spack.config.config` singleton variable too early, before we have a chance to add command line and environment scopes.
2019-11-15 13:32:58 -08:00
Adam J. Stewart
5510bba1fd
Document use of the maintainers field (#12270)
* Document use of the maintainers field

* Use fake GitHub usernames

* GitHub action is not automatic yet
2019-11-15 13:58:24 -06:00
Greg Becker
c587c76537 Config option to allow gpg warning suppression (#13743)
Add a configuration option to suppress gpg warnings during binary
package verification. This only suppresses warnings: a gpg failure
will still fail the install. This allows users who have already
explicitly trusted the gpg key they are using to avoid seeing
repeated warnings that it is self-signed.
2019-11-14 16:22:01 -08:00
Peter Scheibel
3f861e18b0 symlink relativization: determine target relative to the link directory (#13710)
when making a package relative, relocate links relative to link directory

rather than the full link path (which includes the file name) because `os.path.relpath` expects a directory.
2019-11-13 17:51:57 -08:00
Greg Becker
43b0356e14 Allow binary relocation of strings in relative binaries (#13725)
Binaries with relative RPATHS currently do not relocate strings
hard-coded in binaries

This PR extends the best-effort relocation of strings hard-coded
in binaries to those whose RPATHs have been relativized.
2019-11-13 16:03:35 -08:00
William F Godoy
d6274e1a1d Docs update for deprecated spack sha256 (#13701)
* Docs update for deprecated `spack sha256`

* Added macOS shasum

* Update lib/spack/docs/packaging_guide.rst

Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
2019-11-13 15:21:27 -06:00
Todd Gamblin
da386b560a docs: use nicer looking code blocks in docs (#13708)
- [x] Use higher contrast terminal output font
- [x] Use higher contrast code block background color than default
- [x] Use a noticeable prompt character

See also https://github.com/spack/spack-tutorial/pull/10.
2019-11-12 18:18:41 -08:00
Daryl W. Grunau
5a9389a528 verify.py: os.path.exists exception handling (#13656) 2019-11-11 17:47:47 -06:00
Adam J. Stewart
bd9907809e environments: don't try to modify run-env if a spec is not installed (#13589)
Fixes #13529
Fixes #13509
2019-11-11 13:02:36 +01:00
Adam J. Stewart
60070759a0
Improve spack checksum formatting (#13666)
* Improve spack checksum formatting

* Add colon

* Fix Python 2 bug
2019-11-10 16:47:56 -06:00
Todd Gamblin
c9c267518f
bugfix: mirror path works for unknown versions (#13626)
`mirror_archive_path` was failing to account for the case where the fetched version isn't known to Spack.

- [x] don't require the fetched version to be in `Package.versions`
- [x] add regression test for mirror paths when package does not have a version
2019-11-07 17:13:51 -07:00
Andreas Baumbach
7a0bed7893 add depends_on(python) to PythonPackage stub (#13595) 2019-11-05 13:18:17 -06:00
Todd Gamblin
a6ccb5b2fa
bugfix: uninstall should find concrete specs by DAG hash (#13598)
This fixes a regression introduced in #10792.  `spack uninstall` in an
environment would not match concrete query specs properly after the index
hash of enviroments changed.

- [x] Search by DAG hash for specs to remove instead of by build hash
2019-11-05 06:12:12 -07:00
Todd Gamblin
b77974e986
spack find now displays variants and other spec constraints (#13596)
If you do this in a spack environment:

    spack add hdf5+hl

hdf5+hl will be the root added to the `spack.yaml` file, and you should
really expect `hdf5+hl` to display as a root in the environment.

- [x] Add decoration to roots so that you can see the details about what
      is required to build.
- [x] Add a test.
2019-11-05 05:39:56 -07:00
Omar Padron
0784ec1a6d bugfix: fetch prefers to fetch local mirrors over remote resources (#13545)
- [x] insert at beginning of list so fetch grabs local mirrors before remote resources
- [x] update the S3FetchStrategy so that it throws a SpackError if the fetch fails.  
      Before, it was throwing URLError, which was not being caught in stage.py.
- [x] move error handling out of S3FetchStrategy and into web_util.read_from_url()
- [x] pass string instead of URLError to SpackWebError
2019-11-03 17:11:30 -08:00
Greg Becker
aa7c229c8f environments: only write when necessary (#13546)
This changes Spack environments so that the YAML file associated with the environment is *only* written when necessary (i.e., if it is changed *by spack*).  The lockfile is still written out as before.

There is a larger question here of which part of Spack should be responsible for setting defaults in config files, and how we can get rid of empty lists and data structures currently cluttering files like `compilers.yaml`.  But that probably requires a rework of the default-setting validator in `spack.config`, as well as the code that uses `spack.config`.  This will at least help for `spack.yaml`.
2019-11-03 15:46:41 -08:00
Adam J. Stewart
a8228e1aec
spack create: http -> https (#13547) 2019-11-01 17:40:49 -05:00
Peter Scheibel
2e029fc2e5
Bugfix: respect order of mirrors in mirrors.yaml (#13544)
Commands like "spack mirror list" were displaying mirrors in a
different order than what was listed in the corresponding mirrors.yaml
file.

This restores commands to iterate over mirrors in the order that
they appear in the config file.
2019-11-01 14:02:08 -07:00
Greg Becker
24ec9f0ce3
environments: make shell modifications partially unconditional (#13523)
* environments: make shell modifications partially unconditional

* flake

* missing module name

* add regression test

* flake
2019-11-01 12:00:16 -05:00
Marc Mengel
2cea0633fa allow bootstrap buildcache install of patchelf (#13430)
* allow bootstrap buildcache install of patchelf

* file not path_name on one

* style

* add test for relocating patchelf

* blank lines..
2019-11-01 10:54:55 -05:00
Omar Padron
0f816561db bugfix: spack.util.url.join() now handles absolute paths correctly (#13488)
* fix issue where spack.util.url.join() failed to correctly handle absolute path components
* add url util tests
2019-11-01 03:42:43 -07:00
Gregory Lee
3bdab6f686 sbang: use utf-8 for encoding when patching (#13490)
This fixes a UnicodeDecodeError in the sbang patching function.
2019-11-01 03:39:45 -07:00
Massimiliano Culpo
390ffb80e7 Specs with quoted flags containing spaces are parsed correctly (#13521) 2019-11-01 03:38:11 -07:00
Massimiliano Culpo
42b8355269 targets: print a warning message before downgrading (#13513)
* Make package preferences a soft failure for targets, instead of a hard failure.
* Added unit tests for preferences expressed via packages.yaml
2019-11-01 03:36:28 -07:00
Peter Scheibel
6ec39b6c81
Bugfix: BundlePackage staging/installation (#13524)
4af4487 added a mirror_id function to most FetchStrategy
implementations that is used to calculate resource locations in
mirrors. It left out BundleFetchStrategy which broke all packages
making use of BundlePackage (e.g. xsdk). This adds a noop
implementation of mirror_id to BundleFetchStrategy so that the
download/installation of BundlePackages can proceed as normal.
2019-10-31 12:56:38 -07:00
Adam J. Stewart
4b56933643
Docs: unlock requirement versions (#13384) 2019-10-31 14:21:05 -05:00
Adam J. Stewart
536486f0e5
Travis CI: Test Python 3.8 (#13347)
* Travis CI: Test Python 3.8

* Fix use of deprecated cgi.escape method

* Fix version comparison

* Fix flake8 F811 change in Python 3.8

* Make flake8 happy

* Use Python 3.8 for all test categories
2019-10-31 14:20:46 -05:00
Greg Becker
a97faeb3c7 Documentation: Database.query methods share docstrings (#13515)
Currently, query arguments in the Spack core are documented on the
Database._query method, where the functionality is defined.

For users of the spack python command, this makes the python builtin
method help less than ideally useful, as help(spack.store.db.query)
and help(spack.store.db.query_local) do not show relevant information.

This PR updates the doc attributes for the Database.query and
Database.query_local arguments to mirror everything after the first
line of the Database._query docstring.
2019-10-30 16:47:48 -07:00
Greg Becker
230c6aa326
cpu: fix clang flags for generic x86_64 (#13491)
* cpu: differentiate flags used for pristine LLVM vs. Apple's version
2019-10-30 17:16:13 -05:00
Massimiliano Culpo
cf1c38162e cuda: fix conflict statements for x86-64 targets (#13472)
* cuda: fix conflict statements for x86-64 targets

fixes #13462

This build system mixin was not updated after the support for specific
targets has been merged.

* Updated the version range of cuda that conflicts with gcc@8:

* Updated the version range of cuda that conflicts with gcc@8: for ppc64le

* Relaxed conflicts for version > 10.1

* Updated versions in conflicts

Co-Authored-By: Axel Huebl <axel.huebl@plasma.ninja>
2019-10-30 09:25:46 -07:00
Peter Scheibel
c99d367a11 syaml_int type should use int.__repr__ rather than str.__repr__ (#13487) 2019-10-29 13:51:48 -07:00
Tamara Dahlgren
3cf21e6edc bugfix: make spack -d test test_changed_files work (#13229)
The `test_changed_files` in `test/cmd/flake8.py` was failing because it calls
`ArgumentParser.parse_args()` without arguments.  Normally that would just
parse `sys.argv` but it seems to fail because of something in either `spack test`
or `pytest`.  Call it with an empty array so that it doesn't try to touch`sys.argv`
at all.

- [x] allow `-d` spack option for `test_changed_files`
2019-10-29 00:25:59 -07:00
Adam J. Stewart
2264e30d99
Update URL parsing regexes and tests (#13411)
* Update URL parsing regexes and tests

* Get rid of no longer used README

* Merge py-udunits and py-cf-units

* netcdf -> netcdf-c

* setup_environment -> setup_*_environment

* Fix doc tests

* Few last minute fixes

* Simplify prefix removal copypasta
2019-10-28 20:27:54 -05:00
Adam J. Stewart
b4e027e918
Output directory Spack is looking in for MKL libs (#13465) 2019-10-28 11:46:59 -05:00
Todd Gamblin
8eeb64096c
version bump: 0.13.0 2019-10-25 21:53:52 -07:00
Todd Gamblin
847703c13d
tutorial: move tutorial to standalone site (#13450)
* docs: add a spack environment for building the docs

* docs: remove tutorial and link to spack-tutorial.readthedocs.io

The tutorial now has its own standalone website, versioned by instances
of the tutorial.  Link to that instead of versioning it directly with Spack.
2019-10-25 21:49:27 -07:00