Commit graph

12933 commits

Author SHA1 Message Date
Todd Gamblin
e0b94dba14
completion: add bash completion for spack spec --json (#13433) 2019-10-25 11:02:52 -07:00
Andreas Baumbach
ab55890ba8 Make error msg of test_api_for_build_and_run_environment more informative (#13435) 2019-10-25 19:24:41 +02:00
Todd Gamblin
ebeb1ed62f
bugfix: reindexing is not necessary for DB v0.9.3 to v5 upgrade (#13434)
reindexing takes a significant amount of time, and there's no reason to
do it from DB version 0.9.3 to version 5. The only difference is that v5
can contain "deprecated_for" fields.

- [x] Add a `_skip_reindex` list at the start of `database.py`

- [x] Skip the reindex for upgrades in this list. The new version will
  just be written to the file the first time we actually have to write
  the DB out (e.g., after an install), and reads will still work fine.
2019-10-25 07:10:03 -07:00
Todd Gamblin
dbee91f7f1
bugfix: allow fetching no-code packages (#13429)
Previously, spack would error out if we tried to fetch something with no
code, but that would prevent fetching dependencies. In particular, this
would fail:

    spack fetch --dependencies xsdk

- [x] Instead of raising an error, just print a message that there is nothing
      to be fetched for packages like xsdk that do not have code.

- [x] Make BundleFetchStrategy a bit more quiet about doing nothing.
2019-10-24 16:02:28 -07:00
Todd Gamblin
757387dc2a
commands: Add --json argument to spack spec (#13431)
We've had `spack spec --yaml` for a while, and we've had methods for JSON
for a while as well.  We just haven't has a `--json` argument for `spack spec`.

- [x] Add a `--json` argument to `spack spec`, just like `--yaml`
2019-10-24 16:01:31 -07:00
Chris Green
77af4684aa Improvements to detection of AMD architectures. (#13407)
New entry for K10 microarchitecture.

Reorder Zen* microarchitectures to avoid triggering as k10.

Remove some desktop-specific flags that were preventing Opteron Bulldozer/Piledriver/Steamroller/Excavator CPUs from being recognized as such.

Remove one or two flags which weren't produced in /proc/cpuinfo on older OS (RHEL6 and friends).
2019-10-24 15:48:54 -07:00
Greg Becker
a6ea0bbbae
Views: fix python in views when python prefix is under a symlink (#12575)
* Fix python in views when python prefix is under a symlink

* Add todo for future generalization
2019-10-24 16:27:21 -05:00
Adam J. Stewart
0f22e528f3 Add latest version of NetCDF libraries (#13416) 2019-10-24 14:21:12 -06:00
Mikael Simberg
83016e96fd Extend HPX package (#13219)
* Add master branch as version to HPX package

* Remove C++98 option from HPX package

* Add option to turn off examples in HPX package

* Add MPI variant to HPX package
2019-10-24 14:19:25 -06:00
Jannek Squar
5004b20f5c New package: py-cdsapi (#13363)
* New package py-cdsapi to access Climate Data Store for downloading climate and weather reanalysis data

* Improvements
2019-10-24 13:30:21 -05:00
Geoffrey Oxberry
2dd83523e2 mfem: add version 4.0 (without cuda) (#11880)
* mfem: add version 4.0.0 (without cuda, raja, etc)

* mfem@4.0.0: add occa variant

* mfem@4.0.0: add raja variant

* Update package.py

* Update package.py

* Update package.py

* Update package.py

* Update package.py
2019-10-24 13:08:51 -05:00
Chris Green
0913328812 Correctly identify Skylake CPUs on Darwin. (#13377)
* Correctly identify Skylake CPUs on Darwin.

* Add a test for haswell on Mojave.
2019-10-24 12:44:58 -05:00
Greg Becker
f65f2450df database: update DB version and change to integer database versions (#13410)
Update on version format: change to an integer database format version for simplicity, instead of tracking the Spack version.
2019-10-24 10:43:49 -07:00
Nick Forrington
845df79ac6 Patch libtool when using the Arm compiler (#12004)
* Patch libtool when using the arm, clang, and fujitsu compilers

If libtool does not have values for linker/pic flags, patch them in
2019-10-24 09:51:00 -05:00
Eric Martin
44485c56b6 blast-legacy: fix build issues (#13370) 2019-10-23 23:16:22 -05:00
Eric Martin
944ffe44a7 subread: add version 2.0.0 (#13368) 2019-10-23 23:07:11 -05:00
Cody Balos
177f661bb3 sundials: add version 5.0.0 and new variant (#13375)
* sundials: add version 5.0.0 and new variant

plus some cleanup

* sundiaks: fix formatting

* sundials: fix dependencies
2019-10-23 23:06:20 -05:00
Eric Martin
d0d0456fa7 vt: add version 0.5772 (#13367) 2019-10-23 23:05:27 -05:00
Greg Becker
8c7a3e55dd add spack dev-build command; deprecate spack diy (#13374)
Rename the `spack diy` command to `spack dev-build` to make the use case clearer.

The `spack diy` command has some useful functionality for developers using Spack to build their dependencies and configure/build/install the code they are developing.  Developers do not notice it, partly because of the obscure name.

The `spack dev-build` command has a `-u/--until PHASE` option to stop after a given phase of the build. This can be used to configure your project, run cmake on your project, or similarly stop after any stage of the build the user wants. These options are analogous to the existing `spack configure` and `spack build` commands, but for developer builds.

To unify the syntax, we have deprecated the `spack configure` and `spack build` commands, and added a `-u/--until PHASE` option to the `spack install` command as well.

The functionality in `spack dev-build` (specifically `spack dev-build -u cmake`) may be able to supersede the `spack setup` command, but this PR does not deprecate that command as that will require slightly more thought.
2019-10-23 17:08:15 -07:00
Peter Scheibel
f2ddffb840 bugfix: web.push_to_url should not format the local path that is passed to it (#13408)
fd58c98 formats the `Stage`'s `archive_path` in `Stage.archive` (as part of `web.push_to_url`). This is not needed and if the formatted differs from the original path (for example if the archive file name contains a URL query suffix), then the copy fails.

This removes the formatting that occurs in `web.push_to_url`.

We should figure out a way to handle bad cases like this *and* to have nicer filenames for downloaded files.  One option that would work in this particular case would be to also pass `-J` / `--remote-header-name` to `curl`.  We'll need to do follow-up work to determine if we can use `-J` everywhere.

See also: https://github.com/spack/spack/pull/11117#discussion_r338301058
2019-10-23 16:50:51 -07:00
Michael Kuhn
bd7265ed52 bison: Fix parallel build (#13402) 2019-10-23 16:49:18 -05:00
Peter Scheibel
685946b1f1 syaml.dump now allows representing all syaml_ objects (originally this was only available for syaml.dump_config) (#13403) 2019-10-23 14:24:14 -07:00
Massimiliano Culpo
b29eb4212e Users can configure use of RPATH or RUNPATH (#9168)
Add a new entry in `config.yaml`:

    config:
        shared_linking: 'rpath'

If this variable is set to `rpath` (the default) Spack will set RPATH in ELF binaries. If set to `runpath` it will set RUNPATH.

Details:
* Spack cc wrapper explicitly adds `--disable-new-dtags` when linking
* cc wrapper also strips `--enable-new-dtags` from the compile line
    when disabling (and vice versa)
* We specifically do *not* add any dtags flags on macOS, which uses
    Mach-O binaries, not ELF, so there's no RUNPATH)
2019-10-23 13:22:24 -07:00
Greg Becker
cd185c3d28 commands: add spack deprecate command (#12933)
`spack deprecate` allows for the removal of insecure packages with minimal impact to their dependents. It allows one package to be symlinked into the prefix of another to provide seamless transition for rpath'd and hard-coded applications using the old version.

Example usage:

    spack deprecate /hash-of-old-openssl /hash-of-new-openssl

The spack deprecate command is designed for use only in extroardinary circumstances.  The spack deprecate command makes no promises about binary compatibility. It is up to the user to ensure the replacement is suitable for the deprecated package.
2019-10-23 13:11:35 -07:00
Michael Kuhn
420346b275 mysql: Use correct python command (#13393)
* mysql: Use correct python command

python~pythoncmd does not provide a python symlink for python3, so make
sure we pick the right command.

* mysql: Adapt to build env changes
2019-10-23 14:15:30 -05:00
Sarah Osborn
b7536eb332 hypre: Add new variants to expose existing features. (#13373)
* hypre: Add new variants to expost existing features.

* hypre: Add new variants to expose existing features.

* hypre: Shorten description line.

* hypre: Add an explicit else clause to disable some features.
2019-10-23 14:13:21 -05:00
Todd Gamblin
26ec644fc8 url summary: show right and wrong parse counts for each regex
Previously this command only showed total counts for each regular
expression.  This doesn't give you a sense of which regexes are working
well and which ones are not.  We now display the number of right, wrong,
and total URL parses per regex.

It's easier to see where we might improve the URL parsing with this
change.
2019-10-23 09:10:32 -07:00
Adam J. Stewart
33499681a4
Add latest version of py-pbr (#13380) 2019-10-23 10:14:04 -05:00
Adam J. Stewart
df3b23334a
py-argparse: remove Python dependency version (#13378) 2019-10-23 10:13:39 -05:00
Adam J. Stewart
bc6c76baf9
py-traceback2: fix dependencies (#13381) 2019-10-23 10:13:11 -05:00
Michael Kuhn
b75f506a1a bison: Fix build (#13231)
It seems that 3.4.2 includes a change that is supposed to fix parallel
builds (https://lists.gnu.org/archive/html/bug-bison/2019-08/msg00000.html).
Instead, it actually breaks it for me (with -j48) with errors such as:
```
mv: cannot stat 'examples/c/reccalc/scan.stamp.tmp': No such file or directory
Makefile:9323: recipe for target 'examples/c/reccalc/scan.stamp' failed
```
2019-10-23 10:07:22 -05:00
Adam J. Stewart
4bd0a9b286
py-linecache2: pbr is only a build dependency (#13379) 2019-10-23 09:38:05 -05:00
Adam J. Stewart
baeec6a94e
py-unittest2: fix dependencies (#13382) 2019-10-23 09:37:03 -05:00
Adam J. Stewart
79c28331d2
Add py-codecov package (#13383) 2019-10-23 09:36:45 -05:00
t-karatsu
6f41987500 bpp-phyl: Clarify namespace of 'isnan' function. (#13386) 2019-10-23 09:36:01 -05:00
Michael Kuhn
64694dfaf8 magics: Use correct python path (#13394)
python~pythoncmd does not provide a python symlink for python3, so make
sure we pick the right command.
2019-10-23 09:33:42 -05:00
Michael Kuhn
466d07558d bracken: Use correct python command (#13395)
python~pythoncmd does not provide a python symlink for python3, so make
sure we pick the right command.
2019-10-23 09:33:09 -05:00
carlabguillen
44f859849b Ace package. Initial commit (#13211)
* Ace package. Initial commit

* Correcting header now with licence.

* Using sha256

* Making requested changes:removing import line, urls and checksums in one line, and making it an MakefilePackage.

* Removing extra line
2019-10-23 09:31:06 -05:00
Michael Kuhn
464ce2795a glib: Use correct python command (#13392)
python~pythoncmd does not provide a python symlink for python3, so make
sure we pick the right command.
2019-10-23 09:30:46 -05:00
Satish Balay
25d63aa44c petsc: add version 3.12.1 [and variant: batch] (#13390)
* petsc: add version 3.12.1 [and variant: batch]

* fix test_prs_update_old_api errors
2019-10-23 09:29:28 -05:00
Todd Gamblin
af65146ef6 Preserve comments for Spack YAML objects (#11602)
This updates the configuration loading/dumping logic (now called
load_config/dump_config) in spack_yaml to preserve comments (by using
ruamel.yaml's RoundTripLoader). This has two effects:

* environment spack.yaml files expect to retain comments, which
  load_config now supports. By using load_config, users can now use the
  ':' override syntax that was previously unavailable for environment
  configs (but was available for other config files).

* config files now retain user comments by default (although in cases
  where Spack updates/overwrites config, the comments can still be
  removed).

Details:

* Subclasses `RoundTripLoader`/`RoundTripDumper` to parse yaml into
  ruamel's `CommentedMap` and analogous data structures

* Applies filename info directly to ruamel objects in cases where the
  updated loader returns those

* Copies management of sections in `SingleFileScope` from #10651 to allow
  overrides to occur

* Updates the loader/dumper to handle the processing of overrides by
  specifically checking for the `:` character
  * Possibly the most controversial aspect, but without that, the parsed
    objects have to be reconstructed (i.e. as was done in
    `mark_overrides`). It is possible that `mark_overrides` could remain
    and a deep copy will not cause problems, but IMO that's generally
    worth avoiding.
  * This is also possibly controversial because Spack YAML strings can
    include `:`. My reckoning is that this only occurs for version
    specifications, so it is safe to check for `endswith(':') and not
    ('@' in string)`
  * As a consequence, this PR ends up reserving spack yaml functions
    load_config/dump_config exclusively for the purpose of storing spack
    config
2019-10-23 06:29:49 -07:00
Todd Gamblin
b3f5084b96
tests: avoid extra output in env status test (#13344)
`test_envoronment_status()` was printing extra output during tests.

- [x] disable output only for `env('status')` calls instead of disabling
  it for the whole test.
2019-10-22 23:59:28 -07:00
Greg Becker
95a48b27ec env activation: use package defined env setup methods (#13249)
This PR ensures that environment activation sets all environment variables set by the equivalent `module load` operations, except that the spec prefixes are "rebased" to the view associated with the environment.

Currently, Spack blindly adds paths relative to the environment view root to the user environment on activation. Issue #12731 points out ways in which this behavior is insufficient.

This PR changes that behavior to use the `setup_run_environment` logic for each package to augment the prefix inspections (as in Spack's modulefile generation logic) to ensure that all necessary variables are set to make use of the packages in the environment.

See #12731 for details on the previous problems in behavior.

This PR also updates the `ViewDescriptor` object in `spack.environment` to have a `__contains__` method. This allows for checks like `if spec in self.default_view`. The `__contains__` operator for `ViewDescriptor` objects checks whether the spec satisfies the filters of the View descriptor, not whether the spec is already linked into the underlying `FilesystemView` object.
2019-10-22 23:27:40 -07:00
Massimiliano Culpo
b14f18acda microarchitectures: look in /sbin and /usr/sbin for sysctl (#13365)
This PR ensures that on Darwin we always append /sbin and /usr/sbin to PATH, if they are not already present, when looking for sysctl.

* Make sure we look into /sbin and /usr/sbin for sysctl
* Refactor sysctl for better readability
* Remove marker to make test pass
2019-10-22 21:42:38 -07:00
Massimiliano Culpo
8808207ddf Fixed optimization flags support for old GCC versions (#13362)
These changes update our gcc microarchitecture descriptions based on manuals found here https://gcc.gnu.org/onlinedocs/ and assuming that new architectures are not added during patch releases.
2019-10-22 21:40:45 -07:00
Todd Gamblin
a942d2dd3d
Disable macOS builds on PRs while we wait for Travis to catch up. (#13389)
Travis macOS builds are taking too long to run, so we'll only run them on
`develop` until they get faster.
2019-10-22 20:40:47 -07:00
Benjamin Fovet
7338730528 gmsh: add version 4.4.1 (#13366) 2019-10-22 18:18:19 -05:00
Frédéric Simonis
ab5135db1c Boost package: Fix iostream autodetect libraries (#12440)
Boost iostream autodetects the compression libraries libzstd and
liblzma outside of the Spack environment.

This commit disables mentioned libraries. In the future if the
Spack zstd/lzma packages were added as dependencies of the Spack
Boost package, additional work could be done to build Boost with
the Spack-built versions of these libraries.
2019-10-22 14:25:03 -07:00
Axel Huebl
0f931cfc87 ADIOS2: Document endian_reverse variant (#13226)
Update the documentation of the `endian_reverse` variant in ADIOS2.
The short description caused some confusion (for me) :-)
2019-10-22 11:06:06 -05:00
Ruben Di Battista
49291d5a40 git-lfs: Add new version (#13356)
PS: The old versions do not work: it's likely that they changed git
history on submodules
2019-10-22 10:45:45 -05:00