Commit graph

2148 commits

Author SHA1 Message Date
Todd Gamblin
908ba6e3d6 Waste less space when fetching cached archives, simplify fetch messages. (#2264)
* Waste less space when fetching cached archives, simplify fetch messages.

- Just symlink cached files into the stage instead of copying them with curl.
- Don't copy linked files back into the cache when done fetching.

* Fixes for review.

* more updates

* last update
2016-11-07 01:34:30 -08:00
Todd Gamblin
08477f6624 Don't automatically spider remote list_urls before making stage. (#2263)
- This might be useful but we don't want to do it unconditionally.
2016-11-06 20:50:14 -08:00
becker33
49b1c0df2b Changed cflag matching to exact rather than subset. (#2227)
Not the desired eventual behavior, but cflag subset matching is not currently working for anonymous specs and this provides a temporary solution by restricting the feature until it is fixed.
2016-11-06 15:32:05 -08:00
George Hartzell
7f5d1953d4 Wordsmithing/minor-edits to module tutorial (#2256)
* Wordsmithing/minor-edits to module tutorial

A small set of wordsmithing, spell checking and minor edits to the fancy
new modules tutorial!

* Fix typo (sneaky z key...)

* Fix "S:" and "manual<" typos
2016-11-06 14:56:41 -08:00
Todd Gamblin
544b147370 Add detection for redhat enterprise server to distro.py (#2262) 2016-11-06 14:23:21 -08:00
Massimiliano Culpo
1bef2b7e98 variable referenced before assignment (#2252) 2016-11-05 10:15:29 -07:00
Jeffrey Salmond
a260e7e860 Bugfix in concretize_compiler (#2196)
* fix bug in search for concrete compiler

* simplify implementation of concretize_compiler

* fix flake8 errors
2016-11-04 23:20:49 -07:00
scheibelp
0940ee6015 Fixes #1520 (#1822)
Some packages which include resources fetched from source control
repositories terminated package installs because they failed to
archive; specifically, this included all SCM resources which identify
a specific state of the repo - for example a revision in svn or a
tag/revision in git. This is because the resource stage creation
logic did not choose an appropriate archive name for these kinds of
resources.
2016-11-04 20:14:56 -07:00
Adam J. Stewart
4e6d535058 Set Clang as the default compiler on macOS (#2225)
* Set OS-specific default compilers

* Fix flake8 warnings
2016-11-04 17:32:25 -07:00
Todd Gamblin
78154a63e8 Fix bug in distribution detection on unsupported platforms. 2016-11-04 16:49:19 -07:00
Jeffrey Salmond
9cd83a4efb Use code from the distro package to improve linux distro detection. (#1629)
* Use code from the `distro` package to improve linux distro detection.

Various issues [1] [2] with the platform.linux_distribution function led
to the function being deprecated in python 3.4. The pipi package distro
[3] contains compatible improved code that better tracks current methods
for identifying linux distributions (see, for example, [4],[5]).

[1]: https://bugs.python.org/issue20454
[2]: http://bugs.python.org/issue1322
[3]: https://pypi.python.org/pypi/distro
[4]: https://refspecs.linuxbase.org/LSB_3.0.0/LSB-PDA/LSB-PDA/lsbrelease.html
[5]: https://www.freedesktop.org/software/systemd/man/os-release.html

* fix (potential) unicode problems
2016-11-04 16:38:44 -07:00
Axel Huebl
c3d9dda0e5 libSplash: Add 1.6.0 Release (#2244)
* libSplash: Add 1.6.0 Release

Add the latest release of libSplash, version 1.6.0.

* Fix flake8 checks (another loop inversion issue)
2016-11-04 16:27:17 -07:00
Massimiliano Culpo
401b4cb137 module files tutorial : first complete draft (#2228)
* module files tutorial : first complete draft

- first complete draft for module files tutorial
- minor corrections to module file reference

* module file tutorial : first batch of corrections

- module avail spelled out fully
- typos from @adamjstewart
- rewording of a few sentences

* module file tutorial : first batch of corrections

- emphasized lines in yaml files

* module file tutorial : fixes according to @citibeth and @adamjstewart reviews

- used long format for command options
- reworded unclear sentence on tokes
- reworked table in reference manual to make it clearer

* module file tutorial : implemented corrections collected on site from @schlyfts

* module file tutorial : removed comment (@hartzell suggestion)
2016-11-04 12:55:43 -07:00
Gregory Lee
ee6eb508cb patch older config.guess for newer architectures (#2221) 2016-11-04 12:12:37 -07:00
Todd Gamblin
2536029ea9 Better spack spec (#2238)
* Add options for hashes, tree depth, and YAML to `spack spec`.

- Can now display hashes with `spack spec`, like `spack find`.
  - Removed the old "ids" argument to `spack spec` (which
    printed numerical values)b

- Can output YAML spec from `spack spec` with `-y`

- Can control depth of DAG traversal with --cover=[nodes|edges|paths]

- Can print install status (installed, missing, not installed) with -I

* Don't use YAML aliases in specs.

- Force Spack's YAML dumper to ignore aliases.
- aliases cause non-canonical YAML to be used in DAG hash, and result in
  redundant hashes.
- add a test to ensure this behavior stays
2016-11-04 11:47:57 -07:00
Peter J. Scheibel
9a585e6c6a Make spec.format properties case-insensitive 2016-11-04 11:41:20 -07:00
Peter J. Scheibel
74b439b7f1 Support 'COMPILERVERSION' in spec format string 2016-11-04 11:41:20 -07:00
Denis Davydov
1eeef97c78 remove obsolete instructions on hacking clang+gfortran (#2217)
* remove obsolete instructions on hacking clang+gfortran

* add detailed instructions on how to setup Clang+gfortran on macOS
2016-11-03 08:06:33 -07:00
Massimiliano Culpo
36a4ca8b11 spack install: forward sys.stdin to child processes (#2158)
* spack install: forward sys.stdin to child processes fixes #2140

 - [ ] redirection process is spawned in __enter__ instead of __init__
 - [ ] sys.stdin is forwarded to child processes

* log: wrapped __init__ definition
2016-11-03 08:03:10 -07:00
Massimiliano Culpo
5b5894afba spack find: accepts anonymous specs as arguments fixes #2170 (#2188) 2016-11-03 00:45:15 -07:00
Todd Gamblin
b304387308 Fix style checker bug. (#2214)
* Fix style checker bug.

* spack flake8: print cwd-relative paths by default, with root-relative option.
2016-11-02 12:17:35 -05:00
Barry Smith
9455621ec2 Add support for gfortran to be used with clang (#2192)
1) list gfortran as a fc and f77 compiler that can work with clang
2) allow compatible gfortran to ./spack compiler find with clang by matching version numbers

This is based on the discussions in

https://github.com/LLNL/spack/issues/237
https://github.com/dealii/dealii/wiki/deal.II-in-Spack#mixing-gcc-and-clang-on-osx

This is not a long term solution but something to get us through the next months until the compiler
infrastructure is reworked to allow mixing and matching for C/C++ and Fortran compilers

Funded-by: IDEAS
Project: IDEAS/xSDK
Time: 1.5 hours
2016-11-02 11:17:13 -05:00
scheibelp
45a4388164 Update references in debug command (#2204) 2016-11-01 13:55:13 -07:00
Todd Gamblin
edfe2297fd Improved package.py error handling. (#2187)
- Detailed debug information is now handed back to the parent process
  from builds, for *any* type of exception.

- previously this only worked for Spack ProcessErrors, but now it works
  for any type of error raised in a child.

- Spack will print an error message and source code context for build
  errors by default.

- It will print a stack trace when using `spack -d`, even when the error
  occurred in the child process.
2016-10-31 15:32:19 -07:00
Todd Gamblin
1b7f9e24f4 Add spack flake8 command. (#2186)
- Ported old run-flake8-tests qa script to `spack flake8` command.
- New command does not modify files in the source tree
- Copies files to a temp stage modifies them there, and runs tests.
- Updated docs and `run-flake8-tests` script to call `spack flake8`.
2016-10-31 11:40:20 -07:00
Todd Gamblin
4be703cde0 Allow common args to be written the same way regular args are. 2016-10-30 23:55:00 -07:00
Todd Gamblin
8f21332fec Bugfix: '::' only worked on top-level key in config.
- generalized and fixed to work with any key in YAML file
- simplified schema writing, as well
- add more unit tests for the config system
- Rename test/yaml.py to test/spack_yaml.py
  - Add test/yaml.pyc to ignored pyc files.
2016-10-30 23:55:00 -07:00
Todd Gamblin
d155156e32 Documentation for configuration 2016-10-30 23:55:00 -07:00
Todd Gamblin
b962622975 Add spack arch --platform option 2016-10-30 23:55:00 -07:00
Todd Gamblin
58e76721b9 Make provider_index use spack yaml for cleaner output. 2016-10-30 23:55:00 -07:00
Todd Gamblin
da760a66de source_cache, misc_cache, verify_ssl, checksum, & dirty in config.yaml
- Added new preferences to config.yaml:
  - source_cache
  - misc_cache
  - verify_ssl
  - checksum
  - dirty
2016-10-30 23:55:00 -07:00
Todd Gamblin
22b14e0f23 Add module_roots to config.yaml 2016-10-30 23:55:00 -07:00
Todd Gamblin
0da639298c Move temp directory configuration to config.yaml
- Moved temp finding logic to spack.stage
- Updated stage tests
- Added tests for new path substaitution of $user, $spack, $tempdir
2016-10-30 23:55:00 -07:00
Todd Gamblin
9347f86939 Rename install.yaml -> config.yaml, install_area -> "store"
- Added a schema for config.yaml
- Moved install tree configuration to config.yaml
- Moved etc/spack/install.yaml to etc/spack/defaults/config.yaml
- renamed install_area to "store", to use a term in common with guix/nix.
  - in `config.yaml` file, it's called the `install_tree` to be more
    intuitive to users.
  - `install_tree` might've worked in the code, but `install_tree` is
    already a global function in the spack namespace, from
    llnl.util.filesystem.
2016-10-30 23:55:00 -07:00
Benedikt Hegner
24901f7a38 add documentation for new option to set install area 2016-10-30 23:55:00 -07:00
Benedikt Hegner
c557e76565 refactor settings for install area and make them config options 2016-10-30 23:55:00 -07:00
Matt Belhorn
7ccf56a0b7 Removes cyclic dependency on spack.config. (#2121)
Merge #2030 added a cyclic dependency between the Cray platform needing
to read a `targets.yaml` config file and `config.py` needing to get the
platform names.

This commit removes the cyclic dependency in favor of the more general
config scheme. It also removes the now functionless `targets.yaml`
config file. This breaks 'frontend' targets on the Cray platform but
all architecture targets, including the frontend, that are provided by
CrayPE are added to the Platform anyway so users can be explicit about
the architecture targeted by the Cray compiler wrappers:

```
spack spec libelf arch=cray-CNL-frontend
```

becomes

```
spack spec libelf arch=cray-CNL-mc8         # on an XK7 or
spack spec libelf arch=cray-CNL-sandybridge # on an older XC30, etc..
```

The only way the 'frontend' target can be defined after this commit is
through target environment variables.
2016-10-30 21:02:27 -07:00
Massimiliano Culpo
9989f8e267 module file support: major rework of docs (#2169)
* module file support: major rework of docs

* module file support: fixed issues found by @adamjstewart

- list or enumeration should not be indented
- use console instead of bash or csh in things that are not scripts
- other typos

* module file support: fixed other issues found by @adamjstewart

- tables should not be indented
- substitute lines with pyobject to import an entire function
- get help output running commands
- typos

* module file support: fixes according to review comments

- @citibeth moved `spack module loads` after `spack load`
- @glennpj tried to clarify installation table + changes to language
- @tgamblin Removed top level section and moved the whole thing into the reference manual

* module file support: moved directive before spack module loads
2016-10-30 12:41:41 -07:00
Massimiliano Culpo
4d35ac6a16 configuration file for modules : fixed enable keyword (#2176)
- enable keyword works again
- test/modules.py : proper clean-up after tests
2016-10-30 12:30:51 -07:00
Barry Smith
ef2c42a889 Do not check directory layout for external packages (#2172)
External packages do not have an spec.yaml file so don't check for it.
Without this change any time a package depends on an external package
when the new package is installed you will get the error

Install prefix exists but contains no spec.yaml

This problem has also haunted me since I started using Spack since PETSc
depends on Python and I used an external python but fortunately it
was relatively easy to debug once I could reproduce it at will.

Funded-by: IDEAS
Project: IDEAS/xSDK
Time: 1 hour
2016-10-29 20:57:41 -07:00
Matthew Scott Krafczyk
c2d58d39dd Import dso_suffix from spack.build_environment in abi.py (#2174)
Fixes #1845
2016-10-29 16:47:06 -07:00
scheibelp
23683c65de Use Spec.format for token substitution in modules (#1848)
This replaces a custom token-based substitution format with calls to
Spec.format in modules.py

This also resolves a couple issues:

- LmodModules set configuration globally instead of in its initializer
  which meant test-specific configuration was not applied
- Added support for setting hash_length=0 for LmodModules. This only
  affects the module filename and not the directory names for the
  hierarchy tokens in the path. This includes an additional unit test.
2016-10-29 13:56:34 -07:00
Adam J. Stewart
58409a2eaf Fix rpath linking for NAG, add documentation on setup (#2142) 2016-10-27 15:28:09 -07:00
Elizabeth Fischer
12270b9dbf Remove support for deprecated package. See also #2085 and #2086 (#2090) 2016-10-26 21:53:08 -07:00
George Hartzell
bf08bd7076 Fix typo: _choiches -> _choices (#2127)
The breaks `spack create`.
2016-10-26 20:39:08 -07:00
George Hartzell
e652a32f05 typo: vlaues -> values (#2147) 2016-10-26 17:51:01 -07:00
Elizabeth Fischer
3895c974a0 Removed documentation on false paths as per #2083 (#2146)
Removed documentation on false paths as per #2083
2016-10-26 17:11:54 -07:00
Denis Davydov
9f54cea5c5 document recommended usage of system blas/lapack (#2081)
* document recommended usage of system blas/lapack

* update doc

* minor

* reword
2016-10-26 16:22:18 -07:00
Massimiliano Culpo
37dc719d13 ProcessError : now the exception is correctly pickled and passed across processes. (#2143) 2016-10-26 15:17:20 -07:00
Massimiliano Culpo
e73ab84680 spack install : added --log-format option (incorporates test-install command) (#2112)
* spack install : added --log-format option (incorporates test-install command)

fixes #1907

* qa : removed extra whitespace
2016-10-26 14:22:46 -07:00
Denis Davydov
33fb7bbd1b add a unit test for conretization with develop version (#2064) 2016-10-26 01:49:52 -07:00
Denis Davydov
39772e4c4c filter system paths (#2083)
* add filter_system_paths()

* filter system paths in set_build_environment_variables()

* two function: lib/inc + bin

* reverse order in bin

* fix order

* minor

* improvements of the code

* more cleanup

* alternative solution for filter_bins

* fiddle base alalazo's approach

* minor

* minor
2016-10-25 11:25:13 -07:00
Adam J. Stewart
a250792202 Find compatibility versions of GCC (#2099) 2016-10-25 11:07:49 -07:00
Massimiliano Culpo
7a575d2f4b multimethod.py : calls functools.wraps before returning the correct method fixes #2118 (#2119) 2016-10-25 08:30:40 -07:00
Massimiliano Culpo
b27e78cd78 spack setup : fixed regression after #1186 (#2117) 2016-10-25 07:53:11 -07:00
Todd Gamblin
0f6a5cd385 Merge pull request #1186 from epfl-scitas/features/install_with_phases
do_install : allow for an arbitrary number of phases
2016-10-24 17:13:49 -07:00
Todd Gamblin
c1ad4bde28 Rename EditableMakefile to MakefilePackage 2016-10-24 16:41:20 -07:00
Joseph Ciurej
7dd14870ce Bug Fix : Apply Compiler Flags Specified by Manual Compiler Configuration (#1532)
* Fixed a bug causing config-specified compiler flags to be ignored.
Updated the compiler config so all flags are in a separate section.

* Updated the documentation for the `compilers.yaml` file spec.

* Implemented basic testing for the 'flags' section of compiler config.

* Fixed a few minor problems with the manual compiler config documentation.
2016-10-24 14:37:03 -07:00
George Hartzell
e35c023c61 Typo: becuase -> because (#2110) 2016-10-24 14:28:04 -07:00
Elizabeth Fischer
4f0b7cd2c2 Small fix to docs. (#2092) 2016-10-24 11:09:02 -07:00
Massimiliano Culpo
8af9881e9b spack test : exits after ctrl+c fixes #2029 (#2082) 2016-10-24 11:07:08 -07:00
alalazo
7bd735416d package.py : moved each specialized package to its own module file 2016-10-23 22:38:19 +02:00
alalazo
e0f3188970 spack setup : improved error message 2016-10-23 19:10:07 +02:00
alalazo
fa3f07c092 CMakePackage, AutotoolsPackage : added default behavior on check 2016-10-23 19:02:46 +02:00
alalazo
ebbbed166e Merge branch 'develop' of https://github.com/LLNL/spack into features/install_with_phases_rebase 2016-10-23 13:35:12 +02:00
alalazo
284ed13fa6 spack.error : fixed pickling and representation to permit to pass FetchErrors 2016-10-23 13:34:29 +02:00
alalazo
bdf4832269 spack build, spack configure : added commands 2016-10-22 17:08:52 +02:00
alalazo
484aaf50cc CMakePackage : changed method name from wdir to build_directory 2016-10-22 14:54:26 +02:00
alalazo
8091a3d6cb do_install : use build_system_class attribute instead of type(self).__base__ 2016-10-22 10:48:16 +02:00
alalazo
c84123dce5 spack info : shows the build-system class used 2016-10-22 10:39:33 +02:00
Matthew LeGendre
859d296105 Don't clear LD_LIBRARY_PATH and friends from compiler wrappers (#2074)
* Don't clear LD_LIBRARY_PATH and friends from compiler wrappers

* remove debugging print
2016-10-21 16:25:12 -07:00
alalazo
482f60d1d3 packages : moved decorators into AutotoolsPackage and CMakePackage 2016-10-22 01:03:07 +02:00
alalazo
012da99644 spack create : fixed typo 2016-10-22 00:10:37 +02:00
alalazo
e8dafd1090 Removed space before colon in FIXME:, added one line description of guess classes 2016-10-21 23:54:11 +02:00
alalazo
715e029361 spack info : added phases 2016-10-21 22:44:41 +02:00
Matthew LeGendre
5ff08386af Remove unnecessary blackslash for flake8 2016-10-21 13:17:23 -07:00
alalazo
fd2b72fd0f qa : flake8 issues 2016-10-21 22:15:29 +02:00
alalazo
04821c7be8 spack create : now creates packages that are derived from AutotoolsPackage and CMakePackage 2016-10-21 22:12:21 +02:00
Matthew LeGendre
73b46a92bc Fix concretize bug where provider sort couldn't handle version ranges 2016-10-21 11:59:41 -07:00
Adam J. Stewart
52158d9316 Add new Version property to handle joined version numbers (#2062)
* Add new version property to handle joined version numbers

* Add unit test for new joined property

* Add documentation on version.up_to() and version.joined
2016-10-21 07:49:36 -07:00
Massimiliano Culpo
aa86432ec6 patch directive : fixed retrieval from urls ( fixes #1584 ) (#2039)
* patch directive : fixed retrieval from urls fixes #1584

 - add support for 'gz' archives
 - fixed bugs with URL patches
 - updated nwchem

* patch directive : added checksum to UrlPatch

 - refactored classes in patch.py
 - updated nwchem

* patch directive : added caching
2016-10-21 07:32:52 -07:00
alalazo
2251428f0a CMakePackage : changed list() to [] 2016-10-21 13:01:55 +02:00
alalazo
5ce3071143 do_install : removed install_self from the list of arguments (leftover after rebasing #1956) 2016-10-21 12:51:38 +02:00
alalazo
213e3f0860 Merge branch 'develop' of https://github.com/LLNL/spack into features/install_with_phases_rebase
Conflicts:
	lib/spack/spack/cmd/install.py
	lib/spack/spack/cmd/setup.py
2016-10-21 12:38:43 +02:00
scheibelp
8111a4bb9d Don't cache truncated dag hash for spec (#2048)
If Spec.dag_hash was called for the first time with a 'length'
specified, the cached hash was truncated. This ensures that the
full hash is cached.
2016-10-18 22:34:46 -07:00
Massimiliano Culpo
0a3cc5e8e3 Package.do_install : removed install_self from arguments (#1956)
* Removes the extra argument from Package.do_install while maintaining the changes in behavior pulled in #1603

* install : removed -i and -d shorthands (breaks backward compatibility)

* Change ':' to ','
2016-10-18 14:25:07 -07:00
Eric
306eea5e59 Support for bazel (#2023)
* Add package bazel

* Add support for bazel-based builds
2016-10-18 07:46:42 -07:00
Adam J. Stewart
d891143a08 Ignore GPG/PGP signatures/checksums when finding available downloads (#2028)
* Ignore GPG/PGP signatures/checksums when finding available downloads

* Remove duplicated basename
2016-10-16 14:46:29 -07:00
Todd Gamblin
193f68083f Platform-specific config scopes (#2030)
* Add platform-specific configuration scopes.

* Update `spack config` to use the new scope arguments.
2016-10-15 17:00:11 -07:00
Massimiliano Culpo
8d5961ecaf spack purge : updated documentation fixes #1938 (#2025) 2016-10-15 15:51:49 -07:00
Adam J. Stewart
78d3c7e2a2 Major updates to Contribution Guide (#1968)
* Major updates to Contribution Guide

* Grammar changes

* Fix missing/extra backticks

* Rewording, links, and tips added
2016-10-14 09:16:13 -07:00
Todd Gamblin
d861a52ebe Use cYAML if it is available in Python. (#2010) 2016-10-12 18:25:18 -07:00
Ben Boeckel
b369be65d7 Mock up Xcode devdir to make Qt5 work on Mac OS X (#1832)
* build_environment: allow compilers to set up an environment

* clang: mock up a toolchain directory for xcode

Some projects ignore CC and CXX flags and instead use xcode to find the
toolchain. Clang on Apple should set up the environment properly.
Arguably, every compiler could do this on Apple, but let's see how this
works out just for AppleClang for now.

The Documentation directory is ~1.7G and the excluded platforms add up
to about 7G. Ignoring swift saves another 500M. The resulting Xcode.app
copy is in the 2G range.

* compiler: set member variables early

This is required so that later methods can query things such as the
version of the compiler.

* compiler: support finding the real path of the compiler

On Apple, the /usr/bin compilers are actually wrapping tools themselves
which query xcrun for the currently selected Xcode installation. Pierce
this veil and get the real, full path the to underlying compilers
instead.

* icu4c: install with rpath

On macOS, icu installs with a library ID of the library name. Enabling
rpath makes its ID its full installed path which lets Qt5 link against
it successfully.

* qt: no -no-gtkstyle flag on Qt5 on macOS
2016-10-12 09:58:12 -07:00
Massimiliano Culpo
1925db5c77 spack list : updated documentation (#2004)
* spack list : updated documentation

* spack list : removed space before ':'
2016-10-12 09:43:28 -07:00
Glenn Johnson
5b070418e7 Documentation for modules configuration options (#1685)
This PR add documentation for the `suffixes` and `hash_length` options
for modules.yaml.

This resolves #1416.
2016-10-11 23:14:42 -07:00
Todd Gamblin
488e1bab2c Make insecure option work with curl AND git. (#1786) 2016-10-11 23:13:40 -07:00
Massimiliano Culpo
eb17895630 environment modifications : permit to pass parameters along with file to be sourced (#1626) 2016-10-11 22:55:14 -07:00
alalazo
dd56784d28 qa : flake8 issues 2016-10-11 16:44:18 +02:00
alalazo
ab995df777 Merge branch 'develop' of https://github.com/LLNL/spack into features/install_with_phases_rebase
Conflicts:
	lib/spack/spack/build_environment.py
	lib/spack/spack/cmd/install.py
	lib/spack/spack/cmd/setup.py
	lib/spack/spack/package.py
	var/spack/repos/builtin/packages/gmp/package.py
	var/spack/repos/builtin/packages/hdf5/package.py
2016-10-11 16:28:08 +02:00
Mario Melara
b42dbc01fe Use python platform.system for system ID (#1499)
* Rebase and merging using platform.system

Rebasing and merging using platform.system instead of uname -a.

* Add missing import platform statement

* Remove subprocess import

Remove ununsed import subprocess to make changes flak8 compliant
2016-10-11 07:04:29 -07:00