Commit graph

2444 commits

Author SHA1 Message Date
Patrick Gartung
1c8bdd7e24 buildcache fixes: index.html & unsigned installs
This fixes a syntax error in the index.html file generated by the
"spack buildcache" command when creating build caches. This also
fixes support for installing unsigned binaries.
2017-08-16 19:03:14 -07:00
Adam J. Stewart
af02774b3e Add tab completion & update docs for buildcache
This adds tab completion and fixes some formatting issues in the
documentation for the "spack buildcache" command.
2017-08-16 13:58:09 -07:00
Adam J. Stewart
11b3ce27b7 Add better generator support to CMakePackage (#4988)
* Add better generator support to CMakePackage

* List valid CMake generators on error
2017-08-16 12:25:37 -05:00
Adam J. Stewart
db657d938d Refactor IntelInstaller into IntelPackage base class (#4300)
* Refactor IntelInstaller into IntelPackage base class

* Move license attributes from __init__ to class-level

* Flake8 fixes: remove unused imports

* Fix logic that writes the silent.cfg file

* More specific version numbers for Intel MPI

* Rework logic that selects components to install

* Final changes necessary to get intel package working

* Various updates to intel-parallel-studio

* Add latest version of every Intel package

* Add environment variables for Intel packages

* Update env vars for intel package

* Finalize components for intel-parallel-studio package

Adds a +tbb variant to intel-parallel-studio.
The tbb package was renamed to intel-tbb.
Now both intel-tbb and intel-parallel-studio+tbb
provide tbb.

* Overhaul environment variables set by intel-parallel-studio

* Point dependent packages to the correct MPI wrappers

* Never default to intel-parallel-studio

* Gather env vars by sourcing setup scripts

* Use mpiicc instead of mpicc when using Intel compiler

* Undo change to ARCH

* Add changes from intel-mpi to intel-parallel-studio

* Add comment explaining mpicc vs mpiicc

* Prepend env vars containing 'PATH' or separators

* Flake8 fix

* Fix bugs in from_sourcing_file

* Indentation fix

* Prepend, not set if contains separator

* Fix license symlinking broken by changes to intel-parallel-studio

* Use comments instead of docstrings to document attributes

* Flake8 fixes

* Use a set instead of a list to prevent duplicate components

* Fix MKL and MPI library linking directories

* Remove +all variant from intel-parallel-studio

* It is not possible to build with MKL, GCC, and OpenMP at this time

* Found a workaround for locating GCC libraries

* Typos and variable names

* Fix initialization of empty LibraryList
2017-08-16 12:21:07 -05:00
Patrick Gartung
ab56c742ca Create, install and relocate tarballs of installed packages
Adds the "buildcache" command to spack. The buildcache command is
used to create gpg signatures for archives of installed spack
packages; the signatures and archives are placed together in a
directory that can be added to a spack mirror. A user can retrieve
the archives from a mirror and verify their integrity using the
buildcache command. It is often the case that the user's Spack
instance is located in a different path compared to the Spack
instance used to generate the package archive and signature, so
this includes logic to relocate the RPATHs generated by Spack.
2017-08-14 14:32:27 -07:00
Massimiliano Culpo
5184edb15f Added a custom action for --clean and --dirty. (#5081)
The action `CleanOrDirtyAction` has been added. It sets the default
value for `dest` to `spack.dirty`, and changes it according to the flags
passed via command line. Added unit tests to check that the arguments
are parsed correctly. Removed lines in `PackageBase` that were setting
the default value of dirty according to what was in the configuration.
2017-08-14 11:29:18 -07:00
scheibelp
4d9ef49b49 Decode process stream only for python3
Popen.communicate outputs a str object for python2 and a bytes
object for python3. This updates the Executable.__call__ function
to call .decode on the output of Popen.communicate only for python3.
This ensures that Executable.__call__ returns a str for python2 and
python3.
2017-08-14 11:23:03 -07:00
Massimiliano Culpo
41c87a71a8 patch: get correct package directory for a given package
fixes #4236
fixes #5002

When a package is defined in more than one repository,
RepoPath.dirname_for_package_name may return the path
to either definition. This sidesteps that ambiguity by 
accessing the module associated with the package definition.
2017-08-11 19:18:52 -07:00
becker33
79f9548a9a bugfix for module_cmd (#5038)
* bugfix for modulecmd when bash is symlinked to sh

* update test to make sure module_cmd can interpret sh
2017-08-09 12:14:35 -07:00
Massimiliano Culpo
faeb1b77b2 Merged 'purge' command with 'clean' and deleted 'purge' (#4970)
* Merged 'purge' command with 'clean'. Deleted 'purge'. fixes #2942

'spack purge' has been merged with 'spack clean'. Documentation has been
updated accordingly. The 'clean' and 'purge' behavior are not mutually
exclusive, and they log brief information to tty while they go.

* Fixed a wrong reference to spack clean in the docs

* Added tests for 'spack clean'. Updated bash completion.
2017-08-09 10:02:38 -07:00
Sergey Kosukhin
e463461ed1 Bugfixes for compiler detection on the Cray platform. (#3075)
* Typo fixes in docstrings.

* Let OS classes know if the paths they get were explicitly specified by user.

* Fixed regexp for cray compiler version matching.

* Replaced LinuxDistro with CrayFrontend for the Cray platform's frontend.
2017-08-09 09:00:34 -07:00
scheibelp
5a94cee216 Variant satisfaction for indirect dependencies
Fixes #4898

Constraints that were supposed to be conditionally activated for
specified values of a single-valued variant were being activated
unconditionally in the case that the variant was associated with
an implicit dependency. For example if X->Y->Z and Y places a
conditional constraint on Z for a given single-valued variant on
Y, then it would have been applied unconditionally when
concretizing X.
2017-08-07 10:48:48 -07:00
Adam J. Stewart
a31ce17f0b Various fixes to package name URL parsing (#4978) 2017-08-05 10:15:47 -05:00
Adam J. Stewart
c7df12f698 Massive conversion from Package to CMakePackage (#4975) 2017-08-05 10:15:18 -05:00
Adam J. Stewart
7eb263effe Add a SConsPackage base class (#4936)
* Add a SConsPackage base class

* Make Matlab extendable

* Most dependencies are actually required

* Cantera requires older version of fmt
2017-08-04 16:52:10 -05:00
Adam J. Stewart
b8ed61cfea Make CMake the default build system (#4862) 2017-08-04 14:53:05 -05:00
Adam J. Stewart
452f382293 Add a QMakePackage base class (#4925)
* Add a QMakePackage base class

* Fix sqlite linking bug in qt-creator

* Add latest version of qt-creator

* Add latest version of qwt

* Use raw strings for regular expressions

* Increase minimum required version of qt

* Add comment about specific version of sqlite required

* Fixes for latest version of qwt and qt-creator

* Older versions of Qwt only work with older versions of Qt
2017-08-04 08:46:07 -07:00
Adam J. Stewart
4e269510c5 Fix trailing whitespace at the end of headers.cpp_flags (#4957) 2017-08-03 14:24:51 -05:00
George Hartzell
36496b9174 Fix crashes when running spack install under nohup (#4926)
* Fix crashes when running spack install under nohup

Fixes #4919

For reasons that I do not entire understand, duplicate_stream() throws
an '[Errno 22] Invalid argument' exception when it tries to
`os.fdopen()` the duplicated file descriptor generated by
`os.dup(original.fileno())`.  See spack/llnl/util/lang.py, line
394-ish.

This happens when run under `nohup`, which supposedly has hooked
`stdin` to `/dev/null`.

It seems like opening and using `devnull` on the `input_stream` in
this situation is a reasonable way to handle the problem.

* Be more specific about error being handled.

Only catch the specific error that happens when trying to dup
the stdin that nohup provides.

Catching e as a StandardErorr and then
`type(e).__name__` tells me that it's an OSError.

Printing e.errno tells me that it's 22

Double checking tells me that 22 is EINVAL.

Phew.
2017-08-03 10:33:16 -07:00
Sergey Kosukhin
963eb99b7f Account for hyphens in package names when searching for libraries. (#4948) 2017-08-03 09:17:07 +02:00
Todd Gamblin
44ce0adbd5 Fix color bug in Spec.format() introduced by #3013 2017-08-01 17:40:54 -07:00
Todd Gamblin
b88f55e523 Add spack dependencies command and tests for it and dependents. 2017-08-01 17:40:54 -07:00
Todd Gamblin
af3c794ab5 document and make display_specs more versatile 2017-08-01 17:40:54 -07:00
Todd Gamblin
b575d008bd Fix issue with case check and spack -m 2017-08-01 17:40:54 -07:00
Todd Gamblin
c8b2100630 Refactor installed_dependents -> installed_relatives 2017-08-01 17:40:54 -07:00
Todd Gamblin
36b3dd8cfe Package.possible_dependencies() descends into virtuals. 2017-08-01 17:40:54 -07:00
Todd Gamblin
b9606e3157 Add --transitive option to spack dependents 2017-08-01 17:40:54 -07:00
Todd Gamblin
6928cf7a68 spack dependents lists possible dependencies by default. 2017-08-01 17:40:54 -07:00
Todd Gamblin
bd94a17066 Remove last vestiges of "special" deptypes.
- Remove `special_types` dict in spec.py, as only 'all' is still used
- Still allow 'all' to be used as a deptype
- Simplify `canonical_deptype` function
- Clean up args in spack graph
- Add tests
2017-08-01 17:40:54 -07:00
Todd Gamblin
43f576cf19 Remove unused code. 2017-08-01 17:40:54 -07:00
Todd Gamblin
f2ddcfac5f Add --all argument to spack dependents
--all causes spack dependents to list all possible dependents for a
package, rather than actual dependents for an installed spec.
2017-08-01 17:40:54 -07:00
Adam J. Stewart
82735deafd Clarify docs on using a hash in a spec (#4908) 2017-07-31 13:13:39 -07:00
scheibelp
69a6c8ef78 Fix preference for X.Y version when mixed with X.Y.Z versions (#4922)
For packages which contain a mix of versions with formats X.Y and
X.Y.Z, if the user entered an X.Y version as a preference in
packages.yaml, Spack would get confused and favor any version A.B.Z
where X=A and Y=B. In the case where there is a mix of these version
types, this commit updates preferences so Spack will favor an exact
match.
2017-07-31 13:11:08 -07:00
paulhopkins
1c7e5724d9 Add --color=[always|never|auto] argument; fix color when piping (#3013)
* Disable spec colorization when redirecting stdout and add command line flag to re-enable
* Add command line `--color` flag to control output colorization
* Add options to `llnl.util.tty.color` to allow color to be auto/always/never
* Add `Spec.cformat()` function to be used when `format()` should have auto-coloring
2017-07-31 12:57:47 -07:00
Adam J. Stewart
b33f92da34 Improve version detection for URLs with dynamic after version (#4902) 2017-07-26 17:43:54 -05:00
Adam J. Stewart
07aec4366f Add universal build_type variant to CMakePackage (#4797)
* Add universal build_type variant to CMakePackage
* Override build_type in some packages with different possible values
* Remove reference to no longer existent debug variant
* Update CBTF packages with new build_type variant
* Keep note on build size of LLVM
2017-07-25 16:34:43 -07:00
Gregory Lee
42717bd8e8 fix config.guess patch for ppc64le (#4858)
* fix config.guess patch for ppc64le

* explicit patch for config.guess not required
2017-07-25 13:44:51 -07:00
Adam J. Stewart
250ee413e9 Change Version formatting properties and functions to return Version objects (#4834)
* Change version.up_to() to return Version() object
* Add unit tests for Version.up_to()
* Fix packages that expected up_to() to return a string
* Ensure that up_to() preserves separator characters
* Use version indexing instead of up_to
* Make all Version formatting properties return Version objects
* Update docs
* Tests need to test string representation
2017-07-24 13:02:13 -07:00
Todd Gamblin
f159246d1d Make testing spack commands simpler (#4868)
Adds SpackCommand class allowing Spack commands to be easily in Python

Example usage:

    from spack.main import SpackCommand
    info = SpackCommand('info')
    out, err = info('mpich')
    print(info.returncode)

This allows easier testing of Spack commands.

Also:
* Simplify command tests
* Simplify mocking in command tests.
* Simplify module command test
* Simplify python command test
* Simplify uninstall command test
* Simplify url command test
* SpackCommand uses more compatible output redirection
2017-07-22 21:27:54 -07:00
becker33
f962aba6ce Allow packages to control handling of compiler flags (#4421)
* Initial work on flag trapping using functions called <flag>_handler and default_flag_handler

* Update packages so they do not obliterate flags

* Added append to EnvironmentModifications class

* changed EnvironmentModifications to have append_flags method

* changed flag_val to be a tuple

* Increased test coverage

* added documentation of flag handling
2017-07-19 20:12:00 -07:00
Adam J. Stewart
5fc0243d35 Improve version detection of release versions (#4816) 2017-07-18 22:48:39 -05:00
becker33
d2a63d55fa Open ended variants (#4746)
* Change directives to allow open-ended variants more easily

* make None default to open-ended
2017-07-18 10:03:15 -07:00
Adam J. Stewart
1215c3b20c Change path to CMakeLists.txt to be relative to root, not pwd (#4420)
* Change path to CMakeLists.txt to be relative to root, not pwd

* Changes requested during code review

* Revert back to old naming of root_cmakelists_dir
* Make relative directory more clear in docs

* Revert change causing build_type AttributeError

* Fix forgotten abs_path var

* Update CLHEP with new relative path

* Update more packages with new root_cmakelists_dir syntax
2017-07-18 09:58:19 -07:00
Massimiliano Culpo
5a1ee22575 package: removed default no-op patch (#4103)
* package: removed default no-op patch fixes #4085

* do_patch: handles NoSuchMethodError nicely
2017-07-18 09:53:35 -07:00
George Hartzell
ff906faf9a Typo: submdoules -> submodules (#4716) 2017-07-12 07:48:31 +02:00
Gregory Lee
28cb1e4379 patch config.guess after autoreconf step (#4604) 2017-07-06 23:07:54 -05:00
Todd Gamblin
b4d1654e68 Parametrized lock test and make it work with MPI
- Lock test can be run either as a node-local test or as an MPI test.

- Lock test is now parametrized by filesystem, so you can test the
  locking capabilities of your NFS, Lustre, or GPFS filesystem.  See docs
  for details.
2017-07-04 11:41:37 -07:00
Todd Gamblin
bd7a591df1 Make filesytem more resilient to concurrent updates.
- Uses O_CREAT for touch (for guaranteed atomic open on NFS, multi-node)
- Ignore concurrent create errors in mkdirp
2017-07-04 11:41:37 -07:00
Todd Gamblin
326e2f7f66 Ported lock test to pytest. 2017-07-04 11:41:37 -07:00
Adam J. Stewart
9933d759ac Update hdfgroup packages to new URL structure (#4643)
* Update hdfgroup packages to new URL structure

* Update docs now that HDF5 URL isn't that complicated
2017-06-29 21:38:47 -05:00