Commit graph

2770 commits

Author SHA1 Message Date
Sergey Kosukhin
0f35f00b6e Use GitLab's API endpoint for fetching a git snapshot. (#7881)
* Use GitLab's API endpoint for fetching a git snapshot.

* More GitLab packages use the API.

* find_list_url for GitLab's API URLs.

* Flake8

* Url for 'hacckernels'.

* Check GitLab API regexps before the non-API ones.
2018-04-26 16:40:11 -05:00
健美猫
9dad069e89 config_args does NOT apply to CMakePackage. (#7917) 2018-04-26 13:03:45 -05:00
Matthew Scott Krafczyk
1b38631781 Add -d option to diy to specify source path move -j to common args (#5963) 2018-04-18 17:26:14 -05:00
Elizabeth Fischer
d3c1463b0a Fix bug/oversight in formatting error message. (#7810) 2018-04-18 15:05:37 -05:00
George Hartzell
669b85e6bc Activation of activated package just msg's (#7752)
Activating a package that is already activated now sends a `tty.msg`
and returns.

```
-bash-4.2$ ~/spack/bin/spack activate aspell6-en
==> Package aspell6-en/lc4v24f is already activated.
```
2018-04-17 07:44:08 -05:00
Massimiliano Culpo
4bf7ce7d99 Better error message for spack providers (#7748)
* Better error message for spack providers

fixes #1355

`spack providers` now outputs a sensible error message if non-virtual
specs are provided as arguments:
```
$ spack providers mpi zlib petsc
==> Error: non-virtual specs cannot be part of the query [zlib, petsc]
```

Formatting of the output changed slightly.

* Calling 'spack providers' without arguments print the virtual pkg list

Also, the error message in case of a wrong parameter has been improved
to show the list of valid packages.

* Avoid printing headers if stdout is not a tty

* The provider list is formatted with colify if not in a tty

* Added a test to check the list of providers returned from the command
2018-04-17 07:30:32 -05:00
Cameron Smith
e6d3ab8405 cray: add comment about module use in packages.yaml (#7751)
This advises users to prefer `modules:` entries in `packages.yaml` on Cray.
2018-04-13 11:53:26 -07:00
becker33
b9c8cdcbe3
Fix when_spec inheritance for extensions (#7746)
Popping the when spec from kwargs in the extends directive breaks
class inheritance. Inheriting classes do not find their when spec.

We now get the when spec from kwargs instead, leaving it to be found
by any downstream package classes.
2018-04-13 11:38:34 -07:00
Massimiliano Culpo
1307ad3979
Package.provides account for v deps that are provided conditionally (#7716)
fixes #7705

Package.provides now checks constraints to ensure that a spec provides
a given virtual package. Note that 'strict=True' is not passed to
satisfies as this function is also used during concretization.
2018-04-11 21:50:03 +02:00
healther
6a0f9ccf82 make view understand hashes (#7573)
Fixes #7548

This updates the "spack view" command to use the same parsing logic
as "spack install" on the user-provided specs. For example you can
provide a DAG hash to refer to an exact installed spec instead of
specifying name, compiler, etc.
2018-04-04 10:40:56 -07:00
Adam J. Stewart
c50628ebdf Fix bug in test_find_with_globbing unit test (#7652) 2018-04-03 22:27:17 -07:00
Zack Galbreath
f613437a44 Do not call sys.exit() in except block (#7659)
When an invalid spec is encountered by `parse_specs()` we now raise a
`SpackError` instead of calling `sys.exit()`
2018-04-03 22:22:28 -07:00
Adam J. Stewart
cdefbd7475 Avoid double 'lib' for packages whose name already starts with lib (#7651) 2018-03-31 17:33:41 +02:00
Axel Huebl
4c6199923e mirrors.yaml: does not expand ~ (#7631)
Use `$HOME` for cli commands in `mirrors.yaml` and add
"username" example strings in the file itself.
2018-03-28 14:30:18 -05:00
Patrick Gartung
52d55ea317 Fix buildcache relocate: only skip if rpaths are relative (#7605)
This fixes a check that decides when to skip buildcache relocation.
Originally the check was flawed in two ways: it would skip if the
source prefix matched the destination prefix, which no longer
matters since the source prefix is replaced with a placeholder
(so it always needs to be updated); it also would skip relocation
if the rpaths were not relative, when in fact it should be the
opposite (binaries without relative rpaths *should* be relocated,
and those without don't need it).
2018-03-27 10:54:26 -07:00
Todd Gamblin
54f97d1dec
Update copyright on LLNL files for 2018. (#7592) 2018-03-24 12:13:52 -07:00
Todd Gamblin
af0f94a1af
Avoid stat-ing all packages at startup. (#7587)
- FastPackageChecker was being called at startup every time Spack runs,
  which takes a long time on networked filesystems.  Startup was taking
  5-7 seconds due to this call.

- The checker was intended to avaoid importing all packages (which is
  really expensive) when all it needs is to stat them.  So it's only
  "fast" for parts of the code that *need* it.

- This commit makes repositories instantiate the checker lazily, so it's
  only constructed when needed.
2018-03-24 07:39:10 -07:00
Todd Gamblin
998b5a6482
Remove case consistency check at startup. (#7585)
- This was needed when we transitioned to all lowercase packages because
  git didn't handle case changes well on case-insensitive filesystems.

- Now it just adds extra stat calls to startup, and we check for
  all-lowercase package names in tests, so we'll remove it.

- people using really old versions of Spack can re-clone.
2018-03-23 22:38:31 -07:00
Mario Melara
385622953d Cray clean environment (#7582)
* Create unload_module method

Extract code from load_module into unload_module.

* Unload modules to create a clean env on Cray

removes cray-libsci, cray-mpich and darshan to prevent any silent
linking with those packages.
2018-03-23 14:53:52 -07:00
Adam J. Stewart
666e8e3a9b Increment Spack version (#7565) 2018-03-23 14:43:22 -07:00
becker33
1c169ead8b
Grab crayos version from PrgEnv module versions (#7583) 2018-03-23 14:41:43 -07:00
Mario Melara
2970a50ef6 Add the ability to separate architecture in path scheme (#4329)
* Add format to separate target and os for path

spec format can now handle separations of target and os for setting
up the path.

* Added ${PLATFORM} et al to spec.format()

${PLATFORM}, ${OS}, ${TARGET}

* Update tests

Updated tests and got rid of unnecessary code.

* Also update documentation to reflect this new ability.

* Add default path scheme to config.yaml

Added default path scheme to config.yaml. Users can overwrite this
section if they want.
2018-03-23 12:05:07 -07:00
Veselin Dobrev
4ddbc96c7b Speedup the default 'libs' property search ... (#7553)
* Speedup the default 'libs' property search - important for external
packages.

* As advised by @alalazo, use tuples instead of lists inside
_libs_default_handler.
2018-03-22 20:04:28 -05:00
Adam J. Stewart
f721fb3e4c
Fix error raising during install with Python 3 (#6894)
* Fix error raising during install with Python 3

* Fix env_path too
2018-03-22 19:37:49 -05:00
Massimiliano Culpo
5655895865 Record installation date and time in DB (#7334)
* Added installation date and time to the database

Information on the date and time of installation of a spec is recorded
into the database. The information is retained on reindexing.

* Expose the possibility to query for installation date

The DB can now be queried for specs that have been installed in a given
time window. This query possibility is exposed to command line via two
new options of the `find` command.

* Extended docstring for Database._add

* Use timestamps since the epoch instead of formatted date in the DB

* Allow 'pretty date' formats from command line

* Substituted kwargs with explicit arguments

* Simplified regex for pretty date strings. Added unit tests.
2018-03-22 11:07:27 -07:00
Massimiliano Culpo
6699ba8769 Recover the old patching order for packages. (#7558)
fixes #7543

This is very likely an hot-fix, while a more permanent solution is
needed. See this comment for more insight:

 https://github.com/spack/spack/pull/7193#discussion_r176448831

on the problem.
2018-03-22 10:09:43 -07:00
becker33
f8657e3fd9 Propagate architecture information during concretization (#7412)
This updates architecture concretization to

* Search for the nearest parent in the DAG for architecture information
  rather than defaulting to the root of the DAG
* Propagate architecture settings transitively, such that if for
  example the target is set at the root of the dag it will set the
  same target on indirect dependencies (assuming no intermediate
  dependency specifies a separate target). Previously this occurred
  in general but under some conditions did not, for example if an
  intermediate dependency specified some subset of architecture
  properties.
2018-03-21 18:14:01 -07:00
scheibelp
6548f2db5c Enable creation of mirrors for systems with different compilers (#5153)
* Create mirror for system with different compilers

Spack concretizes the spec provided by the user in
"spack mirror create" to ensure downloading the right
dependencies. Under normal circumstances concretization
requires that the chosen compiler exists on the system,
but this is not required when creating download mirrors
for other systems, so this requirement is removed in that
case.

* Add test for disabling compiler existence check

* Update compiler existence checking logic

* improve test for disabling compiler existence check
2018-03-21 14:55:26 -07:00
健美猫
71055a8ef4 Generate spack docset for Dash. (#6981) 2018-03-21 16:37:26 -05:00
Gregory Lee
7504e9997d make py-setuptools a run-time-only dep for py-basemap and patch pytho… (#6989)
* make py-setuptools a run-time-only dep for py-basemap and patch python package to only apply setuptools flag for build deps

* py-qtconsole does not require setuptools
2018-03-21 16:34:27 -05:00
scheibelp
e8a0637606 Support MD5 with a warning when OpenSSL is compiled in FIPS mode (#7531)
* This allows Spack to work with MD5 hashes on machines with openssl in FIPS mode.
* We are still using MD5 for validation in many places, and a later PR will replace all uses of MD5 with SHA256.
* This is a quick fix until that happens.
2018-03-21 14:11:54 -07:00
Gregory Lee
500ac793bc patch config.guess for ARM (aarch64) (#7427) 2018-03-21 13:13:38 -05:00
Peter Scheibel
6458b15bb1 spec: support "full" hashes
This hash includes the content of the package itself as well as the DAG
for the package.
2018-03-20 00:29:54 -07:00
Peter Scheibel
3501f2f4b5 package: add a content_hash method for packages
This calculates a hash which depends on the complete content of the
package including sources and the associated `package.py` file.
2018-03-20 00:29:54 -07:00
Peter Scheibel
e97c28e5b3 package: ensure patches are applied in-order
This helps to ensure that patches are applied consistently and will also
be used as the source for the patch part of full package hashes.
2018-03-20 00:29:54 -07:00
Peter Scheibel
2379ed54b9 package_hash: add code to generate a hash for a package file
This will be included in the full hash of packages.
2018-03-20 00:29:54 -07:00
Peter Scheibel
db81d19ddd PackageBase: add a list of metadata attributes
These attributes are ignored when doing a content hash of a package.
2018-03-20 00:29:54 -07:00
Peter Scheibel
39cb9d0397 fetch_strategy: add a source_id to fetches
This will be included in the full hash of packages.
2018-03-20 00:29:54 -07:00
Todd Gamblin
b4a9e37a98
Fix bugs in spack dependencies command (#7435)
- transitive dependencies were not being handled correctly

- restructure code to do recursion and mark visited packages properly

- add `-V` option to *not* expand virtuals in spack dependencies
2018-03-12 17:26:58 +09:00
Massimiliano Culpo
a1c8ce82f2 Constructing a SpecBuildInterface from another gives no inconsistent MRO (#7457)
fixes #7239
2018-03-12 17:07:08 +09:00
Patrick Gartung
a4e75c7f0e spack buildcache: allow unsigned tarballs (#7440)
This re-adds the option to create and install unsigned tarballs, now
with the -u option (--unsigned) rather than the -y option.

This also changes the "keys" command, replacing the -y/--yes-to-all
option with the -t/--trust option (which has the same effect but is
more-clearly named).
2018-03-09 15:29:10 -08:00
Levi Baber
38bb0e3e7d resource: convert symlink to real path (#7417)
Fixes #7130

shutil.move expects a source path like "/x/y/" to be a directory and
fails if "/x/y" is a symlink. This invokes realpath on the source
path to avoid the issue.
2018-03-09 12:40:57 -08:00
scheibelp
f27f20e5b3
replace dash with underscore in os name/version (#7381)
Fixes #7356

In some cases OperatingSystem (e.g. LinuxDistro) was getting
instantiated with a version that contains dashes. This breaks because
the concretizer later converts this value to a string and re-parses
it, and the '-' character is used to separate architecture components.
This adds a guard in the initializer to convert '-' to '_'.
2018-03-09 10:03:58 -08:00
Massimiliano Culpo
d77dd298eb
Changed CudaPackage base class to PackageBase (#7416)
CudaPackage is always used as a mixin. As such it should inherit from
PackageBase, and not Package.
2018-03-07 13:45:11 +01:00
Patrick Gartung
f5467957bc Improved binary relocation: error checking and reporting (#7314)
Fixes #7237
Fixes #6404
Fixes #6418
Fixes #6369

Identify when binary relocation fails to remove all instances of the
source prefix (and report an error to the user unless they specify
-a to allow the old root to appear). This check occurs at two stages:
during "bincache create" all instances of the root are replaced with
a special placeholder string ("@@@@..."), and a failure occurs if the
root is detected at this point; when the binary package is extracted
there is a second check. This addresses #7237 and #6418.

This is intended to be compatible with previously-created binary
packages.

This also adds:

* Better error messages for "spack install --use-cache" (#6404)
* Faster relocation on Mac OS (using a single call to
  install_name_tool for all files rather than a call for each file)
* Clean up when "buildcache create" fails (addresses #6369)
* Explicit error message when the spack instance extracting the binary
  package uses a different install layout than the spack instance that
  created the binary package (since this is currently not supported)
* Remove the option to create unsigned binary packages with -y
2018-03-06 18:02:53 -08:00
Mario Melara
ccb24b1438 Fix pkgconf on Crays (#7171)
This updates Cray.setup_platform_environment to use cray-specific
pkgconfig paths so that all providers of 'pkgconfig' have access
to them (previously the 'pkg-config' provider had this but the
'pkgconf' provider did not).
2018-03-05 12:08:11 -08:00
scheibelp
8b0552bc23 only filter compiler paths for binaries provided by the compiler (#7369) 2018-03-05 14:43:43 +01:00
Veselin Dobrev
d3ed46e958 [spack/spec.py] raise a query failure error if a property query returns None (#7277)
* [SPACK/spec.py] When a query through ForwardQueryToPackage returns
'None', treat that as query failure and raise RuntimeError with
suitable message. This overrides the current behavior to raise an
AttributeError which is now triggered only when no suitable query
property is found and there is no default handler.

* [spack/spec.py] Fix style.

* [SPACK/spec.py] In case of query failure, i.e. property returning
'None', raise AttributeError instead of RuntimeError in order to
pass the unit test. Also, small update in the logic distinguishing
query failure and lack of relevant property/attribute handling.
2018-02-28 15:25:29 -07:00
Mario Melara
14e3eb78a7 Fix class name in build system tutorials (#7346)
AutoTools -> Autotools
2018-02-27 20:15:35 -08:00
Omar Padron
3a9cd2614e Docker-Based Developer Resource (#5871)
* add docker-based development resources

* incorporate feedback from @ax3l

* fix docs/improve ssh handling

* experiment with custom prompt

* add arch/fix missing core_compilers key

* add centos/minor tweaks

* make prompt experiment optional

* workaround problem with latest fedora docker image

* add docker documentation page to index toc

* try another documentation fix

* switch arch linux base to base/archlinux

* update the git urls in the Dockerfiles

* add opensuse

* switch CUSTOM_PROMPT variable to simply "PROMPT"
2018-02-27 00:36:14 -08:00