Commit graph

1988 commits

Author SHA1 Message Date
Matthew LeGendre
5ff08386af Remove unnecessary blackslash for flake8 2016-10-21 13:17:23 -07:00
Matthew LeGendre
73b46a92bc Fix concretize bug where provider sort couldn't handle version ranges 2016-10-21 11:59:41 -07:00
Adam J. Stewart
52158d9316 Add new Version property to handle joined version numbers (#2062)
* Add new version property to handle joined version numbers

* Add unit test for new joined property

* Add documentation on version.up_to() and version.joined
2016-10-21 07:49:36 -07:00
Massimiliano Culpo
aa86432ec6 patch directive : fixed retrieval from urls ( fixes #1584 ) (#2039)
* patch directive : fixed retrieval from urls fixes #1584

 - add support for 'gz' archives
 - fixed bugs with URL patches
 - updated nwchem

* patch directive : added checksum to UrlPatch

 - refactored classes in patch.py
 - updated nwchem

* patch directive : added caching
2016-10-21 07:32:52 -07:00
scheibelp
8111a4bb9d Don't cache truncated dag hash for spec (#2048)
If Spec.dag_hash was called for the first time with a 'length'
specified, the cached hash was truncated. This ensures that the
full hash is cached.
2016-10-18 22:34:46 -07:00
Massimiliano Culpo
0a3cc5e8e3 Package.do_install : removed install_self from arguments (#1956)
* Removes the extra argument from Package.do_install while maintaining the changes in behavior pulled in #1603

* install : removed -i and -d shorthands (breaks backward compatibility)

* Change ':' to ','
2016-10-18 14:25:07 -07:00
Eric
306eea5e59 Support for bazel (#2023)
* Add package bazel

* Add support for bazel-based builds
2016-10-18 07:46:42 -07:00
Adam J. Stewart
d891143a08 Ignore GPG/PGP signatures/checksums when finding available downloads (#2028)
* Ignore GPG/PGP signatures/checksums when finding available downloads

* Remove duplicated basename
2016-10-16 14:46:29 -07:00
Todd Gamblin
193f68083f Platform-specific config scopes (#2030)
* Add platform-specific configuration scopes.

* Update `spack config` to use the new scope arguments.
2016-10-15 17:00:11 -07:00
Massimiliano Culpo
8d5961ecaf spack purge : updated documentation fixes #1938 (#2025) 2016-10-15 15:51:49 -07:00
Adam J. Stewart
78d3c7e2a2 Major updates to Contribution Guide (#1968)
* Major updates to Contribution Guide

* Grammar changes

* Fix missing/extra backticks

* Rewording, links, and tips added
2016-10-14 09:16:13 -07:00
Todd Gamblin
d861a52ebe Use cYAML if it is available in Python. (#2010) 2016-10-12 18:25:18 -07:00
Ben Boeckel
b369be65d7 Mock up Xcode devdir to make Qt5 work on Mac OS X (#1832)
* build_environment: allow compilers to set up an environment

* clang: mock up a toolchain directory for xcode

Some projects ignore CC and CXX flags and instead use xcode to find the
toolchain. Clang on Apple should set up the environment properly.
Arguably, every compiler could do this on Apple, but let's see how this
works out just for AppleClang for now.

The Documentation directory is ~1.7G and the excluded platforms add up
to about 7G. Ignoring swift saves another 500M. The resulting Xcode.app
copy is in the 2G range.

* compiler: set member variables early

This is required so that later methods can query things such as the
version of the compiler.

* compiler: support finding the real path of the compiler

On Apple, the /usr/bin compilers are actually wrapping tools themselves
which query xcrun for the currently selected Xcode installation. Pierce
this veil and get the real, full path the to underlying compilers
instead.

* icu4c: install with rpath

On macOS, icu installs with a library ID of the library name. Enabling
rpath makes its ID its full installed path which lets Qt5 link against
it successfully.

* qt: no -no-gtkstyle flag on Qt5 on macOS
2016-10-12 09:58:12 -07:00
Massimiliano Culpo
1925db5c77 spack list : updated documentation (#2004)
* spack list : updated documentation

* spack list : removed space before ':'
2016-10-12 09:43:28 -07:00
Glenn Johnson
5b070418e7 Documentation for modules configuration options (#1685)
This PR add documentation for the `suffixes` and `hash_length` options
for modules.yaml.

This resolves #1416.
2016-10-11 23:14:42 -07:00
Todd Gamblin
488e1bab2c Make insecure option work with curl AND git. (#1786) 2016-10-11 23:13:40 -07:00
Massimiliano Culpo
eb17895630 environment modifications : permit to pass parameters along with file to be sourced (#1626) 2016-10-11 22:55:14 -07:00
Mario Melara
b42dbc01fe Use python platform.system for system ID (#1499)
* Rebase and merging using platform.system

Rebasing and merging using platform.system instead of uname -a.

* Add missing import platform statement

* Remove subprocess import

Remove ununsed import subprocess to make changes flak8 compliant
2016-10-11 07:04:29 -07:00
Todd Gamblin
9c5c8b22c8 Fix bug in spack debug create-db-tarball
- Fix a bug handling '/' characters in branch names.

- Make tarballs use a descriptive name for the top-level directory, not
  just `opt`.
2016-10-11 01:55:33 -07:00
Todd Gamblin
f0edfa6edf Roll my my own bit_length function for Python 2.6 compatibility. 2016-10-11 01:55:33 -07:00
Todd Gamblin
222f551c37 Use a single lock file for stages and a single file for prefixes.
- Locks now use fcntl range locks on a single file.

How it works for prefixes:

- Each lock is a byte range lock on the nth byte of a file.

- The lock file is ``spack.installed_db.prefix_lock`` -- the DB tells us
  what to call it and it lives alongside the install DB.  n is the
  sys.maxsize-bit prefix of the DAG hash.

For stages, we take the sha1 of the stage name and use that to select a
byte to lock.

With 100 concurrent builds, the likelihood of a false lock collision is
~5.36e-16, so this scheme should retain more than sufficient paralellism
(with no chance of false negatives), and get us reader-writer lock
semantics with a single file, so no need to clean up lots of lock files.
2016-10-11 01:55:33 -07:00
Todd Gamblin
080a78664e Add tests for locks with byte ranges. 2016-10-11 01:55:33 -07:00
Todd Gamblin
3d8d8d3644 Fix bug with lock upgrades.
- Closing and re-opening to upgrade to write will lose all existing read
  locks on this process.
  - If we didn't allow ranges, sleeping until no reads would work.
  - With ranges, we may never be able to take some legal write locks
    without invalidating all reads. e.g., if a write lock has distinct
    range from all reads, it should just work, but we'd have to close the
    file, reopen, and re-take reads.

- It's easier to just check whether the file is writable in the first
  place and open for writing from the start.

- Lock now only opens files read-only if we *can't* write them.
2016-10-11 01:55:33 -07:00
Todd Gamblin
a024c6df95 Add base32_prefix_bits function to get prefix of DAG hash as an int. 2016-10-11 01:55:32 -07:00
Todd Gamblin
da6bbfb2d4 Add byte-range parameters to llnl.util.lock 2016-10-11 01:55:32 -07:00
Todd Gamblin
ea10e3bab0 Remove need to touch lock files before using.
- Locks will now create enclosing directories and touch the lock file
  automatically.
2016-10-11 01:55:32 -07:00
Todd Gamblin
907fe912ef Make llnl.util.lock use file objects instead of low-level OS fds.
- Make sure we write, truncate, flush when setting PID and owning host in
  the file.
2016-10-11 01:55:32 -07:00
alalazo
f229290880 stage : try to remove dead links only of folder that you actually care about
A use case where the previous approach was failing is :

 - more than one spack process running on compute nodes
 - stage directory is a link to fast LOCAL storage

 In this case the processes may try to unlink something that is "dead" for them, but actually used by other processes on storage they cannot see.
2016-10-11 01:55:32 -07:00
alalazo
f47dcdc47a lockfiles : creates directory for locks if not already present 2016-10-11 01:55:32 -07:00
alalazo
a3fc492d45 diy : removed global write lock on the db 2016-10-11 01:55:32 -07:00
alalazo
74fb1029fa uninstall : removed global lock 2016-10-11 01:55:25 -07:00
alalazo
b3a6e307b9 unit tests : fixed failing tests 2016-10-11 01:38:27 -07:00
alalazo
34fe51a4aa install : finer graned locking for install command 2016-10-11 01:38:27 -07:00
James Wynne III
bd61a36587 Fixed set operation from undefined += to a union (#1963)
Fixed flake8 issues
2016-10-11 01:30:11 -07:00
Massimiliano Culpo
b7a612dcdf spack list : merged package-list into list (#1932)
* spack list : merged package-list into the command

* list : removed option for case sensitivity
2016-10-11 01:22:53 -07:00
Kelly Thompson
ff1108712c Make CMake-based builds more verbose by default. (#1988) 2016-10-11 00:43:36 -07:00
Massimiliano Culpo
b1a2728af6 uninstall : permits to uninstall all installed software fixes #1477 (#1973) 2016-10-11 00:42:31 -07:00
Massimiliano Culpo
c8bf8a5e6e md5 : normalizes input before computing the md5 fixes #1508 (#1977)
Bottomline :

- fetcher change the current working directory
- relative paths were resolved differently depending on the prder f evaluation
2016-10-11 00:37:29 -07:00
Jean-Paul Pelteret
876c26f658 Add documentation for standard python repositories. (#1970)
Fixes #1939
2016-10-07 22:46:22 -07:00
Isuru Fernando
ccf530ba5e Check for -r in ccld mode too (#1972) 2016-10-07 22:44:37 -07:00
Massimiliano Culpo
efae58a4ef fixes #858 (#1961)
Fix spack uninstall -f
2016-10-07 11:30:10 -07:00
Todd Gamblin
dbc864c9db Restore default RPATH settings but allow packages to limit to immediate deps. (#1954)
- Some packages (netcdf) NEED RPATHs for transitive deps.
- Others (dealii) will exceed OS limits when the DAG is too large.
2016-10-06 15:57:23 -07:00
Adam J. Stewart
415ddeecf8 Fix bugs preventing readthedocs from rebuilding the documentation (#1945) 2016-10-06 12:50:49 -07:00
Elizabeth Fischer
98f8f40896 [Bug Fix (and docs too)] : Do not select @develop version by default (#1933)
* This fixes a bug in concretization.  Before the recent change to the
algorithm, the intent was that the @develop version, although
"greater" than numberic versions, is never preferred BY DEFAULT over
numeric versions.

To test this... suppose you have a package with no `preferred=True` in
it, and nothing in `packages.yaml`, but with a `develop` version.  For
the sake of this example, I've hacked my `python/package.py` to work
this way.

Without bugfix (WRONG: user should never get develop by default):

```
  python@develop%clang@7.3.0-apple~tk~ucs4 arch=darwin-elcapitan-x86_64
      ...
```

With bugfix (RIGHT: largest numeric version selected):

```
  python@3.5.2%clang@7.3.0-apple~tk~ucs4 arch=darwin-elcapitan-x86_64
      ...
```

* Documented version selection in concretization algo.

* Fix typos

* flake8
2016-10-06 09:08:15 -07:00
Denis Davydov
2ccb3d5531 add contribution guide focused on Git and PRs (#1664) 2016-10-06 07:45:47 -07:00
Adam J. Stewart
83a074eea6 Fix various documentation bugs (#1678)
* Fix various documentation bugs

* Keep long option names, but don't include in Command Index

* Use long option name

* Explicitly designate sections to be listed in the Command Index

* Consistent menu bar titles
2016-10-06 02:49:44 -07:00
Matt Belhorn
6b98e86056 Fixes bug closing Executable file io-streams (#1890)
Input/output/error streams not directed to None or 'str' were not being closed
because `close()` method was being called on the argument value (a string type)
instead of the open file descriptor object.
2016-10-06 02:41:28 -07:00
Elizabeth Fischer
3de3664671 Fix fetch() method. Was being called with "wrong" arguments, raising exception. (#1916)
DIYFetchBugfix
2016-10-06 02:40:24 -07:00
Elizabeth Fischer
208537f6f2 Fix Issues with non-numeric versions, as well as preferred=True (#1561)
* Fix bug in handling of precedence of preferred=True vs. versions given in packages.yaml (#1556)

* Standardized comparison of versions: numeric versions are always greater than non-numeric versions; and non-numeric versions are sorted alphabetically.

This is
   a) simple
   b) ensures that non-numeric versions (such as 'develop') in package.py are not chosen ahead of numeric versions, when nothing is specified in packages.yaml

Fixes Issue #1557

* Removed debugging output

* Fix variable shadowing bug

* Ensure develop < numeric version.

* Bug fix.

* Passes all unit tests in versions.py

* flake8 fixes

* flake8 fixes

* Changed type test to be more correct.

See http://stackoverflow.com/questions/8203336/difference-between-int-and-numbers-integral-in-python
2016-10-06 02:35:34 -07:00
scheibelp
508d79c475 Handle packages with unparseable extensions (#1758)
This closes #1757 which provides an example of a url scheme where the
version appears after the extension. Instead of extending the parsing
logic to handle this case, this commit allows the user to specify
their extension type. This helps Spack choose the appropriate
decompressor and mirror archive filename.
2016-10-05 22:45:02 -07:00