Commit graph

637 commits

Author SHA1 Message Date
Gregory Lee
df5dc1c9bb Merge pull request #253 from LLNL/bugfix/244-uninstall-errors
Bugfix/244 uninstall errors
2015-12-19 08:06:36 -08:00
Todd Gamblin
52e3364efa Fix #244: errors on uninstall
- Extension logic didn't take conditional deps into account.

- Extension methods now check for whether the extnesion is
  in the extendee map AND whether the dependency is actually present
  in the spec yet.
2015-12-18 22:24:35 -08:00
Todd Gamblin
b7e926eef6 minor formatting. 2015-12-18 22:24:22 -08:00
Todd Gamblin
ce8bd20944 Fix import error. 2015-12-18 22:23:51 -08:00
Todd Gamblin
abd3d3946d Format to 80 char width. 2015-12-18 19:23:45 -08:00
Gregory Lee
9cab88071b Merge pull request #239 from LLNL/bugfix/238-db-specs-not-concrete
Fix #238: Database specs need to be concrete when read in as well.
2015-12-16 11:37:51 -08:00
Mike Nolta
82aa366adf minor comment tweak in modules.py 2015-12-16 11:43:07 -05:00
Todd Gamblin
2f9ee88408 Fix #238: Database specs need to be concrete when read in as well. 2015-12-15 17:15:37 -08:00
Todd Gamblin
6e074a196a Fix #206: need to make deps AND root concrete when read in. 2015-12-14 21:06:32 -08:00
Todd Gamblin
afbd0e77d0 Make internal hash dep sort order match external one. 2015-12-11 16:47:34 -08:00
Todd Gamblin
3dd6cbc556 Fix #217: update spec_dag test for new _cmp_key. 2015-12-11 13:07:20 -08:00
Todd Gamblin
6ee2eb21dd Fix #217: Use MUCH faster hashing, reduce number of DAG copies.
This changes the hash algorithm so that it does much less object
allocation and copying, and so that it is correct.

The old version of `_cmp_key()` would call `sorted_deps`, which would
call `flat_dependencies` to get a list of dependencies so that it
could sort them in alphabetical order.  This isn't necessary in the
`_cmp_key()`, and in fact we want more DAG structure than that to be
included in the `_cmp_key()`.

The new version constructs a tuple without copying the Spec DAG, and
the tuple contains hashes of sub-DAGs that are computed recursively
in-place.  This is way faster than the previous algorithm and reduces
the numebr of copies significantly. It is also a correct DAG hash.

Example timing and copy counts for the different hashing algorithms
we've tried:

Original (wrong) Spec hash:
```
106,170 copies
real    0m5.024s
user    0m4.949s
sys     0m0.104s
```

Spec hash using YAML `dag_hash()`:
```
3,794 copies
real    0m5.024s
user    0m4.949s
sys     0m0.104s

New no-copy, no-YAML hash:
```
3,594 copies
real    0m2.543s
user    0m2.435s
sys     0m0.104s
```

So now we have a hash that is correct AND faster.

The remaining ~3k copies happen mostly during concretization, and as
all packages are initially loaded.  I believe this is because Spack
currently has to load all packages to figure out virtual dependency
information; it could also be becasue there ar a lot of lookups of
partial specs in concretize.  I can investigate this further.
2015-12-11 12:40:27 -08:00
Todd Gamblin
7526a89463 Fix #217: Make package cache use DAG hash instead of sorted deps.
- Gets rid of last vestige of old-style specs.
- Uses new hashing for lookup
2015-12-11 02:17:17 -08:00
Todd Gamblin
537451f0b1 bugfix: doc build needed additional sys.path due to externals change. 2015-12-09 01:32:12 -08:00
Todd Gamblin
c41b9b7ddc Change github.com/scalability-llnl to github.com/llnl everywhere. 2015-12-09 01:24:15 -08:00
Todd Gamblin
7383bd393e Fixed bug #42: problem with satisfies() for virtual dependencies.
- _cross_provider_maps() had suffered some bit rot (map returned was
  ill-formed but still worked for cases with one vdep)

- ProviderIndex.satisfies() was only checking whether the result map
  was non-empty.  It should check whether all common vdeps are *in*
  the result map, as that indicates there is *some* way to satisfy
  *all* of them.  We were checking whether there was some way to
  satisfy *any one* of them, which is wrong.

- Above would cause a problem when there is more than one vdep provider.

- Added test that covers this case.

- Added `constrained()` method to Spec. Analogous to `normalized()`:
  `constrain():constrained() :: normalize():normalized()`
2015-11-27 23:06:18 -08:00
Todd Gamblin
cf3d236b9f Merge branch 'scheibelp-features/externaldeps' into develop 2015-11-23 17:50:49 -08:00
Todd Gamblin
1447fd0a12 Merge pull request #160 from SteVwonder/stevwonder/develop
Fixed -l flag for 'spack extensions'
2015-11-23 15:50:18 -08:00
Tom Scogland
5e93dd6592 modified warning to reference spack clean
clean --dist has been removed, command fixed to something that works.
2015-11-20 11:49:46 -08:00
Stephen Herbein
9c30e0210b Fixed -l flag for 'spack extensions' 2015-11-12 08:15:38 -08:00
Peter Scheibel
6d7b26d4e0 Insert lib/spack/external into sys.path. This avoids cases where the system
python install and lib/spack/external have the same library installed. This
requires modifying the names of some modules in lib/spack/external in cases
where both the system python and backported features of future python versions
(i.e. after 2.6) are used (previously distinguished by "from external import X"
and "import X").
2015-11-11 18:04:22 -08:00
Todd Gamblin
0d993947ee Fix SPACK-93, SPACK-94, GitHub #150
- `remove_prefix` was modified to remove from the DB, but the package
  may not have been added to the DB yet when `remove_prefix` is called
  from `cleanup`.

- Made `remove_prefix` a pure utility function (it just removes the prefix)

- Added `installed_db.remove()` call only after the `remove_prefix` in
  `uninstall`.
2015-11-04 16:44:33 -08:00
Todd Gamblin
339da1da3d Make architecture reflect OS *and* machine. Use Python's platform module. 2015-11-04 07:46:17 -08:00
Massimiliano Culpo
6db2a58008 Fixed typo : write_lock() -> write_transaction() 2015-10-30 07:42:58 +01:00
Todd Gamblin
a1c42c2753 Merge Greg Becker's build database support into develop. 2015-10-29 02:22:20 -07:00
Todd Gamblin
671faa4b99 Merge pull request #124 from scheibelp/features/testinstall-cmd
Features/testinstall cmd
2015-10-29 00:16:52 -07:00
Todd Gamblin
ed0c80e1cb Merge pull request #145 from markcmiller86/v-char-bugfix
generalize one of the cases to handle many more kinds of formatting o…
2015-10-29 00:02:01 -07:00
Todd Gamblin
fec6674597 Merge pull request #139 from mathstuf/use-uname-for-arch
architecture: use uname if available
2015-10-29 00:01:30 -07:00
miller86
21e9d5c5c3 generalize one of the cases to handle many more kinds of formatting options 2015-10-27 17:05:27 -07:00
Todd Gamblin
a58ae0c5d0 Build database working with simple transaction support; all tests passing. 2015-10-27 16:36:44 -07:00
Todd Gamblin
bf8479bec6 Fix stupid lock bug.
- Code simplification ignored case where exception was raised.
- If LockError was raised, read and write counts were incremented erroneously.
- updated lock test.
2015-10-27 16:34:26 -07:00
Peter Scheibel
50d0a2643b Not all package stages have an archive file (e.g. source code repos) but all of
them do have a source_path: use this instead to check whether the package
resources were successfully retrieved.
2015-10-27 13:34:46 -07:00
Todd Gamblin
af7b96c14a Lock acquires return True/False depending on whether they got POSIX lock. 2015-10-27 00:35:06 -07:00
Ben Boeckel
6c9b10f73d architecture: remove custom mac_type method 2015-10-26 18:55:10 -04:00
Ben Boeckel
17a58ee0a9 architecture: use uname if available 2015-10-26 18:54:04 -04:00
Peter Scheibel
3b554c709b Fetch errors were also terminating runs of test-install with system exit, so
stage.fetch() was updated to raise a FetchError instead of calling tty.die().
Output is the same for spack install in case of a fetch error.
2015-10-26 15:26:08 -07:00
Ben Boeckel
9d90cb6962 python: use the setdefault method on dict
It allows more concise code and skips some key lookups.
2015-10-26 17:38:19 -04:00
Peter Scheibel
9576860f8c Making SpackError reference consistent. 2015-10-26 14:27:44 -07:00
Peter Scheibel
6a16040462 Automatically create a 'test-output' directory in the current directory if no
output path is specified. Test output files are placed in this directory.
Furthermore the filenames now have the prefix "test" (but otherwise are the
string representation of the spec ID as before).
2015-10-26 11:58:52 -07:00
Todd Gamblin
ead8ac58c6 Working Lock class, now uses POSIX fcntl locks, extensive unit test.
- llnl.util.lock now uses fcntl.lockf instead of flock
  - purported to have more NFS compatibility.

- Added an extensive test case for locks.
  - tests acquiring, releasing, upgrading, timeouts, shared, & exclusive cases.
2015-10-24 19:57:43 -07:00
Todd Gamblin
908a93a470 Add a multiprocess Barrier class to use for testing parallel code. 2015-10-24 19:54:52 -07:00
Peter Scheibel
cc0ee3dc29 The HTML number conversion regex operating against a byte string will only
convert individual bytes, so therefore incorrectly converts utf-8 encoded
characters. Decoding byte strings to unicode objects results in correct HTML
number encodings.
2015-10-23 20:56:06 -07:00
Peter Scheibel
d76c923623 1. Rename CommandError -> InstallError
2. InstallError now subclasses SpackError vs. StandardError (so it is now
handled by the spack shell script)
2015-10-23 16:18:06 -07:00
Peter Scheibel
ea872f8098 1. Added CommandError exception to build_environment
2. The parent of a failed child process in build_environment.fork no longer
calls sys.exit - instead it raises a CommandError (from [1])
3. test-install command now attempts to install all packages even if one fails
2015-10-22 17:44:16 -07:00
Peter Scheibel
246423b4b4 Generate test results (designated as skipped) for parents of failed dependencies 2015-10-22 16:00:03 -07:00
Todd Gamblin
b6833f08df Merge pull request #129 from mathstuf/separate-prefix
spack: split spack_root from prefix
2015-10-21 16:04:36 -04:00
Todd Gamblin
3c788147ca Add Spack logo. 2015-10-18 19:14:40 -07:00
Todd Gamblin
7598612b86 Better info variant output. 2015-10-17 17:34:39 -07:00
Peter Scheibel
39f0f000f8 Created unit test for core logic in test-install command. 2015-10-15 22:02:14 -07:00
Peter Scheibel
6cd976d036 Better description for test-install command 2015-10-15 20:13:08 -07:00