Commit graph

1157 commits

Author SHA1 Message Date
Todd Gamblin
afbd0e77d0 Make internal hash dep sort order match external one. 2015-12-11 16:47:34 -08:00
Todd Gamblin
3dd6cbc556 Fix #217: update spec_dag test for new _cmp_key. 2015-12-11 13:07:20 -08:00
Todd Gamblin
6ee2eb21dd Fix #217: Use MUCH faster hashing, reduce number of DAG copies.
This changes the hash algorithm so that it does much less object
allocation and copying, and so that it is correct.

The old version of `_cmp_key()` would call `sorted_deps`, which would
call `flat_dependencies` to get a list of dependencies so that it
could sort them in alphabetical order.  This isn't necessary in the
`_cmp_key()`, and in fact we want more DAG structure than that to be
included in the `_cmp_key()`.

The new version constructs a tuple without copying the Spec DAG, and
the tuple contains hashes of sub-DAGs that are computed recursively
in-place.  This is way faster than the previous algorithm and reduces
the numebr of copies significantly. It is also a correct DAG hash.

Example timing and copy counts for the different hashing algorithms
we've tried:

Original (wrong) Spec hash:
```
106,170 copies
real    0m5.024s
user    0m4.949s
sys     0m0.104s
```

Spec hash using YAML `dag_hash()`:
```
3,794 copies
real    0m5.024s
user    0m4.949s
sys     0m0.104s

New no-copy, no-YAML hash:
```
3,594 copies
real    0m2.543s
user    0m2.435s
sys     0m0.104s
```

So now we have a hash that is correct AND faster.

The remaining ~3k copies happen mostly during concretization, and as
all packages are initially loaded.  I believe this is because Spack
currently has to load all packages to figure out virtual dependency
information; it could also be becasue there ar a lot of lookups of
partial specs in concretize.  I can investigate this further.
2015-12-11 12:40:27 -08:00
Todd Gamblin
7526a89463 Fix #217: Make package cache use DAG hash instead of sorted deps.
- Gets rid of last vestige of old-style specs.
- Uses new hashing for lookup
2015-12-11 02:17:17 -08:00
Gregory L. Lee
3163d016db install python files to libxml2 prefix instead of python prefix and ignore non-python files when activating 2015-12-09 14:10:05 -08:00
Todd Gamblin
f6eb000381 Merge pull request #225 from epfl-scitas/packages/git
Adding https support in git even if system lib are missing
2015-12-09 10:59:47 -08:00
Nicolas Richart
31863c68d8 Adding curl and expat support to git for https repos 2015-12-09 11:00:33 +01:00
Todd Gamblin
e46af913be Merge pull request #148 from PaulDapolito/glib-dep
added zlib dependency to glib package specification
2015-12-09 01:44:00 -08:00
Todd Gamblin
8ebac18f50 Merge pull request #224 from epfl-scitas/packages/gdb
gdb : added package
2015-12-09 01:35:04 -08:00
Todd Gamblin
537451f0b1 bugfix: doc build needed additional sys.path due to externals change. 2015-12-09 01:32:12 -08:00
Todd Gamblin
e009a910cc Fix travis badge URL. 2015-12-09 01:28:54 -08:00
Todd Gamblin
c41b9b7ddc Change github.com/scalability-llnl to github.com/llnl everywhere. 2015-12-09 01:24:15 -08:00
Todd Gamblin
b8bb24d4bf Update README.md and LICENSE with new github.com/llnl URLs 2015-12-09 01:10:14 -08:00
alalazo
4d5eb4ce7d gdb : added package 2015-12-09 09:06:10 +01:00
Todd Gamblin
0356ba34a0 Merge pull request #223 from alalazo/fixes/gcc_sysroot_and_objdir
Fix for #177
2015-12-08 10:15:40 -08:00
alalazo
c5b9732aed Should fix #177
gcc : changed objdir to something out of srcdir and added '--with-sysroot=/'
binutils : added '--with-sysroot=/'
2015-12-08 14:06:04 +01:00
Todd Gamblin
e7ab8d476d Merge pull request #219 from eschnett/eschnett/gmp
Update gmp to 6.1.0
2015-12-07 16:46:44 -08:00
Todd Gamblin
6bc86a6baa Merge pull request #220 from eschnett/eschnett/mpc
Update mpc to 1.0.3
2015-12-07 16:46:32 -08:00
Todd Gamblin
b0434fa486 Merge pull request #221 from davidbeckingsale/features/cleverleaf-boost
Update CleverLeaf package to use Boost and fix SAMRAI MPI usage
2015-12-07 16:46:15 -08:00
Todd Gamblin
fcabf0ef0d Merge pull request #222 from mplegendre/bugfix/github-203-gcc5-builds
Fix issue with gcc 5.x not building on RHEL6
2015-12-07 15:55:21 -08:00
Matthew LeGendre
f8e046bb33 Fix issue with gcc 5.x not building on RHEL6 2015-12-07 15:49:47 -08:00
Gregory L. Lee
1a22a507d6 mesa requires +python variant of libxml2 2015-12-07 15:36:36 -08:00
David Beckingsale
36272c8136 Fix MPI selection in SAMRAI and remove narrow dependency specification 2015-12-07 14:58:16 -08:00
David Beckingsale
77bf133cd2 Update CleverLeaf package to use Boost and fix SAMRAI MPI usage 2015-12-07 14:33:31 -08:00
Erik Schnetter
aff95ee587 Update mpc to 1.0.3 2015-12-05 11:23:40 -05:00
Erik Schnetter
29537b9991 Update gmp to 6.1.0 2015-12-05 11:21:47 -05:00
Gregory L. Lee
5891847ae4 added py-pillow and updated py-basemap to depend on it 2015-12-03 15:35:28 -08:00
Gregory L. Lee
a56c7e5f1a Merge branch 'develop' of https://github.com/scalability-llnl/spack into develop 2015-12-03 15:02:49 -08:00
Gregory L. Lee
0f4efc018d updated package versions 2015-12-03 15:01:44 -08:00
Todd Gamblin
5a5990be09 Merge pull request #214 from epfl-scitas/packages/boost
Boost: mpi, python and compression for iostream variants
2015-12-03 09:24:21 -06:00
Nicolas Richart
1a132e4fdb Missing declaration of the debug variant 2015-12-02 16:18:29 +01:00
Nicolas Richart
4bb6f23ae1 Adding variant for mpi and python + compression for iostream 2015-12-02 13:26:00 +01:00
Todd Gamblin
114e4d3fe6 Merge pull request #213 from khuck/develop
Had to force a patch to the OMPT support, so the hash for v0.1 has changed.
2015-11-30 21:42:49 -08:00
Kevin Huck
cd1207329b Updating the OMPT hash again... 2015-11-30 10:32:07 -08:00
Kevin Huck
d5c7cd5e50 Had to force a patch to the OMPT support, so the hash for v0.1 has
changed.
2015-11-30 09:48:15 -08:00
Todd Gamblin
55237b094a Use new travis insfrastructure (sudo:false) 2015-11-29 22:09:11 -08:00
Todd Gamblin
45ffcdee7b Add very basic Travis CI support. 2015-11-29 22:02:13 -08:00
Tom Scogland
b253308eee returning flux to a building state for web release
Flux package reworked to include all new dependencies, fixed issues with a
clean-build of hwloc, lua and czmq as well that prevented flux from building
cold on a minimal system.
2015-11-29 19:22:31 -08:00
Todd Gamblin
05b791a621 Add citation information to README 2015-11-28 19:45:36 -08:00
Todd Gamblin
7383bd393e Fixed bug #42: problem with satisfies() for virtual dependencies.
- _cross_provider_maps() had suffered some bit rot (map returned was
  ill-formed but still worked for cases with one vdep)

- ProviderIndex.satisfies() was only checking whether the result map
  was non-empty.  It should check whether all common vdeps are *in*
  the result map, as that indicates there is *some* way to satisfy
  *all* of them.  We were checking whether there was some way to
  satisfy *any one* of them, which is wrong.

- Above would cause a problem when there is more than one vdep provider.

- Added test that covers this case.

- Added `constrained()` method to Spec. Analogous to `normalized()`:
  `constrain():constrained() :: normalize():normalized()`
2015-11-27 23:06:18 -08:00
Tom Scogland
f8ffb005c8 make cram an extension of python 2015-11-25 15:10:18 -08:00
Todd Gamblin
c526c71f56 Merge pull request #201 from alfredo-gimenez/develop
Fixed Mitos dependency and added new version
2015-11-24 16:12:12 -08:00
Alfredo Gimenez
a0d388875e Fixed Mitos dependency and added new version 2015-11-24 15:12:31 -08:00
Tom Scogland
a1553dd12b adding google sparsehash and cityhash libraries 2015-11-24 09:23:36 -08:00
Todd Gamblin
e2412e3f21 Merge branch 'eschnett-openblas' into develop 2015-11-24 00:52:48 -08:00
Todd Gamblin
58d497ab7a Merge pull request #200 from scalability-llnl/revert-189-openssl-use-64-bits
Revert "WIP: openssl: set KERNEL_BITS to make ./config work"
2015-11-24 00:30:37 -08:00
Todd Gamblin
52cce2d710 Revert "WIP: openssl: set KERNEL_BITS to make ./config work" 2015-11-24 00:30:08 -08:00
Todd Gamblin
cf3d236b9f Merge branch 'scheibelp-features/externaldeps' into develop 2015-11-23 17:50:49 -08:00
Todd Gamblin
894fcd90e7 Add a fix/warning so that stale .pyc files don't kill Spack.
- Can't think of a better way to do this.
- The externals integration will cause spack to die in weird ways for
  users who just pull from develop.
2015-11-23 17:49:39 -08:00
Erik Schnetter
1d90d98e40 Remove commented-out code 2015-11-23 20:32:06 -05:00