Commit graph

570 commits

Author SHA1 Message Date
Todd Gamblin
a58ae0c5d0 Build database working with simple transaction support; all tests passing. 2015-10-27 16:36:44 -07:00
Todd Gamblin
bf8479bec6 Fix stupid lock bug.
- Code simplification ignored case where exception was raised.
- If LockError was raised, read and write counts were incremented erroneously.
- updated lock test.
2015-10-27 16:34:26 -07:00
Todd Gamblin
af7b96c14a Lock acquires return True/False depending on whether they got POSIX lock. 2015-10-27 00:35:06 -07:00
Todd Gamblin
ead8ac58c6 Working Lock class, now uses POSIX fcntl locks, extensive unit test.
- llnl.util.lock now uses fcntl.lockf instead of flock
  - purported to have more NFS compatibility.

- Added an extensive test case for locks.
  - tests acquiring, releasing, upgrading, timeouts, shared, & exclusive cases.
2015-10-24 19:57:43 -07:00
Todd Gamblin
908a93a470 Add a multiprocess Barrier class to use for testing parallel code. 2015-10-24 19:54:52 -07:00
Gregory Becker
5fda7daf57 an ordered database test 2015-09-27 17:54:24 -07:00
Todd Gamblin
d0e22b2240 Add ref counting to database. This does not handle removal properly yet. 2015-09-18 11:40:05 -07:00
Todd Gamblin
fb73979345 Allow custom timeout for database locking. 2015-09-17 16:09:59 -07:00
Todd Gamblin
e17ad6a684 Simplify lock context managers. 2015-09-17 01:05:19 -07:00
Todd Gamblin
ccf311c9c6 Several changes to DB implementation.
1. Database stores a file version, so we can add to it in the future.
2. Database indexed by hashes and not numerical indexes.
3. Specs built by database have consistent hashes and it's checked.
4. minor naming and whitespace changes.
2015-09-17 00:16:12 -07:00
Gregory Becker
cd23d2eaa2 Added spack fsck and re-read from glob if the database file does not exist. Allows older versions to smoothly upgrade to the database. 2015-09-15 14:20:19 -07:00
Gregory Becker
9c8e46dc22 Added conservative locking to the spack commands that access the database at _index 2015-09-03 09:21:19 -07:00
Gregory Becker
c3246ee8ba Removed incorrect stage removal code from cleanup() in do_install() 2015-08-31 09:46:55 -07:00
Gregory Becker
f406fcb843 Fixed several issues from code review
Most importantly wrote the Lock, Read_Lock_Instance, and Write_Lock_Instance classes in lock.py
Updated the locking in database.py
TODO: Lock on larger areas
2015-08-31 09:38:38 -07:00
Gregory Becker
9345e78779 Fixed inaccurate comment in spec.py 2015-08-25 16:31:09 -07:00
Gregory Becker
ce8df65d7b Eliminated unnecessary differences in pull request 2015-08-25 16:28:55 -07:00
Gregory Becker
e32c59f805 Fixed file locking. Fix is slightly ugly (lock integer added) but it gets the job done
It avoids having to spin simply on the OSError.
2015-08-25 15:32:45 -07:00
Gregory Becker
4a2bd1753a Added dependency indices to database, ensuring correctly reconstructed specs from database
Began work on file locking, currently commented out.
2015-08-25 15:11:18 -07:00
Gregory Becker
fb1874165b Eliminated all calls that relied on finding all packages in the opt directory
Replaced them all with references to the database

Implemented caching in the database. The database now only re-reads data
if the database file exists and was changed since this file last wrote to it.

Added the installed_db field to the spack instance

Left the call to all_specs from testdirectory_layout.py for now.
2015-08-21 17:00:39 -07:00
Gregory Becker
55f68bb2b0 Added hashes to the database 2015-08-21 13:04:27 -07:00
Gregory Becker
1da56e5290 Added a database of installed packages.
No methods use the database so far.

Also, a bug fix:
      Previous version did not remove the staging directory on a failed install
      This led to spack refusing to uninstall dependencies of the failed install
      Added to cleanup() to blow away the staging directory on failed install.
2015-08-21 11:32:12 -07:00
Todd Gamblin
6f75f5bd6a Merge pull request #88 from trws/git_depth_attempt
adding a fallback on failure with git --depth
2015-08-12 23:59:13 -07:00
Todd Gamblin
1792115b4d Use absolute paths to spack compiler wrappers in CC/CXX/F77/FC 2015-08-04 09:57:06 -07:00
Tom Scogland
9174c06598 adding a fallback on failure with git --depth
Certain remote protocols don't support the `--depth` option.  Since this can't
be checked by URL type or in any sane way locally, this version attempts to
clone git repositories with the --depth option, and if that fails attempts the
clone again without it.
2015-07-27 14:03:04 -07:00
Todd Gamblin
27ca697b43 Add Python version test to detect {} in version strings.
- {} is not compatible with Python 2.6
2015-07-24 14:22:28 -07:00
Todd Gamblin
1e2f421faa Fix Python 2.6 compatibility issue. 2015-07-23 17:01:55 -07:00
Todd Gamblin
19123b4c48 Fix ProviderIndex.update(), which didn't remove stale providers. 2015-07-16 01:41:11 -07:00
Todd Gamblin
e097696390 Update concretize to check for more changes and iterate further. 2015-07-16 01:12:11 -07:00
Todd Gamblin
9087f26537 Fix mangled error message. 2015-07-14 23:59:03 -07:00
Todd Gamblin
0d308ca30f Fix regression of install -j. 2015-07-14 23:45:58 -07:00
Todd Gamblin
9d1f8f6bfd Add test to check packages for Python2.6 compatibility, as well as core. 2015-07-13 18:37:48 +00:00
Florent Pruvost
5acd7cad9a add env. var. in modules: LIBRARY_PATH and PKG_CONFIG_PATH 2015-07-02 02:12:10 -07:00
Todd Gamblin
6892d7189f Merge branch 'features/toggle-build-output' into develop 2015-07-02 00:51:03 -07:00
Todd Gamblin
b3e34be972 Better python template for 'spack create' 2015-07-02 00:39:33 -07:00
Todd Gamblin
edfcac32c3 Fix bug in uninstall (regression in 0fc3b58) 2015-06-20 21:46:52 -07:00
Todd Gamblin
6ed5dd7427 log_output now allows echo to be toggled wtih 'v' key 2015-06-20 20:20:28 -07:00
Todd Gamblin
cfb883646f Fix for git pull --tags when using git 1.7.1
- Added `ignore_errors` option to `Executable.__call__`
- Can avoid raising errors on *specific* error return values.
2015-06-13 17:52:00 -07:00
Tom Scogland
0b5ca25358 Fix for repos with many tags
Ensures all tags are ready before checkout, using `--branch` if possible and
an extra pull if that is not available.  Also adds `--depth 1` to create
shallow clones if the git version is sufficient.

Fixes #64.
2015-06-13 15:29:55 -07:00
Tom Scogland
277df08676 removing vestigial lines from a previous commit 2015-06-13 14:10:45 -07:00
Tom Scogland
d92ac2a6b2 Small ruby enhancement and tmuxinator package
It is currently less painful to pull the source from github, compile it into a
gem, then install the gem, than it is to download a gem and install it.  This
still lacks an activation mechanism, but `spack use tmuxinator` is functional.
2015-06-11 13:51:53 -07:00
Todd Gamblin
0a92349f90 Try a little harder in concretize_version() -- concretize unsafe versions too.
- This can result in the user being prompted to download an unsafe
  version.

- Avoids overly strict errors when something *could* be satisfiable
  but we don't know about hte version.
2015-06-07 15:40:01 -07:00
Todd Gamblin
3f3ceb24c4 Add some comments b/c I didn't understand my own test. 2015-06-07 15:39:40 -07:00
Todd Gamblin
0fc3b58890 SPACK-38: Allow specs to be indexed by virtual dependencies.
- The following now work differently:

      spec['mpi']
      spec['blas']

  This can return a spec for openmpi, mpich, mvapich, etc., EVEN if
  the spec is already concretized.  This means that in a package that
  `depends_on('mpi')`, you can do `spec['mpi']` to see what it was
  concretized to.  This should simplify MPI and BLAS packages.

      'mpi' in spec
      'blas' in spec

  Previously, if the spec had been concretized, these would be `False`
  because there was not a dependency in the DAG with either of these
  names. These will now be `True` even if the spec has been
  concretized.  So, e.g., this will print "YES"

      s = Spec('callpath ^mpich')
      if 'mpi' in spec:
          print "YES"

- Similarly, this will be True:

      Spec('mpich').satisfies('mpi')

- Because of the way virtual dependencies are currently implemented,
  the above required some fiddling around with `package.py` so that it
  would never call `Spec.__contains__` (and result in endless
  recursion).

- This should be fixed by allowing virutal dependnecies to have their
  own package class.
  - This would allow a quicker check for vdeps, without a call to
    `all_packages`.
  - For the time being, `package.py` shouldn't call `__contains__`
2015-06-07 15:36:53 -07:00
Todd Gamblin
eb9859ce75 Executables now have a useful __str__ function. 2015-06-06 16:15:52 -07:00
Todd Gamblin
53feb12ea0 Cleanup and consolidate error handling 2015-06-06 15:50:01 -07:00
Todd Gamblin
5d7a6c0c46 Add FileFilter convenience class to llnl.util.filesystem. 2015-06-06 15:26:48 -07:00
Saravan Pantham
454d06c25c Removed autosave files from previous commit. 2015-06-06 15:26:37 -07:00
Saravan Pantham
c174fe6199 Fixed BLAS and Lapack installations 2015-06-06 15:26:31 -07:00
Todd Gamblin
d19538af07 Fix #48: Ignore hidden files when deciding how to stage.
- Expanding archvies like MAGMA 1.6.2 creates extra hidden files that
  confuse Spack's staging mechanism.

- Added a special case to ignore hidden files when checking whether
  the tarball exploded.
2015-06-01 10:25:58 -07:00
Todd Gamblin
123778dec2 Add -S/--stages option to spack cd/spack location.
- can now change dir to top-level stage
2015-06-01 09:56:35 -07:00