- Tests pass with URL fetching and new scheme.
- Lots of refactoring
- Infrastructure is there for arbitrary fetch policies and more attribtues on the version() call.
- Mirrors do not currently work properly, and they get in the way of a proper git fetch
- Added FetchStrategy class to Spack
- Isolated pieces that need to be separate from Stage for git/svn/http
- Added URLFetchStrategy for curl-based fetching.
- Make spack packages RPATH *ALL* dependencies (i.e. the whole tree)
- prevents callpath link from finding wrong libelf -- always uses the one dyninst used.
# By Todd Gamblin (4) and David Beckingsale (2)
# Via Todd Gamblin
* commit 'b601fd08caf21b5fc11e6998a5ebd20a04ac57ad':
Bugfixes for csh environment modules.
Bugfixes, more consolidation of modules code.
Add csh/tcsh support for modules
Consolidate most module code into spack.modules and spack.cmd.module
Fixed up module support
Added inital module support
- One file with all the module classes (spack/modules.py)
- Has an EnvModule superclass that does most of the work and consolidates common code
- Subclasses have specializations for different module systems (TclModule, Dotkit)
- One command (spack module) for all the types of modules to use
- the one command is used by the scripts, only need to maintain in one place
- has some subcommands for different module types, but they're handled mostly generically.
- Consolidate zsh support into a single setup-env.sh script.
- spack find -p works properly (get path from spec, not package)
- directory layout and PackageDB normalize things automatically unless
they're unknown packages (need to do this for spack find -l)
- install test made robust to mock/main package conflicts
- Spec.copy() does not create superfluous nodes and preserves DAG
connections.
- Spec.normalize() doesn't create extra dependency nodes or throw out
old ones like before.
- Added better test cases for above changes.
Minor things:
- Fixed bug waiting to happen in PackageDB.get()
- instances was keyed by name, not by spec, so caching wasn't really
working at all.
- removed unused PackageDB.compute_dependents function.
- Fixed PackageDB.graph_dependencies() so that spack graph works again.
- Spec comparison is now less strict
- compares based on sorted list of dependencies but not
their structure
- Makes comparison easy when a spec is not normalized.
- This makes the dep_hash consistent for specs read in from a
directory layout. - Can now reliably read in a spec for which the
package has gone away, and still be able to delete its install.
- easy switching between git branches
- Fixed latent bug in Spec.flat_dependencies() (was including root)
- added a test for the directory layout so that this code will get
more exercise.
# By Matthew LeGendre (2) and Todd Gamblin (1)
# Via Todd Gamblin
* commit 'd7a3c7e555bfd93fbf93ec55608d7fc6aa8052f8':
Fix up Matt's openss packages.
Add sqlite to spack
Add libmonitor to spack. Still needs svn support for checkout
- Spack needed symlinks in env for fortran compilers.
- 'fc' is a bash bulitin, so can't use it for fortran compiler.
- switched to 'f90' for this.
- mpich 3 builds with fortran now.
- enabled shared libs in mpich package
- New spack.hooks package
- contains modules with pre and post install hooks
- New dotkit hook module
- generates/removes dotkits on install/uninstall
- New spack use, spack unuse commands
- use same syntax as install/uninstall
- New setup-env.bash script
- Sets up path, dotkit support
- new spack dotkit command
- used by script to parse specs, generate
specs of installed pckages for dotkit file names
- TAU doesn't install to directories with '@' in the name.
- Need to fix up its scripts.
- routines to filter files as sed would, but using python regular expressions.
- TAU package uses this.
- Mirrors now appear in ~/.spackconfig, can be edited in that file.
- spack mirror command allows adding, listing, removing mirrors
- Also still creates mirror directories.
- Fast compiler finding in path and for other directories
- first time spack runs, it searches path.
- user can add more compilers with 'spack compiler add'
- Finds intel, gcc, clang, and pgi compilers with custom version args.
- Builds can plug in alternate compilers with ease (e.g. %intel@12.1)
- no more need for compiler python files.
- Default compilers are found in user's environment and added
to ~/.spackconfig automatically
- User can add new compilers by editing configuration file
- Changed how satisfies() is defined for the various version classes
- Can't just use overlaps() with version lists -- need to account for
more and less specific versions.
If the version is more specific than the constriant (e.g., 4.7.3 is more
specific than 4.7), then it should satisfy the constraint, because if
a user asks for 4.7 they likely do not care about the minor version. If they
do, they can specify it. New Version.satisfies() takes this into account.
- spack.spec.Compiler is now spack.spec.CompilerSpec
- Can load a spack.compilers.* module for a particular spec
- e.g. load Gcc module for gcc@4.7 spec.
The number of variables in the string printed for a checksum failure
was not correct. Now a useful error message is printed instead of a
stack trace and a TypeError.
- Previously, URLs like this wouldn't work with spack create:
http://www.hdfgroup.org/ftp/HDF5/current/src/hdf5-1.8.13.tar.bz2
The '5' in hdf5 would interfere with version wildcard substitution beacuse
the wildcard regex would subsume it.
We now take the name of the package OUT of the URL before splitting it up
and adding version wildcards. This prevents names with numbers from breaking
url.wildcard_version.
Also added a package sanity check test that ensures all builtin packages
work with wildcard_version.
# By Todd Gamblin
# Via Todd Gamblin
* commit 'a0c91791c5ac995b80111a92c94d8fb760caa6e3':
SPACK-25: Add package for boost. Works on Linux currently.
Better temp directory selection.
Status messages now show install and stage directories.
Better version wildcard handling, better spidering
- checksum --dirty and create --dirty now changed to --keep-stage
- install --dirty is now --keep-prefix
- uninstall --force now works properly
- commands use keyword args instead of package instance vars
where possible (less weird package state)
- except clause now catches anything, not just exception
- sys.exit() changed to os._exit() to avoid interfering with unit tests
and to avoid raising SystemExit exception when child processes quit.
- Allow version wildcards to match [_-.] instead of the exact separators
the version was constructed with.
- Handles the fact that boost versions are written both 1.55.0 and 1_55_0.
- Update spidering to handle parse errors and warn that Python < 2.7.3 has
less robust HTML parsing abilities.
- this allows each install to have full control over its environment,
and over spack.
- build process can do whatever it wants and doesn't affect main Spack
process.
- Consolidated build environment stuff from package.py into
build_environment.py
- package.py calls build_environment.py functions just before calling
install(), in do_install()
- Organization is better b/c SPACK_xxxx environment variables are now
in build_environment, the only place they're used. Were previously
cluttering globals.py.
The dependency hash wasn't being generated in a consistent way: when
read from .spec file in the install directory, the spec DAG was not
normalized, but when generated on install, it was. Thus the same
spec can have two possible hashes, which is obviously not good.
In this commit:
- Added a dep_hash function to spec, which should be used to get the
dependency hash in all cases. This function normalizes the spec
before generating the hash.
- removed sha1 functions from Spec and DependencyHash. The one on
Spec is not needed and is confusing (it hashed the whole spec, not
just hash the deps) and the one on DependencyHash wasn't guaranteed
to be safe because it can't normalize its parent spec before
hashing.
Hopefully this makes things consistent and easy.
- Packages now live in <package_name>/package.py
- spack.packages refactored to use a PackageDB object instead of
monolithic module.
- Implementation of mock_packages_test.py is greatly simplified
- Added test to exercise install/uninstall code because that wasn't
covered by existing tests and kept breaking.
- less kludgy matching -- we nw extract the version straight from a
capture group in the original match, rather than by trying to match
again.
- Avoids problems like finding "2" as the version in mvapich2-1.9, when
"2" here is part of the package name.
- packages can provide patch() directive to specify a patch file that
should be applied to source code after expanding it and before
building.
- patches can have a when spec, so they're only applied under certain
conditions
- patches can be local files in the package's own directory, or they
can be URLs which will be fetched from the internet.
- Formerly required that one and only one spec match
- Now allows first match in a list (more flexible and more intuitive)
- introduces new bug that provides() doesn't do the correct thing
when a version is not in a range that has been explicitly provided.
- TODO: fix this.
- anonymous specs have no name
- "local spec" came from these first being used in multimethods, i.e. "name of the local Package"
- not the most intuitive name.
- Normalize now updates the provider index as it addes package dependencies.
- Fixes problem where this breaks:
a depends_on mpi
a depends_on b
b depends_on mpich
- Packages now restrict the mpi dependency to mpich
- Added checks to constrain() so that it is consistent with satisfies()
- Added many more test cases for satisfiability and constraints on deps
- Virtual packages are handled properly in satisfies() and constrain()
- bugfix: mpileaks^mpich2 would satisfy mpileaks^mpi@3:
- this case is now handled.
Added more test cases for multimethods. In doing so, (re)discovered that
satisfies() really needs to handle dependencies properly.
Implemented support for dependencies in satisfies, but constrain() now
isn't consistent (as we do not currently constrain deps), so need to
implement that.
Virtual dependency support probably needs some deeper thought. i.e.,
there is probably an intermediate DAG form that would make the needed
checks easier. Right now we have to build ProviderIndexes to figure out
how virtual dependencies are set up. If the vdep were preserved in the DAG,
then we could just check for things like incompatible providers directly.
All of these do the same thing. So they are all now generalized
to a single closure function; just the name of the updated variable
in the package is different.
'spack test -a' formerly ran all test and 'spack test' listed available tests.
Now, 'spack test' runs all tests and 'spack test -l' lists available ones.
New operations for manipulating spec and package DAGs.
For specs:
flatten: gather all deps to the root
normalize: Merge constraints and make spec match package DAG
For packages:
validate_dependencies: Make sure spec constraints in package DAG are sane.
Added tests for above methods. Also added beginnings of concretization logic,
to turn abstract spec into a concrete one.
Still need proper tests for normalize().
- Specs can be "constrained" by other specs, throw exceptions when constraint
can't be satisfied.
- Normalize will put a spec in DAG form and merge all package constraints with
the spec.
- Ready to add concretization policies for abstract specs now.
Experimental feature automatically parses versions out of web pages and prints what
it thinks are avaialble versions of a package, e.g.:
$ spack list -v libunwind
1.1 1.0 0.98.6 0.98.4 0.98.2 0.98 0.96 0.93 0.91 0.1
1.0.1 0.99 0.98.5 0.98.3 0.98.1 0.97 0.95 0.92 0.9 0.0
spec.py can parse full dependence specs like this:
openmpi@1.4.3:1.4.5%intel+debug ^hwloc@1.2
These will be used to specify how to install packages and their dependencies,
as well as to specify restrictions (e.g., on particular versions) for
dependencies. e.g.:
class SomePackage(Package):
depends_on('boost@1.46,1.49')
This would require either of those two versions of boost.
This also moves depends_on out to relations.py and adds
"provides", which will allow packages to provide virtual
dependences.
This is just initial implementation of the parsing and objects
to represent specs. They're not integrated with packages yet.
You can now do this:
class MyPackage(Package):
def install(self):
...default install...
@platform('bgqos_0')
def install(self):
...specialized install for bgq...
This works on functions other than install, as well (as long as they're in a Package)
