- `_expand_virtual_packages` now gets a candidate list and will try
all the candidates.
- Good news: If the first virtual in the list conflicts with something else in
the spec, we'll keep trying until we find a good one.
- Bad news: Only looks as far as the next normalize(); can't see
conflicts further ahead than that if they're inevitable
some other virtual expansion.
- Refactor `concretize.py` to keep all the nasty spec graph stitching in
`spec.py`. This is more similar to before externals support.
- `concretize.py` now just returns a list of candidates sorted by
ABI compatibility to `_expand_virtual_packages`, and `spec.py`
handles testing the candidates.
- Refactor the way external paths are handled in `config.py` and `concretize.py`:
- previously, `spec_externals` returned spec/path pairs. Now it
returns specs with `external` set. Makes code in `concretize.py`
more natural.
- Shouldn't call .package from within things like normalize() and
concretize() beacuse spec may be inconsistent.
- Add `.package_class` property so that we can get at package metadata
without constructing a Package with a Spec.
- should be faster than `.package` was, anyway. Use where possible.
- Move `Spec.__cmp__` out of spec, into concretize as `cmp_specs`.
- `Spec.__cmp__` was never called (except explicitly) due to rich
comparison operators from `key_ordering`
- Refactor `_find_other_spec` to free function `find_spec`. Add a test
for it to make sure it works.
- Had attempted to add more functionality by assigning different
meanign None, True, and False values "keep_stage" (where False was
"always delete").
- Turns out that's not really worth the complexity. Having the third
"always delete" sense is hardly ever useful but makes the code hard
to understand.
- Fix bug introduced during merge of stage refactor.
- install prefix was not created before build_environment.fork()
- build_environment.fork() calls setup_dependent_environment
- python's setup_dependent_environment can inadvertently create
the install prefix before directory_layout expects it.
- Clean up Package.do_install:
- simplify control flow: parent process now entirely responsible for
creating/destroying the install prefix. cleanup is now in one place.
- Hoisting cleanup out of the child improves nesting of try/catch in
`real_work`.
- `real_work` renamed to `build_process`
- Allows skipping the expand step for downloads.
- Fixed stage so that it knows expansion didn't fail when there is a
no-expand URLFetchStrategy.
- Updated docs to reflect new option, and provided an example.
- package.py uses context manager more effectively.
- Stage.__init__ has easier to understand method signature now.
- keep can be used to override the default behavior either to keep
the stage ALL the time or to delete the stage ALL the time.
- This reverts commit c5d9ee8924.
- merged too soon before
- reverting and fixing bugs now.
Conflicts:
lib/spack/spack/mirror.py
lib/spack/spack/package.py
This does several things:
- Add `sbang`: a script to run scripts with long shebang lines.
- Documentation for `sbang` is in `bin/sbang`.
- Add an `sbang` hook that filters the `bin` directory after install
and modifies any scripts wtih shebangs that are too long to use
`sbang` instead.
- `sbang` is at the top level, so it should be runnable (not much we
can do if spack itself is too deep for shebang)
- `sbang`, when used as the interpreter, runs the *second* shebang
line it finds in a script.
- shoud fix issues with too long shebang paths.
Compiler detection was not getting triggered properly with some of the
new config logic. Adjust the conditions under which Spack will serach
for compilers.
- Recent external repo refactoring made `repo_for_pkg` raise an error
when the package was not known.
- Correct behavior is to return the highest precedence repo.
- Old test: did not attempt to actually fetch mirrored packages.
- New test:
1. Creates a temporary mirror,
2. Registers it with spack,
3. Fetches from it, and
4. Verifies that the fetched archive matches the original
- This test will hopefully mean that `spack mirror` is less brittle
from now on.
- simplify output, error, and input redirection
- `return_output=True` is now `output=str`
- `return_output=True` will still work for the time being but is
deprecated.
- previously you could say `return_output=True` and `output=<stream>`,
which wouldn't actually write to the stream. Now you actually can't
specify erroneous cases since there is only one parameter with
mutually exclusive options..
This is necessary for the NAG Fortran compiler, which prints its version message to STDERR instead of STDOUT. This message was previously being ignored, preventing spack from finding the compiler's version or automatically adding it to the compilers.yaml configuration file.
* develop: (77 commits)
Make libxcb compile with gcc 4.9.
disable parallel install for glib (found races)
disable cairo-trace, which is incompatible with older libiberty.h.
Fix LaunchMon on newer gcc versions.
Simplify output redirection in spack.util.executable
Allow completely empty config files.
Update cmake package to use http in all URLs.
Fix create, diy, edit, and repo commands to use multiple repos.
Add namespace option to find command.
Temporary fix: Clang is the default compiler on Mac OS X.
Netcdf requires cmake >=2.8.12
Mbedtls depends on cmake
Update libtool to 2.4.6
Update Julia to 0.4.3
Make binutils build on OS X
Add verbs and psm variants
py-matplotlib: depend on freetype
py-numpy: make blas/lapack optional dependencies
qhull: apply patch to work with libc++'s standard library
New package GNU tar
...
- By default inherit parent's input/output descriptor
- Only use pipes if we need to return output.
- Allows subprocesses (like emacsclient) to detect terminal correctly
- This moves var/spack/packages to var/spack/repos/builtin/packages.
- Packages that did not exist in the source branch, or were changed in
develop, were moved into var/spack/repos/builtin/packages as part of
the integration.
Conflicts:
lib/spack/spack/test/unit_install.py
var/spack/repos/builtin/packages/clang/package.py
- mirrors.yaml now uses dict order for precedence, instead of lists of
dicts.
- spack.cmd now specifies default scope for add/remove and for list
with `default_modify_scope` and `default_list_scope`.
- commands that only read or list default to all scopes (merged)
- commands that modify configs modify user scope (highest
precedence) by default
- These vars are used in setup_paraser for mirror/repo/compiler.
- Spack's argparse supports aliases now.
- added 'rm' alias for `spack [repo|compiler|mirror] remove`
This update significantly reworks the llvm and clang packages. The llvm
package now includes variants allowing it to build and install any and
all of:
* clang
* lldb
* llvm's libunwind (why, WHY did they name it this?!?)
* polly (including building it directly into the clang tools, 3.7.0 only)
* clang extra tools
* compiler-rt (sanitizers)
* clang lto (the gold linker plugin that allows same to work)
* libcxx/libcxxabi
* libopenmp, also setting the default openmp runtime to same, when
parameters happen this shoudl be an option of libomp or libgomp
Ideally, this should have rpath setup like the gcc package does, but
clang's driver has no support for specs as such, and no clearly
equivalent mechanism either. If anyone has ideas on this, they would be
welcome.
One significant note related to gcc though, if you test this on LLNL
systems, or anywhere that has multiple GCCs straddling the dwarf2
boundary and sharing a libstdc++, build a gcc with spack and use that to
build clang. If you use a gcc4.8+ to build this with an older
libstdc++ it will fail on missing unwind symbols because of the
discrepancy.
Resource handling has been changed slightly to move the unpacked archive
into the target rather than use symlinks, because symlinks break certain
kinds of relative paths, and orders resource staging such that nested
resources are unpacked after outer ones.
This solution doesn't really make me happy, but does seem to work. It
sorts the resources by the length of the string representing their
destination. Since any nested resource must contain another resource's
name in its path, it seems that should work, but there should be a
better way to do this.
This allows resources to be placed into subdirectory trees that may not
exist in the base package, and may depend on other resources to be
staged later.
- All of these work:
- `spack mirror add`
- `spack mirror remove`
- `spack mirror list`
- `spack mirror` subcommands (except create) now have their own
--scope argument.
- Mirror config is now stored sanely as an ordered list.
- `spack compiler` subcommands now take an optional --scope argument.
- no more `remove_from_config` in `config.py` -- `update` just
overwrites b/c it's easier to just call `get_config`, modify YAML
structures directly, and then call `update`.
- Implemented `spack compiler remove`.
- Configs are now parsed with `spack.util.spack_yaml.load/dump`
- Parser annotates returned data with `_start_mark` and `_end_mark`
properties, so that we can recover what lines/files they came from.
- Parser uses `OrderedDict` instead of `dict`. This will help
maintain some sanity when round-tripping config files.
- User and site config are now kept separately in memory.
- Merging is done on demand when client code requests the configuration.
- Allows user/site config to be updated independently of each other by commands.
- simplifies config logic (no more tracking merged files)
- Stage and fetcher were not being set up properly when fetching using
a different fetch strategy than the default one for the package.
- This is fixed but fetch/stage/mirror logic is still too complicated
and long-term needs a rethink.
- Spack will now print a warning when fetching a checksum-less tarball
from a mirror -- users should be careful to use https or local
filesystem mirrors for this.
- Move `find_versions_of_archive` from spack.package to `spack.util.web`.
- `spider` funciton now just uses the link parsing it already does to
return links. We evaluate actual links found in the scraped pages
instead of trying to reconstruct them naively.
- Add `spack url-parse` command, which you can use to show how Spack
interprets the name and version in a URL.
Versions found by wildcard URLs are different from versions found by
parse_version, etc. The wildcards are constructed more haphazardly
than the very specific URL patterns in url.py, so they can get things
wrong. e.g., for this URL:
https://software.lanl.gov/MeshTools/trac/attachment/wiki/WikiStart/mstk-2.25rc1.tgz
We miss the 'rc' and only return 2.25r as the version if we ONLY use
URL wildcards.
Future: Maybe use the regexes from url.py to scrape web pages, and
then compare them for similarity with the original URL, instead of
trying to make a structured wildcard URL pattern? This might yield
better results.
