Commit graph

723 commits

Author SHA1 Message Date
Ben Boeckel
6c9800bcc3 typos: fix some letter transposes 2016-01-19 15:22:16 -05:00
Todd Gamblin
cee7bfa9f0 Simplify output redirection in spack.util.executable
- By default inherit parent's input/output descriptor
- Only use pipes if we need to return output.
- Allows subprocesses (like emacsclient) to detect terminal correctly
2016-01-19 01:16:14 -08:00
Todd Gamblin
ad32f64ef6 Allow completely empty config files.
- Previous version would give validation error for an empty file.
  Now this is properly ignored.

- Also includes bugfix in ConfigFormatError
2016-01-19 01:16:14 -08:00
Todd Gamblin
8d6342c53d Merge branch 'mplegendre-multi_pkgsrc_roots' into develop
- This moves var/spack/packages to var/spack/repos/builtin/packages.

- Packages that did not exist in the source branch, or were changed in
  develop, were moved into var/spack/repos/builtin/packages as part of
  the integration.

Conflicts:
	lib/spack/spack/test/unit_install.py
	var/spack/repos/builtin/packages/clang/package.py
2016-01-19 01:16:08 -08:00
Todd Gamblin
c1e57e49ae Merge pull request #202 from scheibelp/features/unittest-xml-output-nose
Features/unittest xml output nose
2016-01-17 19:14:49 -08:00
Todd Gamblin
97b492756a Fix create, diy, edit, and repo commands to use multiple repos. 2016-01-17 18:14:35 -08:00
Todd Gamblin
5984bc2ad3 Add namespace option to find command. 2016-01-17 18:12:53 -08:00
Todd Gamblin
358b2ab4ba Temporary fix: Clang is the default compiler on Mac OS X. 2016-01-17 18:12:08 -08:00
Todd Gamblin
aa00999fca update compiler config test. 2016-01-14 12:00:56 -08:00
Todd Gamblin
d869d8e86f add sanity check on config write as well as validation on read. 2016-01-14 12:00:44 -08:00
Todd Gamblin
b567cb57e1 Add jsonschema validation for config files. 2016-01-14 10:41:35 -08:00
Todd Gamblin
05b30bf83e Make text wrapping off by default in tty, add a kwarg for it. 2016-01-14 10:26:31 -08:00
Todd Gamblin
b02faf5641 add/remove/list working for new config format.
- mirrors.yaml now uses dict order for precedence, instead of lists of
  dicts.

- spack.cmd now specifies default scope for add/remove and for list
  with `default_modify_scope` and `default_list_scope`.
  - commands that only read or list default to all scopes (merged)
  - commands that modify configs modify user scope (highest
    precedence) by default
  - These vars are used in setup_paraser for mirror/repo/compiler.

- Spack's argparse supports aliases now.
  - added 'rm' alias for `spack [repo|compiler|mirror] remove`
2016-01-03 02:27:50 -08:00
Todd Gamblin
21fae634a5 Add Python 3 aliases to our argparse backport. 2016-01-03 01:19:03 -08:00
Todd Gamblin
56937416a9 All tests that call concretize() need to be MockPackagesTests. 2016-01-02 18:18:00 -08:00
Todd Gamblin
20b7f8a8e0 Fix bug in tests. 2016-01-02 14:25:10 -08:00
Tom Scogland
4ae98f8b21 significant llvm update
This update significantly reworks the llvm and clang packages.  The llvm
package now includes variants allowing it to build and install any and
all of:

* clang
* lldb
* llvm's libunwind (why, WHY did they name it this?!?)
* polly (including building it directly into the clang tools, 3.7.0 only)
* clang extra tools
* compiler-rt (sanitizers)
* clang lto (the gold linker plugin that allows same to work)
* libcxx/libcxxabi
* libopenmp, also setting the default openmp runtime to same, when
  parameters happen this shoudl be an option of libomp or libgomp

Ideally, this should have rpath setup like the gcc package does, but
clang's driver has no support for specs as such, and no clearly
equivalent mechanism either.  If anyone has ideas on this, they would be
welcome.

One significant note related to gcc though, if you test this on LLNL
systems, or anywhere that has multiple GCCs straddling the dwarf2
boundary and sharing a libstdc++, build a gcc with spack and use that to
build clang.  If you use a gcc4.8+  to build this with an older
libstdc++ it will fail on missing unwind symbols because of the
discrepancy.

Resource handling has been changed slightly to move the unpacked archive
into the target rather than use symlinks, because symlinks break certain
kinds of relative paths, and orders resource staging such that nested
resources are unpacked after outer ones.
2015-12-30 16:59:39 -08:00
Tom Scogland
fcdf08e4d7 allow nested resources
This solution doesn't really make me happy, but does seem to work.  It
sorts the resources by the length of the string representing their
destination.  Since any nested resource must contain another resource's
name in its path, it seems that should work, but there should be a
better way to do this.
2015-12-30 16:59:39 -08:00
Tom Scogland
cc8c783bb2 adding errno import for the deep directory creation patch 2015-12-30 16:59:39 -08:00
Tom Scogland
30f5ccb80d create leading directories for resources
This allows resources to be placed into subdirectory trees that may not
exist in the base package, and may depend on other resources to be
staged later.
2015-12-30 16:59:39 -08:00
Todd Gamblin
e8e6368cc8 Rework mirror configuration.
- All of these work:
  - `spack mirror add`
  - `spack mirror remove`
  - `spack mirror list`

- `spack mirror` subcommands (except create) now have their own
  --scope argument.

- Mirror config is now stored sanely as an ordered list.
2015-12-28 01:14:41 -08:00
Todd Gamblin
1f8ba53ca7 Rework compiler configuration and simplify config.py logic.
- `spack compiler` subcommands now take an optional --scope argument.

- no more `remove_from_config` in `config.py` -- `update` just
  overwrites b/c it's easier to just call `get_config`, modify YAML
  structures directly, and then call `update`.

- Implemented `spack compiler remove`.
2015-12-28 00:46:51 -08:00
Todd Gamblin
ff0d871612 Remove mock_configs; tests no longer modify spack home directory. 2015-12-27 21:13:18 -08:00
Todd Gamblin
39e360f93a Add custom YAML loader & dumper to track lines & maintain dict order.
- Configs are now parsed with `spack.util.spack_yaml.load/dump`

- Parser annotates returned data with `_start_mark` and `_end_mark`
  properties, so that we can recover what lines/files they came from.

- Parser uses `OrderedDict` instead of `dict`.  This will help
  maintain some sanity when round-tripping config files.
2015-12-27 17:51:11 -08:00
Todd Gamblin
4e8e298eb3 Uncomment disabled tests. 2015-12-27 17:40:31 -08:00
Todd Gamblin
487ac9cd19 Merge pull request #277 from LLNL/bugfix/vcs-mirror-fetch
Fix #85 and #228: errors fetching VCS packages from a mirror.
2015-12-26 09:28:38 -08:00
Todd Gamblin
bef52570ae Default to scope with highest precedence instead of user scope,
- Generalizes config scopes a bit more: nothing assumes there is a
  'user' scope (this would break testing sometimes).
2015-12-25 18:42:06 -08:00
Todd Gamblin
c65fd3a289 Merge branch 'develop' into mplegendre-multi_pkgsrc_roots
Conflicts:
	lib/spack/spack/cmd/create.py
	lib/spack/spack/cmd/extensions.py
	lib/spack/spack/cmd/fetch.py
	lib/spack/spack/cmd/uninstall.py
	lib/spack/spack/config.py
	lib/spack/spack/database.py
	lib/spack/spack/directory_layout.py
	lib/spack/spack/packages.py
	lib/spack/spack/spec.py
2015-12-25 16:35:55 -08:00
Todd Gamblin
34401cf0c3 Rework Spack config: keep user & site config in memory.
- User and site config are now kept separately in memory.
- Merging is done on demand when client code requests the configuration.
- Allows user/site config to be updated independently of each other by commands.
- simplifies config logic (no more tracking merged files)
2015-12-25 14:00:33 -08:00
Todd Gamblin
52625de52c Fix #154 -- better log messages for do_patch() 2015-12-24 11:37:40 -08:00
Todd Gamblin
f37f872e5f Fix #85 and #228: errors fetching VCS packages from a mirror.
- Stage and fetcher were not being set up properly when fetching using
  a different fetch strategy than the default one for the package.

- This is fixed but fetch/stage/mirror logic is still too complicated
  and long-term needs a rethink.

- Spack will now print a warning when fetching a checksum-less tarball
  from a mirror -- users should be careful to use https or local
  filesystem mirrors for this.
2015-12-23 16:23:58 -08:00
Todd Gamblin
5ca5884ad6 Fix #235: Smarter web spidering -- use parsed links instead of reconstructing.
- Move `find_versions_of_archive` from spack.package to `spack.util.web`.

- `spider` funciton now just uses the link parsing it already does to
  return links.  We evaluate actual links found in the scraped pages
  instead of trying to reconstruct them naively.

- Add `spack url-parse` command, which you can use to show how Spack
  interprets the name and version in a URL.
2015-12-23 13:16:57 -08:00
Todd Gamblin
d63cb8b537 Fix bug in URL regex. 2015-12-22 17:05:23 -08:00
Todd Gamblin
0d4b77a24b revert accidental print statements. 2015-12-22 17:02:14 -08:00
Todd Gamblin
2b89d9b1db More consistent URL parsing when finding versions.
Versions found by wildcard URLs are different from versions found by
parse_version, etc.  The wildcards are constructed more haphazardly
than the very specific URL patterns in url.py, so they can get things
wrong.  e.g., for this URL:

    https://software.lanl.gov/MeshTools/trac/attachment/wiki/WikiStart/mstk-2.25rc1.tgz

We miss the 'rc' and only return 2.25r as the version if we ONLY use
URL wildcards.

Future: Maybe use the regexes from url.py to scrape web pages, and
then compare them for similarity with the original URL, instead of
trying to make a structured wildcard URL pattern?  This might yield
better results.
2015-12-22 16:59:00 -08:00
Todd Gamblin
b1d40efd99 Merge pull request #261 from LLNL/features/prefer-versions-in-packages
Add ability to prefer particular versions in packages.
2015-12-22 08:25:52 -08:00
Todd Gamblin
157e42de22 Merge pull request #264 from nolta/typos
fix a few comment typos
2015-12-22 07:56:54 -08:00
Todd Gamblin
792b033255 Slightly more robust tmp directory search.
- remove getcwd() check (seems arbitrary -- if users set their TMPDIR
  to this why stop them?)

- try a number of common locations and try per-user directories in
  them first.
2015-12-22 07:54:25 -08:00
Mike Nolta
61f07b0c4d fix a few comment typos 2015-12-21 19:24:16 -05:00
Mike Nolta
2ba6bb21fb fix #249 (tmp spack-stage directories conflict) 2015-12-21 18:50:58 -05:00
Todd Gamblin
fe0fdf60b4 Add ability to prefer particular versions in packages.
- Adding `preferred=True` to a version directive will change its sort
  order in concretization.

- This provides us a rudimentary ability to keep the Spack stack
  stable as new versions are added.

- Having multiple stacks will come next, but this at least allows us
  to specify default versions of things instead of always taking the
  newest.
2015-12-21 15:35:47 -08:00
Todd Gamblin
73ea15db8e Merge pull request #208 from epfl-scitas/features/resource_directive
resource directive : implementation + clang / llvm use case
2015-12-21 10:34:16 -08:00
Todd Gamblin
dccfcd10f7 Merge pull request #255 from LLNL/bugfix/254-libtool-compiler-name-issues
Fix #254: libtool & distutils want certain compiler names.
2015-12-20 16:56:59 -08:00
Todd Gamblin
1b086c928c Merge pull request #227 from trws/fetch-deps
fetch: add options to fetch missing or all deps
2015-12-19 21:02:42 -08:00
Todd Gamblin
9edba18489 Merge pull request #230 from nolta/fix-env-mod-sys-type
fix environment module path when $SYS_TYPE isn't defined
2015-12-19 20:59:13 -08:00
Todd Gamblin
32f96b5c73 Fix stale code in python version test so that it actually works. 2015-12-19 20:04:00 -08:00
Mike Nolta
429e15c4a6 spack python: add -c option
Allows passing program in as a string. Example:

    $ spack python -c 'print 2+3'
    5

Also imports spack module by default into the environment.
2015-12-19 16:32:17 -05:00
Gregory Lee
df5dc1c9bb Merge pull request #253 from LLNL/bugfix/244-uninstall-errors
Bugfix/244 uninstall errors
2015-12-19 08:06:36 -08:00
Todd Gamblin
7cc4d88726 Fix #254: libtool & distutils want certain compiler names.
This changes the compiler wrappers so that they are called by the same
name as the wrapped compiler.  Many builds make assumptions about
compiler names, and we need the spack compilers to be recognizable so
that build systems will get their flags right.

This adds per-compiler subdirectories to lib/spack/spack/env directory
that contain symlinks to cc for the C, C++, F77, and F90
compilers. The build now sets CC, CXX, F77, and F90 to point to these
links instead of to the generically named cc, c++, f77, and f90
wrappers.
2015-12-19 02:06:47 -08:00
Todd Gamblin
52e3364efa Fix #244: errors on uninstall
- Extension logic didn't take conditional deps into account.

- Extension methods now check for whether the extnesion is
  in the extendee map AND whether the dependency is actually present
  in the spec yet.
2015-12-18 22:24:35 -08:00