hlrs-software-stack/spack
7
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity
3084 commits 3 branches 52 tags 188 MiB
Commit graph

1379 commits

Author SHA1 Message Date
Todd Gamblin
14d48eba46 Revert c5d9ee8924 for bug fixes. 
- This reverts commit c5d9ee8924.

- merged too soon before
- reverting and fixing bugs now.

Conflicts:
	lib/spack/spack/mirror.py
	lib/spack/spack/package.py
 2016-03-05 18:40:28 -08:00
Todd Gamblin
097df180e2 Merge pull request #493 from adamjstewart/features/pgi 
Modifications to get PGI working
 2016-03-05 14:19:15 -08:00
Todd Gamblin
bfce2c7508 Fix bug in hook: ensure bin directory exists before listing. 2016-03-05 04:42:52 -08:00
Todd Gamblin
fc6aa7374a Fix #104, #54: issues with overlong shebang in deep directories. 
This does several things:

- Add `sbang`: a script to run scripts with long shebang lines.
  - Documentation for `sbang` is in `bin/sbang`.

- Add an `sbang` hook that filters the `bin` directory after install
  and modifies any scripts wtih shebangs that are too long to use
  `sbang` instead.

- `sbang` is at the top level, so it should be runnable (not much we
  can do if spack itself is too deep for shebang)
  - `sbang`, when used as the interpreter, runs the *second* shebang
    line it finds in a script.
  - shoud fix issues with too long shebang paths.
 2016-03-05 04:18:48 -08:00
becker33
6449e8b1b9 Merge pull request #492 from adamjstewart/typos/compiler 
Typo fix in compiler docstring
 2016-03-04 14:25:56 -08:00
Adam J. Stewart
5db9673904 Typo fix in compiler docstring 2016-03-04 10:51:41 -06:00
citibeth
202fbc6812 Rmoved periods. Should help use of cut'n'paste from console windows. 2016-03-03 23:40:29 -05:00
Adam J. Stewart
311d71ec86 Modifications to get PGI working 2016-03-03 16:50:51 -06:00
Todd Gamblin
c5d9ee8924 Revert "refactoring proposal : turn Stage into a context manager" 2016-03-03 14:41:20 -08:00
Luigi Calori
a531a4f5e3 help spack versions paraview to get versions without v 2016-03-03 18:00:50 +01:00
alalazo
db92699c02 test : fixed tests that were broken by import optimization 2016-03-03 11:05:36 +01:00
alalazo
7f4a69319a Merge branch 'develop' of https://github.com/LLNL/spack into refactoring/stage 
Conflicts:
	lib/spack/spack/package.py
 2016-03-03 10:19:45 +01:00
alalazo
5ef3621556 stage : minor syntax fix 2016-03-03 10:10:30 +01:00
Todd Gamblin
6701977f1a Merge pull request #475 from LLNL/features/env-and-package-provenance 
Features/env and package provenance
 2016-03-03 00:42:40 -08:00
alalazo
353d12d2e6 stage : added create function as decided in #477 comments 2016-03-03 09:18:47 +01:00
alalazo
717bcb8624 test : optimized import statements 2016-03-03 09:17:49 +01:00
Gregory Becker
4f9a309de8 made spack more conservative in compiler strategies attempted 2016-03-02 16:35:02 -08:00
Gregory Becker
487b2495a1 made spack more conservative in compiler strategies attempted 2016-03-02 16:33:10 -08:00
Mario Melara
2b48728741 Changed os subclasses CNL to change attribute to MODULES 2016-03-02 16:15:34 -08:00
Mario Melara
58efe1550d Default PATH arg 2016-03-02 16:14:42 -08:00
Gregory Becker
16d8c25b23 fixed floating 'target' 2016-03-02 16:12:32 -08:00
Gregory Becker
65d4169f00 fixed type/bug 2016-03-02 16:03:11 -08:00
Gregory Becker
ffb9574312 fixed type/bug 2016-03-02 15:59:23 -08:00
Gregory Becker
6e7b00a0f6 Made module handling much saner and safer 2016-03-02 15:54:23 -08:00
Mario Melara
b4298979fe Merge branch 'features/newarch' of https://github.com/NERSC/spack into features/newarch 2016-03-02 14:18:19 -08:00
Gregory Becker
676591ffc0 fixed a bug and a test bug 2016-03-02 14:17:46 -08:00
Mario Melara
975cba295b Added 'better' tests 2016-03-02 14:16:57 -08:00
Mario Melara
e46bac19fa Changed architecture to dict 2016-03-02 12:09:02 -08:00
Mario Melara
90e90f61c1 Got rid of the compiler strategy from target 2016-03-02 12:08:43 -08:00
Mario Melara
47c8e1366f Changed to_dictionary and from_dictionary methods 2016-03-02 12:08:17 -08:00
Gregory Becker
25f20b1967 bug hunting 2016-03-02 11:40:21 -08:00
Glenn Johnson
6bbadbfaac Add CPATH pointing to include directories. 2016-03-02 13:18:43 -06:00
Mario Melara
340ecf6e9a Merge branch 'features/newarch' of https://github.com/NERSC/spack into features/newarch 2016-03-02 11:07:52 -08:00
Gregory Becker
86e90bba87 another partial, narrowing in on proper arch 2016-03-02 11:05:05 -08:00
Mario Melara
81e236b2de Changed Arch to class instead of namedtuple. Now platform is automatically set upon creation of class 2016-03-02 11:04:55 -08:00
Gregory Becker
45887dec8e partial work on bug hunting 2016-03-02 10:56:46 -08:00
Mario Melara
f27f2f8e49 Fixed some indentation errors 2016-03-02 10:27:12 -08:00
Mario Melara
4d74784209 Changed add_architecture_from_string and split the work done into two methods add_target_from_string and add_operating_system_from_string 2016-03-02 10:27:00 -08:00
Mario Melara
527bb7abfe Changed _cmp_key 2016-03-02 10:26:09 -08:00
Todd Gamblin
9f6ebd7c6e Merge pull request #465 from citibeth/efischer/160229-RemoteRootBugfix 
Bug Fix: When Spack create roots around for other versions, it someti…
 2016-03-02 10:24:17 -08:00
Gregory Becker
615ea969f8 made arch instantiate as a tuple 2016-03-02 10:14:50 -08:00
Gregory Becker
4601c36f57 cleaned up concretize architecture 2016-03-02 09:55:44 -08:00
alalazo
21cd05aad5 package : updated doc 2016-03-02 17:28:21 +01:00
alalazo
ca41909ec5 package : removed do_clean() 2016-03-02 17:19:27 +01:00
alalazo
4d63544fe9 remove_link_tree : moved to llnl.util.filesystem 2016-03-02 16:55:57 +01:00
alalazo
901e4851b9 _cleanup_dead_links : factored method into a function and put it in llnl.filesystem 2016-03-02 16:19:32 +01:00
alalazo
9001b9ed3c package : minor syntax fixes 
mirror : extracted add_single_spec from create
 2016-03-02 15:56:09 +01:00
alalazo
d649b715ed stage : updated functions doc 
_cleanup_dead_links : fixed minor bug
 2016-03-02 13:16:04 +01:00
alalazo
65b2a24f7c stage : removed _setup method 2016-03-02 13:03:40 +01:00
alalazo
726b350689 test : fixed failing unit tests 2016-03-02 12:52:38 +01:00
Todd Gamblin
a0c6519de9 Fixes #476: create was broken for FTP URLs. 2016-03-02 01:47:54 -08:00
Todd Gamblin
52081c46d6 Fixes #469: Store package.py files in the .spack directory. 
- Adds packages in spack repos inside the .spack directory, so that
  packages can be rebuilt automatically later.
 2016-03-02 00:09:24 -08:00
Todd Gamblin
21d125c914 Fixes #175: Dump environment provenance as well as build log. 2016-03-02 00:08:36 -08:00
Todd Gamblin
be306d09e9 Move repo creation code into repository.py 2016-03-02 00:04:46 -08:00
Todd Gamblin
c488f7c4d8 Fix bug in install permission setting. 2016-03-01 23:57:34 -08:00
Adam J. Stewart
a9f0b98d4d Typo fix 2016-03-01 15:53:53 -06:00
Elizabeth F
a339ac0a72 Bug Fix: When Spack create roots around for other versions, it sometimes finds files it thinks are tarballs, but are not. Previously, it would crash if any such files are found. This change allows it to simply skip them and move on, processing the rest of the files it finds correctly. 2016-02-29 23:04:11 -05:00
Ian Lee
67f327f805 Updated links to use new llnl.gov address 2016-02-28 19:47:19 -08:00
Mario Melara
b591d2b501 Changed operating_system path 2016-02-25 18:49:52 -08:00
Mario Melara
ccd4a79b39 changed architecture.target to architecture.platform_os, so that compiler_for_spec uses operating system strategy 2016-02-25 18:49:27 -08:00
Mario Melara
eb96f38298 Got rid of debug comment 2016-02-25 18:48:25 -08:00
Mario Melara
f0149faf88 Added doc strings to functions in architecture.py 2016-02-25 18:48:00 -08:00
Mario Melara
ac55ce989d Changed compiler finding strategy to come from operating system instead of target 2016-02-25 18:47:41 -08:00
becker33
7176e5ef07 Merge pull request #299 from epfl-scitas/enhancement/os_detection 
enhancement proposal : boolean support for when=<arg>
 2016-02-25 13:38:03 -08:00
Mario Melara
5989e3f65d Changed from_dict to arch_from_dict method 2016-02-25 11:53:02 -08:00
Mario Melara
5e4d3e7b82 Changed naming convention of operating_system, makes it easier to create instance from dictionary just using name without the version attached. Object will stil display as name+version 2016-02-25 11:48:43 -08:00
Mario Melara
550df4cdd6 Some cleaning up. Finally got arch_from_dict working successfully. 2016-02-25 11:47:27 -08:00
Mario Melara
9ac2556285 Deleted old operating_system (without the s) file 2016-02-25 10:41:28 -08:00
Mario Melara
b43a5498a1 Changed import path to operating_systems 2016-02-25 10:36:47 -08:00
Mario Melara
f161609929 Renamed operating_system to operating_systems 2016-02-25 10:36:12 -08:00
Mario Melara
34be473b7c Deleted files 2016-02-25 10:35:58 -08:00
Mario Melara
863a5bc4fa Fix for 'Parent module spack.operating_systems not found while handling absolute import' 2016-02-25 10:34:55 -08:00
Mario Melara
8d1c06d141 Changed to dict method. Didn't realize this was being assigned to a dictionary already so got rid of d['architecture'] 2016-02-24 15:38:33 -08:00
Mario Melara
bd7c189c12 Added more comments 2016-02-24 15:37:50 -08:00
Mario Melara
5c3c6e7f06 Got rid of print statements 2016-02-24 15:34:32 -08:00
Mario Melara
234681306c Changed to appropriate class names for operating system 2016-02-24 15:32:23 -08:00
Mario Melara
2de81cfc62 Changed name to appropriate camelcase 2016-02-24 15:31:21 -08:00
Mario Melara
9e844d974c Added comments. Need to figure out how to make arch_tuple from_dict 2016-02-24 15:30:20 -08:00
Mario Melara
7779974143 arch_from_dict worked on 2016-02-24 15:02:40 -08:00
Mario Melara
339f4bfd23 Added unit testing for to_dict method 2016-02-24 12:16:09 -08:00
Mario Melara
5d5d3c5858 Added to dictionary method 2016-02-24 12:15:52 -08:00
alalazo
6f42dd556d stage : on-going refactoring 2016-02-24 17:37:58 +01:00
alalazo
5ce97eeda5 intel : polished code 2016-02-24 09:17:04 +01:00
Massimiliano Culpo
4316f1cd31 Modified wrapper to have a different behavior when modeis vcheck 2016-02-23 16:57:19 +01:00
Mario Melara
cfa7c4feb8 Added CNL10 as back_os and default_os 2016-02-22 16:18:25 -08:00
Mario Melara
083b7b46d9 Added more tests 2016-02-22 16:17:54 -08:00
Mario Melara
3c87d137a3 Added compiler strategy entry and also added new operating system subclass called CNL for compute node linux (name will probably change because I can't find docs on the EXACT name) 2016-02-22 16:17:33 -08:00
Mario Melara
ebc5e26c2d Added compiler strategy field to Operating System class 2016-02-22 16:16:20 -08:00
Mario Melara
a385dae1ae Unit tests to test operating system subclass creation and whether compiler strategy is set correctly 2016-02-22 14:54:30 -08:00
Mario Melara
1367ccab93 New folder that will hold operating system subclasses 2016-02-22 14:53:55 -08:00
Mario Melara
22bf4bc080 import linux distro subclass. Added method to add operating system to platform dictionary 2016-02-22 14:53:16 -08:00
Mario Melara
1a7d6ed49a import MacOSX subclass and added add_operating_system method to use subclass 2016-02-22 14:52:52 -08:00
Mario Melara
725d6d5fce import linux distro operating system subclass, set front-end os to LinuxDistro() 2016-02-22 14:52:09 -08:00
Mario Melara
a8de45ce28 Got rid of methods used to detect mac osx and linux osx. Now those methods are used my operating system subclasses 2016-02-22 14:50:51 -08:00
Mario Melara
18ddbae60e Added new module path to operating system file 2016-02-22 14:49:53 -08:00
Todd Gamblin
3c1aa9a4ad Merge pull request #298 from epfl-scitas/refactoring/resource_directive 
refactoring proposal : composite for Stage and FetchStrategy
 2016-02-18 23:13:12 -08:00
Mario Melara
28218755a5 Changed spec.target to spec.architecture 2016-02-18 13:06:55 -08:00
Todd Gamblin
d8a70166d3 Fixes #434 
Compiler detection was not getting triggered properly with some of the
new config logic.  Adjust the conditions under which Spack will serach
for compilers.
 2016-02-18 00:56:29 -08:00
Todd Gamblin
c112cf66fe Merge pull request #443 from trws/dicts-like-string-keys 
remove the unprotected key name from create.py
 2016-02-17 17:06:51 -08:00
Tom Scogland
00125e4688 remove the unprotected key name from create.py 
Small fix to use dicts correctly.
 2016-02-17 16:43:36 -08:00
Mario Melara
93a92a00ee Changed abstract.target to abstract.architecture.target and abstract.architecture.target.concrete 2016-02-17 14:53:09 -08:00
Mario Melara
d9e8bf1807 Added ARCHITECTURE field when showing specs 2016-02-17 14:51:14 -08:00
Mario Melara
54042e399b Changed target to architecture.target 2016-02-17 14:50:10 -08:00
Mario Melara
1d484dbe5f Changed so that modules use correct file path with spec.architecture instead of spec.target 2016-02-17 14:49:36 -08:00
Mario Melara
ab4006ee2c Changed so that directory layout uses the platform-os-target string version of the arch tuple 2016-02-17 14:48:53 -08:00
Mario Melara
524e9b372e From now on all targets are now architecture. Architecture is a named tuple so any time we need access to target we do architecture.target. 2016-02-17 14:47:29 -08:00
Mario Melara
2c20fc2ebf Changed module loading to use arch tuple instead of regular target.module 2016-02-17 14:45:53 -08:00
Mario Melara
2a79537ba1 Fixed silly typo that gave errors in tests 2016-02-17 14:44:38 -08:00
Mario Melara
a3039c4c67 Changed add_architecture_from_string, it now loops through the string and checks whether each piece of string is a valid platform, operating system and target. If the operating system or target are none it will use the defaults. Updated the documentation for that method. One thing that bothers me is how to account for the multitude of operating systems when cross compiling. If someone wants to compile with another operating system not found on current platform. How can spack check to see if it is valid? 2016-02-16 15:26:07 -08:00
Mario Melara
62b0293963 Cleaned up the file. Got rid of commented out sections of code since they weren't going to be used anyway 2016-02-16 15:23:38 -08:00
Mario Melara
31ab238306 Beginning attemps to fix concretization method to account for the new tuple architecture 2016-02-16 15:22:23 -08:00
Mario Melara
19dca4bcc9 Added some _cmp_key methods to OperatingSystem class, and also changed the way Platforms are compared. Created a mini inherited class named Arch that inherits from namedtuple. The reason for this is to override the namedtuple __str__ method to make arch tuples look prettier when printed out. Rather than Arch(platform= ... ) it prints to platform-os-target which makes directory paths to packages look cleaner. 2016-02-16 15:21:36 -08:00
Mario Melara
35a602baaf Changed target checking to architecture checking for abi compatible 2016-02-16 15:17:57 -08:00
Erik Schnetter
9868333e8e Shell quoting is difficult 2016-02-14 11:34:08 -05:00
Erik Schnetter
a06e29fecb More quoting experiments 2016-02-14 10:43:55 -05:00
Erik Schnetter
43670cbbd0 More games with quoting 2016-02-13 22:01:36 -05:00
Erik Schnetter
52647b9a5d Using regexes instead of globbing to match path names 2016-02-13 17:55:14 -05:00
Mario Melara
c799301011 Changed the method in which architecture is converted from string to namedtuple. Also added some TODO comments to later consider changing how architecture is handled 2016-02-13 14:37:07 -08:00
Mario Melara
7732f375d2 Changed operating system getter so that linux and darwin autodetected operating systems are set as class default_os 2016-02-13 14:36:01 -08:00
Erik Schnetter
69064395eb Add debug output 2016-02-13 15:12:09 -05:00
Erik Schnetter
9a2c1090a6 Handle -Wl,-rpath,... syntax 2016-02-13 14:54:18 -05:00
Erik Schnetter
dc6a33b716 Handle multiple -Wl,-rpath,... paths 2016-02-13 14:54:18 -05:00
Erik Schnetter
5038a38e29 Correct -Xlinker arguments 2016-02-13 14:54:18 -05:00
Erik Schnetter
2cd9ad8ce6 Use "-Wl,-rpath," instead of "-Wl,-rpath=" 
The former translates to a linker argument "-rpath DIR", whereas the latter translates to "-rpath=DIR". The latter is not support on OS X.
 2016-02-13 14:54:18 -05:00
Erik Schnetter
ff81aff254 Remove source-code optimization and cleanup 2016-02-12 13:05:42 -05:00
Erik Schnetter
64436e7752 Merge branch 'develop' into correct-cc 2016-02-12 12:55:26 -05:00
David Beckingsale
90e1b1f0ea Use hash to make shorter module and dotkit names, fixes #433 2016-02-12 09:33:19 -08:00
Mario Melara
8e8c63bd67 Using pylint, fixed some of the indentation and spacing errors 2016-02-11 11:47:39 -08:00
Mario Melara
3e1be63b0f Changed some variables to account for the fact that target is now a tuple 2016-02-11 11:15:50 -08:00
Mario Melara
b9d09202c9 changed some variables to account for the fact that target is now a tuple 2016-02-11 11:15:19 -08:00
Mario Melara
77e93e1b79 Added add_os method to platform subclasses 2016-02-10 16:16:58 -08:00
Mario Melara
cb8d5ab183 Added new add_os method 2016-02-10 16:16:35 -08:00
Mario Melara
fb234205c2 Added method to concretize target and os pair 2016-02-10 16:16:11 -08:00
Mario Melara
d0ae6dd401 Changed add_target_from_string to include methods to concretize operating_system 2016-02-10 16:15:25 -08:00
Mario Melara
2650c60374 Added operating system class prototype and some autodetect features for operating system 2016-02-10 16:14:50 -08:00
Todd Gamblin
bcf705b85e Merge pull request #432 from RemoteConnectionManager/paraview_fix 
Paraview fix
 2016-02-10 13:59:08 -08:00
Mario Melara
21a5a34041 Revert "new branch and also pushing some architecture changes where os is detected by linux and darwin and manually set by cray and bgq" 
This reverts commit 70088be24b2ed34076f7f5292a2a465a8655a886.
 2016-02-10 10:59:16 -08:00
Luigi Calori
25e2be3efd RPATH was collecting all lib from all dependencies, when dep tree is deep like in ParaView the path was growing too much, showing install problems now just direct dependencies rpath is added 2016-02-10 13:28:01 +01:00
Todd Gamblin
f7134990bd Fix #430: edit -f fails with UnknownPackageError 
- Recent external repo refactoring made `repo_for_pkg` raise an error
  when the package was not known.

- Correct behavior is to return the highest precedence repo.
 2016-02-09 08:57:27 -08:00
Mario Melara
0358522533 new branch and also pushing some architecture changes where os is detected by linux and darwin and manually set by cray and bgq 2016-02-06 15:41:22 -08:00
Nicolas Richart
824546d343 correcting a bug when url and files are mixed 2016-01-28 14:43:30 +01:00
Nicolas Richart
5850d8530e Adding the stager to checksum any url that spack can handle 2016-01-28 14:22:28 +01:00
alalazo
0cf03518f3 reverted deletion of the wrong architecture.py 2016-01-28 12:06:30 +01:00
alalazo
603f5e69ee removed outdated code : os detection (will be handled by platforms) and openssl modifications 2016-01-28 11:30:28 +01:00
alalazo
e76b722e28 Merge branch 'develop' of https://github.com/LLNL/spack into enhancement/os_detection 2016-01-28 11:26:01 +01:00
alalazo
07bb6fef01 resource directive : now works with all the fetch strategies available 2016-01-28 10:58:56 +01:00
alalazo
47035671e8 unit tests : now compliant with 2.6 2016-01-27 17:22:12 +01:00
alalazo
f7f192e12b Added unit tests for util.pattern 2016-01-27 17:12:24 +01:00
Gregory L. Lee
7891346a4f fix for issue 398, added extensions_for in RepoPath 2016-01-26 09:52:01 -08:00
alalazo
ee6f69a227 Unit tests : hack to make them pass while a decision on how to proceed is made 2016-01-26 14:25:16 +01:00
alalazo
5e3b7a4247 Fixes issues introduced after merge with conflicts 2016-01-26 14:11:23 +01:00
alalazo
093b831799 Fixes issues introduced after merge with conflicts 2016-01-26 12:08:28 +01:00
alalazo
6f11a64af5 Merge branch 'develop' of https://github.com/LLNL/spack into refactoring/resource_directive 
Conflicts:
	lib/spack/spack/fetch_strategy.py
	lib/spack/spack/mirror.py
	lib/spack/spack/package.py
 2016-01-26 08:59:25 +01:00
Matthew LeGendre
d4a771a623 Merge pull request #1 from alalazo/features/external-packages 
doc : minor typos fixed
 2016-01-25 10:53:37 -08:00
Matthew LeGendre
fa888a4ba1 Merge branch 'develop' into features/external-packages 
Conflicts:
	lib/spack/spack/cmd/mirror.py
	lib/spack/spack/concretize.py
	lib/spack/spack/config.py
	lib/spack/spack/spec.py
	lib/spack/spack/stage.py
	var/spack/packages/mvapich2/package.py
 2016-01-25 10:52:17 -08:00
Todd Gamblin
004c99ab2f Fixes #259: Apple clang compiler detection 
- Clang compilers are now versioned with the raw LLVM version or, on
  macs, with Apple's version with a suffix.
 2016-01-25 03:48:22 -08:00
Adam J. Stewart
fb5274145a Fix regex to find version on OS X 2016-01-25 03:10:13 -08:00
Todd Gamblin
41cd8f8e6d Fixes #74: minor fixes to spack mirror. 2016-01-25 02:57:23 -08:00
Todd Gamblin
5502fd1054 More thorough mirror test: test full round-trip 
- Old test: did not attempt to actually fetch mirrored packages.

- New test:
  1. Creates a temporary mirror,
  2. Registers it with spack,
  3. Fetches from it, and
  4. Verifies that the fetched archive matches the original

- This test will hopefully mean that `spack mirror` is less brittle
  from now on.
 2016-01-25 01:59:02 -08:00
Todd Gamblin
a48d0a494f Refactor mock_repo: add a destroy() method. 
- classes using mock_repo need not know about its stage now.
 2016-01-25 00:34:33 -08:00
Todd Gamblin
d3ff8ca00f Fixes #382: Issues with spack fetch. 
- urljoin() was compliant with RFC 1808 but not with my understanding
  of how paths should be joined.
- updated path joining logic to comply.
 2016-01-24 20:14:40 -08:00
Todd Gamblin
fe50593c66 Minor line width reductions. 2016-01-24 20:14:21 -08:00
Todd Gamblin
be354e85c9 Better errors for mkdirp failure in mirror. 2016-01-24 16:16:43 -08:00
Todd Gamblin
64a9549225 Minor cleanup. 80 char width for command description. 2016-01-24 15:31:07 -08:00
Adam J. Stewart
571386cdf4 Remove cxx symlink, tell NAG to look for c++ symlink 2016-01-21 13:55:59 -06:00
Adam J. Stewart
ca3edc3673 Add symlink cxx to point to cc 2016-01-21 13:20:39 -06:00
Todd Gamblin
10848c2e9a Refactor args for Executable.__call__ 
- simplify output, error, and input redirection
- `return_output=True` is now `output=str`
- `return_output=True` will still work for the time being but is
  deprecated.

- previously you could say `return_output=True` and `output=<stream>`,
  which wouldn't actually write to the stream.  Now you actually can't
  specify erroneous cases since there is only one parameter with
  mutually exclusive options..
 2016-01-21 10:46:33 -08:00
Todd Gamblin
1c2a2dde8e Merge pull request #350 from adamjstewart/features/nag 
Add NAG Fortran Compiler support
 2016-01-20 17:09:31 -08:00
Adam J. Stewart
088b6f0ea9 Fixed typo when running spack help 2016-01-20 17:19:01 -06:00
Adam J. Stewart
e25150296a Redirect STDERR to STDOUT for compiler version 
This is necessary for the NAG Fortran compiler, which prints its version message to STDERR instead of STDOUT. This message was previously being ignored, preventing spack from finding the compiler's version or automatically adding it to the compilers.yaml configuration file.
 2016-01-20 14:44:43 -06:00
Mario Melara
840b41c450 Removed unneccessary $. Still learning regexp 2016-01-20 10:36:15 -08:00
Mario Melara
15713219e5 Better regular expression searching. Tested on edison was finding test compilers and then spack was giving multiple compiler match errors 2016-01-20 10:32:56 -08:00
alalazo
72d5cdf9ab fixed two minor typos 2016-01-20 11:19:36 +01:00
Ben Boeckel
6c9800bcc3 typos: fix some letter transposes 2016-01-19 15:22:16 -05:00
Adam J. Stewart
83de658ee4 Modify nag.py to match new compiler package structure 2016-01-19 13:28:35 -06:00
Adam J. Stewart
e6aa2610bc Add symlink for NAG Fortran Compiler 2016-01-19 13:07:07 -06:00
Adam J. Stewart
fe640d764a Merge branch 'develop' into features/nag 
* develop: (77 commits)
  Make libxcb compile with gcc 4.9.
  disable parallel install for glib (found races)
  disable cairo-trace, which is incompatible with older libiberty.h.
  Fix LaunchMon on newer gcc versions.
  Simplify output redirection in spack.util.executable
  Allow completely empty config files.
  Update cmake package to use http in all URLs.
  Fix create, diy, edit, and repo commands to use multiple repos.
  Add namespace option to find command.
  Temporary fix: Clang is the default compiler on Mac OS X.
  Netcdf requires cmake >=2.8.12
  Mbedtls depends on cmake
  Update libtool to 2.4.6
  Update Julia to 0.4.3
  Make binutils build on OS X
  Add verbs and psm variants
  py-matplotlib: depend on freetype
  py-numpy: make blas/lapack optional dependencies
  qhull: apply patch to work with libc++'s standard library
  New package GNU tar
  ...
 2016-01-19 09:56:02 -06:00
Todd Gamblin
cee7bfa9f0 Simplify output redirection in spack.util.executable 
- By default inherit parent's input/output descriptor
- Only use pipes if we need to return output.
- Allows subprocesses (like emacsclient) to detect terminal correctly
 2016-01-19 01:16:14 -08:00
Todd Gamblin
ad32f64ef6 Allow completely empty config files. 
- Previous version would give validation error for an empty file.
  Now this is properly ignored.

- Also includes bugfix in ConfigFormatError
 2016-01-19 01:16:14 -08:00
Todd Gamblin
8d6342c53d Merge branch 'mplegendre-multi_pkgsrc_roots' into develop 
- This moves var/spack/packages to var/spack/repos/builtin/packages.

- Packages that did not exist in the source branch, or were changed in
  develop, were moved into var/spack/repos/builtin/packages as part of
  the integration.

Conflicts:
	lib/spack/spack/test/unit_install.py
	var/spack/repos/builtin/packages/clang/package.py
 2016-01-19 01:16:08 -08:00
Todd Gamblin
c1e57e49ae Merge pull request #202 from scheibelp/features/unittest-xml-output-nose 
Features/unittest xml output nose
 2016-01-17 19:14:49 -08:00
Todd Gamblin
97b492756a Fix create, diy, edit, and repo commands to use multiple repos. 2016-01-17 18:14:35 -08:00
Todd Gamblin
5984bc2ad3 Add namespace option to find command. 2016-01-17 18:12:53 -08:00
Todd Gamblin
358b2ab4ba Temporary fix: Clang is the default compiler on Mac OS X. 2016-01-17 18:12:08 -08:00
Gregory Becker
8e54babf10 minor parsing bug fix 2016-01-15 17:21:32 -08:00
Gregory Becker
e1a0af8745 cleanup 2016-01-15 16:51:41 -08:00
Gregory Becker
16e9d658d5 Fixed the way flags on deps are attached so that they aren't moved onto the root before normalize/concretize 2016-01-15 16:18:40 -08:00
Gregory Becker
342f4bc2e0 Fixed virtual/cflag combination bug 2016-01-15 14:27:50 -08:00
Todd Gamblin
aa00999fca update compiler config test. 2016-01-14 12:00:56 -08:00
Todd Gamblin
d869d8e86f add sanity check on config write as well as validation on read. 2016-01-14 12:00:44 -08:00
Todd Gamblin
b567cb57e1 Add jsonschema validation for config files. 2016-01-14 10:41:35 -08:00
Todd Gamblin
05b30bf83e Make text wrapping off by default in tty, add a kwarg for it. 2016-01-14 10:26:31 -08:00
Adam J. Stewart
f9fbb57d31 Add NAG Fortran Compiler support 2016-01-13 12:32:04 -06:00
Gregory Becker
848728858c Removed extra plus signs from command line syntax. Did not yet remove them from printed format 2016-01-12 18:34:26 -08:00
Gregory Becker
5e3c883f2c Cleanup and proper flag satisfaction for partial specs 2016-01-12 16:25:54 -08:00
Gregory Becker
528f9cd583 Implemented flags as lists for subsetting 2016-01-12 15:22:15 -08:00
Gregory Becker
2ac5ea42af Fixed concretization 2016-01-11 13:55:11 -08:00
Mario Melara
aab1a67d05 Adding needed import modules. Changed platform_name to platform to prevent syntax errors 2016-01-09 09:42:36 -08:00
Gregory Becker
9615efd940 cleanup: Removing an unnecessary line in an otherwise unchanged file 2016-01-08 18:53:19 -08:00
Gregory Becker
41046499c6 minor bug chasing 2016-01-08 18:51:38 -08:00
Gregory Becker
6e5238d037 Fixing the fix and removing debug printing 2016-01-08 17:44:16 -08:00
Gregory Becker
2b2d4bae4e Fixing multimethod test for new platforms 2016-01-08 17:38:37 -08:00
Gregory Becker
2b4dd8b9af Fixed target satisfaction and updated tests accordingly 2016-01-08 16:30:27 -08:00
Gregory Becker
aa28e4e81f Improved error messages for compiler_for_spec when either zero or multiple compilers returned. 2016-01-08 15:13:48 -08:00
Gregory Becker
d3d37ad0ce removed the submodule as part of reversion of extraneous commits 2016-01-08 14:29:42 -08:00
Gregory Becker
ba63111f45 brought the architecture test up to date with new changes 2016-01-07 13:05:58 -08:00
Gregory Becker
53d4f82ce1 Improved cray architecture class 2016-01-07 13:01:05 -08:00
Gregory Becker
83917c4c30 Improved target specification 2016-01-07 12:43:39 -08:00
Gregory Becker
6ccd9d6fa4 Cleaned up naming conventions for architecture, split into platform and target 2016-01-07 11:49:01 -08:00
Gregory Becker
b4a0004f44 Improved target cmp_key and to/from yaml functions 2016-01-06 14:50:31 -08:00
Gregory Becker
51bd91edc0 made the compiler strategy more easily controllable 2016-01-06 13:00:22 -08:00
Gregory Becker
61b03b72b0 improved concretize efficiency for determining whether compilers come from the proper strategy 2016-01-05 19:03:25 -08:00
Gregory Becker
93c9c45580 minor tweak of compiler priority while waiting for customizable compiler priorities 2016-01-05 17:19:04 -08:00
Gregory Becker
29e0ff61d7 fixed compiler finding so as not to identify non-existent versions of the intel compiler based on the version numbers of the PrgEnv-intel module 2016-01-05 17:14:35 -08:00
Gregory Becker
c649610473 fixed concretization to only consider compilers found by the proper strategy 2016-01-05 17:00:28 -08:00
Gregory Becker
d6768cf921 minor bug fix 2016-01-05 16:23:32 -08:00
Gregory Becker
48b9023de4 removed debug printing 2016-01-05 14:51:18 -08:00
Gregory Becker
751503c434 fixed haswell targeting bug 2016-01-05 14:47:14 -08:00
Gregory Becker
1edbaa4e62 removed debug printing 2016-01-05 14:13:09 -08:00
Gregory Becker
9848ad32fd fixed errors caused by crayport on linux and added a darwin architecture 2016-01-05 13:39:53 -08:00
Gregory Becker
d2c2c46541 bug hunting 2016-01-05 11:12:14 -08:00
Gregory Becker
751208cedf pick last compiler instead of first, will be overwritten by more advanced preferences as mentioned in the email group 2016-01-04 16:35:06 -08:00
Gregory Becker
53808f254e Support for cray external dependencies implemented in modules 2016-01-04 12:36:48 -08:00
Gregory Becker
ff82e41404 Merge branch 'externals' into crayport 2016-01-04 10:46:25 -08:00
Todd Gamblin
b02faf5641 add/remove/list working for new config format. 
- mirrors.yaml now uses dict order for precedence, instead of lists of
  dicts.

- spack.cmd now specifies default scope for add/remove and for list
  with `default_modify_scope` and `default_list_scope`.
  - commands that only read or list default to all scopes (merged)
  - commands that modify configs modify user scope (highest
    precedence) by default
  - These vars are used in setup_paraser for mirror/repo/compiler.

- Spack's argparse supports aliases now.
  - added 'rm' alias for `spack [repo|compiler|mirror] remove`
 2016-01-03 02:27:50 -08:00
Todd Gamblin
21fae634a5 Add Python 3 aliases to our argparse backport. 2016-01-03 01:19:03 -08:00
Todd Gamblin
56937416a9 All tests that call concretize() need to be MockPackagesTests. 2016-01-02 18:18:00 -08:00
Todd Gamblin
20b7f8a8e0 Fix bug in tests. 2016-01-02 14:25:10 -08:00
alalazo
bf4231dd4b depends_on and when multimethod support boolean 2016-01-02 22:48:14 +01:00
alalazo
d95d169ac5 fixed broken unit tests 2016-01-02 19:04:06 +01:00
alalazo
f499f71f64 Package : factored out code in do_stage and do_fetch, changed mirror command accordingly 2016-01-02 16:15:30 +01:00
alalazo
dcddb19e5b added class decorator to define composite classes 2016-01-01 17:35:01 +01:00
Tom Scogland
4ae98f8b21 significant llvm update 
This update significantly reworks the llvm and clang packages.  The llvm
package now includes variants allowing it to build and install any and
all of:

* clang
* lldb
* llvm's libunwind (why, WHY did they name it this?!?)
* polly (including building it directly into the clang tools, 3.7.0 only)
* clang extra tools
* compiler-rt (sanitizers)
* clang lto (the gold linker plugin that allows same to work)
* libcxx/libcxxabi
* libopenmp, also setting the default openmp runtime to same, when
  parameters happen this shoudl be an option of libomp or libgomp

Ideally, this should have rpath setup like the gcc package does, but
clang's driver has no support for specs as such, and no clearly
equivalent mechanism either.  If anyone has ideas on this, they would be
welcome.

One significant note related to gcc though, if you test this on LLNL
systems, or anywhere that has multiple GCCs straddling the dwarf2
boundary and sharing a libstdc++, build a gcc with spack and use that to
build clang.  If you use a gcc4.8+  to build this with an older
libstdc++ it will fail on missing unwind symbols because of the
discrepancy.

Resource handling has been changed slightly to move the unpacked archive
into the target rather than use symlinks, because symlinks break certain
kinds of relative paths, and orders resource staging such that nested
resources are unpacked after outer ones.
 2015-12-30 16:59:39 -08:00
Tom Scogland
fcdf08e4d7 allow nested resources 
This solution doesn't really make me happy, but does seem to work.  It
sorts the resources by the length of the string representing their
destination.  Since any nested resource must contain another resource's
name in its path, it seems that should work, but there should be a
better way to do this.
 2015-12-30 16:59:39 -08:00
Tom Scogland
cc8c783bb2 adding errno import for the deep directory creation patch 2015-12-30 16:59:39 -08:00
Tom Scogland
30f5ccb80d create leading directories for resources 
This allows resources to be placed into subdirectory trees that may not
exist in the base package, and may depend on other resources to be
staged later.
 2015-12-30 16:59:39 -08:00
Todd Gamblin
e8e6368cc8 Rework mirror configuration. 
- All of these work:
  - `spack mirror add`
  - `spack mirror remove`
  - `spack mirror list`

- `spack mirror` subcommands (except create) now have their own
  --scope argument.

- Mirror config is now stored sanely as an ordered list.
 2015-12-28 01:14:41 -08:00
Todd Gamblin
1f8ba53ca7 Rework compiler configuration and simplify config.py logic. 
- `spack compiler` subcommands now take an optional --scope argument.

- no more `remove_from_config` in `config.py` -- `update` just
  overwrites b/c it's easier to just call `get_config`, modify YAML
  structures directly, and then call `update`.

- Implemented `spack compiler remove`.
 2015-12-28 00:46:51 -08:00
Todd Gamblin
ff0d871612 Remove mock_configs; tests no longer modify spack home directory. 2015-12-27 21:13:18 -08:00
Todd Gamblin
39e360f93a Add custom YAML loader & dumper to track lines & maintain dict order. 
- Configs are now parsed with `spack.util.spack_yaml.load/dump`

- Parser annotates returned data with `_start_mark` and `_end_mark`
  properties, so that we can recover what lines/files they came from.

- Parser uses `OrderedDict` instead of `dict`.  This will help
  maintain some sanity when round-tripping config files.
 2015-12-27 17:51:11 -08:00
Todd Gamblin
4e8e298eb3 Uncomment disabled tests. 2015-12-27 17:40:31 -08:00
Todd Gamblin
487ac9cd19 Merge pull request #277 from LLNL/bugfix/vcs-mirror-fetch 
Fix #85 and #228: errors fetching VCS packages from a mirror.
 2015-12-26 09:28:38 -08:00
Todd Gamblin
bef52570ae Default to scope with highest precedence instead of user scope, 
- Generalizes config scopes a bit more: nothing assumes there is a
  'user' scope (this would break testing sometimes).
 2015-12-25 18:42:06 -08:00
Todd Gamblin
c65fd3a289 Merge branch 'develop' into mplegendre-multi_pkgsrc_roots 
Conflicts:
	lib/spack/spack/cmd/create.py
	lib/spack/spack/cmd/extensions.py
	lib/spack/spack/cmd/fetch.py
	lib/spack/spack/cmd/uninstall.py
	lib/spack/spack/config.py
	lib/spack/spack/database.py
	lib/spack/spack/directory_layout.py
	lib/spack/spack/packages.py
	lib/spack/spack/spec.py
 2015-12-25 16:35:55 -08:00
Todd Gamblin
34401cf0c3 Rework Spack config: keep user & site config in memory. 
- User and site config are now kept separately in memory.
- Merging is done on demand when client code requests the configuration.
- Allows user/site config to be updated independently of each other by commands.
- simplifies config logic (no more tracking merged files)
 2015-12-25 14:00:33 -08:00
Todd Gamblin
52625de52c Fix #154 -- better log messages for do_patch() 2015-12-24 11:37:40 -08:00
Todd Gamblin
f37f872e5f Fix #85 and #228: errors fetching VCS packages from a mirror. 
- Stage and fetcher were not being set up properly when fetching using
  a different fetch strategy than the default one for the package.

- This is fixed but fetch/stage/mirror logic is still too complicated
  and long-term needs a rethink.

- Spack will now print a warning when fetching a checksum-less tarball
  from a mirror -- users should be careful to use https or local
  filesystem mirrors for this.
 2015-12-23 16:23:58 -08:00
Todd Gamblin
5ca5884ad6 Fix #235: Smarter web spidering -- use parsed links instead of reconstructing. 
- Move `find_versions_of_archive` from spack.package to `spack.util.web`.

- `spider` funciton now just uses the link parsing it already does to
  return links.  We evaluate actual links found in the scraped pages
  instead of trying to reconstruct them naively.

- Add `spack url-parse` command, which you can use to show how Spack
  interprets the name and version in a URL.
 2015-12-23 13:16:57 -08:00
Todd Gamblin
d63cb8b537 Fix bug in URL regex. 2015-12-22 17:05:23 -08:00
Todd Gamblin
0d4b77a24b revert accidental print statements. 2015-12-22 17:02:14 -08:00
Todd Gamblin
2b89d9b1db More consistent URL parsing when finding versions. 
Versions found by wildcard URLs are different from versions found by
parse_version, etc.  The wildcards are constructed more haphazardly
than the very specific URL patterns in url.py, so they can get things
wrong.  e.g., for this URL:

    https://software.lanl.gov/MeshTools/trac/attachment/wiki/WikiStart/mstk-2.25rc1.tgz

We miss the 'rc' and only return 2.25r as the version if we ONLY use
URL wildcards.

Future: Maybe use the regexes from url.py to scrape web pages, and
then compare them for similarity with the original URL, instead of
trying to make a structured wildcard URL pattern?  This might yield
better results.
 2015-12-22 16:59:00 -08:00
Erik Schnetter
9017ec5865 Merge branch 'develop' into correct-cc 2015-12-22 15:49:43 -05:00
Erik Schnetter
3427174eb0 Correct quoting 2015-12-22 15:49:14 -05:00
Todd Gamblin
b1d40efd99 Merge pull request #261 from LLNL/features/prefer-versions-in-packages 
Add ability to prefer particular versions in packages.
 2015-12-22 08:25:52 -08:00
Todd Gamblin
157e42de22 Merge pull request #264 from nolta/typos 
fix a few comment typos
 2015-12-22 07:56:54 -08:00
Todd Gamblin
792b033255 Slightly more robust tmp directory search. 
- remove getcwd() check (seems arbitrary -- if users set their TMPDIR
  to this why stop them?)

- try a number of common locations and try per-user directories in
  them first.
 2015-12-22 07:54:25 -08:00
Mike Nolta
61f07b0c4d fix a few comment typos 2015-12-21 19:24:16 -05:00
Mike Nolta
2ba6bb21fb fix #249 (tmp spack-stage directories conflict) 2015-12-21 18:50:58 -05:00
Todd Gamblin
fe0fdf60b4 Add ability to prefer particular versions in packages. 
- Adding `preferred=True` to a version directive will change its sort
  order in concretization.

- This provides us a rudimentary ability to keep the Spack stack
  stable as new versions are added.

- Having multiple stacks will come next, but this at least allows us
  to specify default versions of things instead of always taking the
  newest.
 2015-12-21 15:35:47 -08:00
Todd Gamblin
73ea15db8e Merge pull request #208 from epfl-scitas/features/resource_directive 
resource directive : implementation + clang / llvm use case
 2015-12-21 10:34:16 -08:00
Erik Schnetter
611537eaed Merge branch 'develop' into correct-cc 2015-12-21 13:18:35 -05:00
Erik Schnetter
ba22fc8b78 Simplify Spack cc script 2015-12-21 10:21:33 -05:00
Todd Gamblin
dccfcd10f7 Merge pull request #255 from LLNL/bugfix/254-libtool-compiler-name-issues 
Fix #254: libtool & distutils want certain compiler names.
 2015-12-20 16:56:59 -08:00
Todd Gamblin
1b086c928c Merge pull request #227 from trws/fetch-deps 
fetch: add options to fetch missing or all deps
 2015-12-19 21:02:42 -08:00
Todd Gamblin
9edba18489 Merge pull request #230 from nolta/fix-env-mod-sys-type 
fix environment module path when $SYS_TYPE isn't defined
 2015-12-19 20:59:13 -08:00
Todd Gamblin
32f96b5c73 Fix stale code in python version test so that it actually works. 2015-12-19 20:04:00 -08:00
Mike Nolta
429e15c4a6 spack python: add -c option 
Allows passing program in as a string. Example:

    $ spack python -c 'print 2+3'
    5

Also imports spack module by default into the environment.
 2015-12-19 16:32:17 -05:00
Gregory Lee
df5dc1c9bb Merge pull request #253 from LLNL/bugfix/244-uninstall-errors 
Bugfix/244 uninstall errors
 2015-12-19 08:06:36 -08:00
Todd Gamblin
7cc4d88726 Fix #254: libtool & distutils want certain compiler names. 
This changes the compiler wrappers so that they are called by the same
name as the wrapped compiler.  Many builds make assumptions about
compiler names, and we need the spack compilers to be recognizable so
that build systems will get their flags right.

This adds per-compiler subdirectories to lib/spack/spack/env directory
that contain symlinks to cc for the C, C++, F77, and F90
compilers. The build now sets CC, CXX, F77, and F90 to point to these
links instead of to the generically named cc, c++, f77, and f90
wrappers.
 2015-12-19 02:06:47 -08:00
Todd Gamblin
52e3364efa Fix #244: errors on uninstall 
- Extension logic didn't take conditional deps into account.

- Extension methods now check for whether the extnesion is
  in the extendee map AND whether the dependency is actually present
  in the spec yet.
 2015-12-18 22:24:35 -08:00
Todd Gamblin
b7e926eef6 minor formatting. 2015-12-18 22:24:22 -08:00
Todd Gamblin
ce8bd20944 Fix import error. 2015-12-18 22:23:51 -08:00
Todd Gamblin
abd3d3946d Format to 80 char width. 2015-12-18 19:23:45 -08:00
Gregory Becker
217a2d9ea5 fixed yaml error 2015-12-17 15:26:27 -08:00
Erik Schnetter
5d61888657 Correct Spack cc script 2015-12-17 10:13:39 -05:00
Gregory Lee
9cab88071b Merge pull request #239 from LLNL/bugfix/238-db-specs-not-concrete 
Fix #238: Database specs need to be concrete when read in as well.
 2015-12-16 11:37:51 -08:00
Mike Nolta
82aa366adf minor comment tweak in modules.py 2015-12-16 11:43:07 -05:00
Todd Gamblin
2f9ee88408 Fix #238: Database specs need to be concrete when read in as well. 2015-12-15 17:15:37 -08:00
Todd Gamblin
6e074a196a Fix #206: need to make deps AND root concrete when read in. 2015-12-14 21:06:32 -08:00
Todd Gamblin
afbd0e77d0 Make internal hash dep sort order match external one. 2015-12-11 16:47:34 -08:00
Todd Gamblin
3dd6cbc556 Fix #217: update spec_dag test for new _cmp_key. 2015-12-11 13:07:20 -08:00
Todd Gamblin
6ee2eb21dd Fix #217: Use MUCH faster hashing, reduce number of DAG copies. 
This changes the hash algorithm so that it does much less object
allocation and copying, and so that it is correct.

The old version of `_cmp_key()` would call `sorted_deps`, which would
call `flat_dependencies` to get a list of dependencies so that it
could sort them in alphabetical order.  This isn't necessary in the
`_cmp_key()`, and in fact we want more DAG structure than that to be
included in the `_cmp_key()`.

The new version constructs a tuple without copying the Spec DAG, and
the tuple contains hashes of sub-DAGs that are computed recursively
in-place.  This is way faster than the previous algorithm and reduces
the numebr of copies significantly. It is also a correct DAG hash.

Example timing and copy counts for the different hashing algorithms
we've tried:

Original (wrong) Spec hash:
```
106,170 copies
real    0m5.024s
user    0m4.949s
sys     0m0.104s
```

Spec hash using YAML `dag_hash()`:
```
3,794 copies
real    0m5.024s
user    0m4.949s
sys     0m0.104s

New no-copy, no-YAML hash:
```
3,594 copies
real    0m2.543s
user    0m2.435s
sys     0m0.104s
```

So now we have a hash that is correct AND faster.

The remaining ~3k copies happen mostly during concretization, and as
all packages are initially loaded.  I believe this is because Spack
currently has to load all packages to figure out virtual dependency
information; it could also be becasue there ar a lot of lookups of
partial specs in concretize.  I can investigate this further.
 2015-12-11 12:40:27 -08:00
Mario Melara
b5216f6ec8 Commented out cxxflag. Not 100% sure it's needed 2015-12-11 12:07:32 -08:00
Mario Melara
31df2dd9dd Fixed targets ti be x86_64, important note: have not tested this on linux machine 2015-12-11 12:04:47 -08:00
Mario Melara
fee88d289d Rewrote docstrings for sys_type() and got rid of unused functions 2015-12-11 12:03:13 -08:00
Todd Gamblin
7526a89463 Fix #217: Make package cache use DAG hash instead of sorted deps. 
- Gets rid of last vestige of old-style specs.
- Uses new hashing for lookup
 2015-12-11 02:17:17 -08:00
Tom Scogland
281a869ef6 fix path resolution for mirror packages, especially with dependency fetching 2015-12-10 09:24:01 -08:00
Tom Scogland
786f4cd2c2 add dependency fetching to mirror creation 2015-12-10 08:32:51 -08:00
Tom Scogland
25f2b01a3c fetch: add options to fetch missing or all deps 
Small additions to fetch to make it easier to fetch all files necessary for a
build on a system without network connectivity.
 2015-12-10 07:27:06 -08:00
alalazo
20e67bc5e6 clang : solve the issue with missing default include paths for OpenMP and libc++ 
resource : support for finer grained linking of resources
 2015-12-09 17:15:26 +01:00
Todd Gamblin
537451f0b1 bugfix: doc build needed additional sys.path due to externals change. 2015-12-09 01:32:12 -08:00
Todd Gamblin
c41b9b7ddc Change github.com/scalability-llnl to github.com/llnl everywhere. 2015-12-09 01:24:15 -08:00
Peter Scheibel
530e049d4a Added nose 1.3.7 source along with with the lgpl license (renamed from lgpl.txt 
to LICENSE)
 2015-12-04 17:55:20 -08:00
Massimiliano Culpo
50bd4d2e4e mirror : deals correctly with variants that optionally enable resources (if they are archive URLs) 2015-12-03 11:21:11 +01:00
Mario Melara
028cca16e6 Got rid of the unusued default strategy=None 2015-12-02 10:15:33 -08:00
Massimiliano Culpo
39a3cfd4d9 reource directive accepts 'basename' keyword 
llvm : libc++ variant
 2015-12-02 12:24:37 +01:00
Massimiliano Culpo
4b2168ab8e resources : added missing check after fetch 2015-12-02 10:13:39 +01:00
Mario Melara
d7fdb8e015 Got rid of hard coded strings for adding default 2015-12-01 11:57:54 -08:00
Mario Melara
58f2b39bc8 Changed default to CPU_TARGET env var. Helps deal with target differences between cori and edison 2015-12-01 11:57:17 -08:00
Mario Melara
89fbe4fdfa Changed add_target to include variables rather than hard-coded strings 2015-12-01 11:55:38 -08:00
Massimiliano Culpo
b85dccca92 resources : added error handling for destination keyword on resource directive 2015-12-01 15:12:26 +01:00
Massimiliano Culpo
d3d9b5401b resources : changed name of stage folder 2015-12-01 13:26:44 +01:00
Massimiliano Culpo
a075d581ef resource : fetch strategy constructed from kwargs instead or hardcoded URLFetchStrategy 2015-12-01 12:56:46 +01:00
Todd Gamblin
dfcf567de0 Add a cleaned up repo command. 2015-11-28 19:00:24 -08:00
Todd Gamblin
73ef06018e Integrate namespace attribute into spec, spec DAG, spec YAML. 2015-11-28 16:26:23 -08:00
Todd Gamblin
7383bd393e Fixed bug #42: problem with satisfies() for virtual dependencies. 
- _cross_provider_maps() had suffered some bit rot (map returned was
  ill-formed but still worked for cases with one vdep)

- ProviderIndex.satisfies() was only checking whether the result map
  was non-empty.  It should check whether all common vdeps are *in*
  the result map, as that indicates there is *some* way to satisfy
  *all* of them.  We were checking whether there was some way to
  satisfy *any one* of them, which is wrong.

- Above would cause a problem when there is more than one vdep provider.

- Added test that covers this case.

- Added `constrained()` method to Spec. Analogous to `normalized()`:
  `constrain():constrained() :: normalize():normalized()`
 2015-11-27 23:06:18 -08:00
Todd Gamblin
763d850a0e Add tests for importing packages directly. 
Added test for, e.g.:

    import spack.pkg.builtin.mock.mpich
    import spack.pkg.builtin.mock.mpich as mpich
    from spack.pkg.builtin.mock.mpich import Mpich

Among others.  These ensure that direct package imports work so that
packages can be extended.
 2015-11-26 14:23:12 -08:00
Todd Gamblin
89d5127900 New, cleaner package repository structure. 
Package repositories now look like this:

    top-level-dir/
        repo.yaml
        packages/
            libelf/
                package.py
            mpich/
                package.py
            ...

This leaves room at the top level for additional metadata, source,
per-repo configs, indexes, etc., and it makes it easy to see that
something is a spack repo (just look for repo.yaml and packages).
 2015-11-26 14:19:27 -08:00
Massimiliano Culpo
1fe626ec7c resource directive : sketch of implementation + clang / llvm use case 2015-11-26 17:53:33 +01:00
Peter Scheibel
fc96f62794 Updated documentation/naming in tally_plugin to reflect that the count of the 
total number of tests run excludes skipped tests
 2015-11-25 18:08:17 -08:00
Peter Scheibel
d50a18d9eb XML output for unit tests is now enabled with an option (disabled by default) 2015-11-25 17:10:45 -08:00
Peter Scheibel
60f7756626 Executable.__call__ original semantics were to avoid redirecting process output 
to the output stream if return_output was set. This branch undid that and this
commit restores those semantics.
 2015-11-25 12:28:43 -08:00
Todd Gamblin
04f032d6e3 Rename spack.db -> spack.repo 
Making distinction between install database and package repositories clearer.
 2015-11-25 10:01:44 -08:00
Todd Gamblin
e6d232bfef Rename packages.py to repository.py, PackageFinder to RepoPath. 2015-11-25 09:58:10 -08:00
Peter Scheibel
18dea24df8 Use the xunit nose plugin to generate JUnit XML test results from Spack's unit 
tests.
 2015-11-24 19:08:37 -08:00
Peter Scheibel
0adf93853d Update log_output to handle cases where sys.stdout/stderr streams do not have 
an associated file descriptor (e.g. holds for nose tests where sys.stdout is
assigned to a StringIO object)
 2015-11-24 17:58:17 -08:00
Peter Scheibel
5081ba6802 It appears the same test object was returned multiple times for different 
unit tests, so tracking tests with sets wouldn't work unless I extracted the
details relevant to the particular test. For now a simple count will work so
using a set was unnecessary anyways.
 2015-11-24 10:45:07 -08:00
Peter Scheibel
70049185a5 Activation of Tally plugin to count tests. It doesn't appear to be actually 
counting the individual unit tests correctly so I need to chase that down.
 2015-11-24 10:39:42 -08:00
Peter Scheibel
099fa1df34 Use nose to run unit tests. 
1. Adding a plugin to keep track of the total number of tests run as well as the
number of tests with failures/errors.
2. Some nose plugins (including xunit which will be added in a future commit)
assign stdout to a stream object that does not have a .fileno attribute.
spack.util.executable.Executable now avoids passing stdout to subprocess (and
always uses subprocess.PIPE)

TODO:
1. Still need to figure out how to activate the plugin (as of now it is
being ignored by nose). Newer versions of nose appear to make this simpler
(e.g. the "addplugins" argument to nose.run)
2. Need to include new version of nose in order to use xunit
 2015-11-23 18:59:48 -08:00
Todd Gamblin
cf3d236b9f Merge branch 'scheibelp-features/externaldeps' into develop 2015-11-23 17:50:49 -08:00
Todd Gamblin
1447fd0a12 Merge pull request #160 from SteVwonder/stevwonder/develop 
Fixed -l flag for 'spack extensions'
 2015-11-23 15:50:18 -08:00
Todd Gamblin
8c06b92225 Working PackageFinder / Repo 2015-11-22 20:28:59 -08:00
Tom Scogland
5e93dd6592 modified warning to reference spack clean 
clean --dist has been removed, command fixed to something that works.
 2015-11-20 11:49:46 -08:00
Mario Melara
fceb5a75b0 Adding new files to compilers 2015-11-13 11:53:38 -08:00
Mario Melara
bfd05d3d27 Changed name of file and class to craype to avoid spack freaking out about versions 2015-11-13 11:48:49 -08:00
Mario Melara
72fa3c0492 Added craype support 2015-11-13 11:48:15 -08:00
Mario Melara
9458f7c7d0 Got rid of my old method of finding in LOADEDMODULES 2015-11-13 11:47:36 -08:00
Mario Melara
a5ba69d68d Added a _cmp_key for the architecture class 2015-11-13 11:46:13 -08:00
Todd Gamblin
38fdd063d9 Fix and move NamespaceTrie to spack.util.naming 
- fix up routines in namespace trie.
- trie can now hold intermediate elements.
- trie now has a test case.
 2015-11-12 15:17:39 -08:00
Todd Gamblin
72c9604bcb Small fix to prevent this test from interfering with others. 2015-11-12 15:17:09 -08:00
Todd Gamblin
55662eca69 Merge branch 'develop' into mplegendre-multi_pkgsrc_roots 
Conflicts:
	lib/spack/spack/__init__.py
	lib/spack/spack/directives.py
	lib/spack/spack/packages.py
 2015-11-12 15:16:06 -08:00
Gregory Becker
7c89f9d18c Added the basic ouline of a cray compiler class in lib/spack/spack/compilers/cray.py 2015-11-12 14:31:41 -08:00
Gregory Becker
e78b8c16be used key_comparator decorator from util.lang to compare targets. Fixes find bug 2015-11-12 13:27:10 -08:00
Stephen Herbein
9c30e0210b Fixed -l flag for 'spack extensions' 2015-11-12 08:15:38 -08:00
Peter Scheibel
6d7b26d4e0 Insert lib/spack/external into sys.path. This avoids cases where the system 
python install and lib/spack/external have the same library installed. This
requires modifying the names of some modules in lib/spack/external in cases
where both the system python and backported features of future python versions
(i.e. after 2.6) are used (previously distinguished by "from external import X"
and "import X").
 2015-11-11 18:04:22 -08:00
Gregory Becker
eb2c083159 Fixed the to_dict from_dict for targets on cray. First version to build/find properly on Cray 2015-11-11 17:29:47 -08:00
Gregory Becker
185f40eb8b fixed my git problems 2015-11-11 16:33:22 -08:00
Gregory Becker
3067705c10 made a to_dict for targets. Also a from_dict for sanity 2015-11-11 16:16:47 -08:00
Gregory Becker
8a13d344a8 bug fix in spec.py 2015-11-11 15:07:57 -08:00
Gregory Becker
271a839957 First possibly working version of the crayport. Not sufficiently tested at all. 2015-11-11 14:22:07 -08:00
Gregory Becker
95a34628a3 Add modules to compilers. Changed compiler to take paths as a list. Changed compiler_for_spec to be aware of different compiler stratigies 2015-11-11 11:32:18 -08:00
Gregory Becker
7e6fc79eb2 Merge branch 'features/spackathon' of github.com:NERSC/spack into crayport 
Conflicts:
	lib/spack/spack/compiler.py
 2015-11-11 10:35:59 -08:00
Gregory Becker
5347f460b4 adding module find to find command 2015-11-11 10:31:37 -08:00
Gregory Becker
cd69681ae5 merged in work from other computer and starting to track down bugs 2015-11-11 10:09:03 -08:00
Todd Gamblin
5417a514e9 Eliminated two bugs found by spack test and rewrote the parser to be more sane 2015-11-10 18:44:01 -08:00
Gregory Becker
5e75a5c81c Fixed bug that spack.db.exists() returned True for anonymous specs 2015-11-10 15:45:22 -08:00
Gregory Becker
6f339939c4 Removed "any-pkg-name" and replaced it with empty string. Also changed cflag concretizer to concretize each flag individually, allowing us to have unconcretized FlagMap objects for find and uninstall. Now empty flags in find match any, whereas specifying +cflags=\'\' matches only those with empty strings for flags 2015-11-10 15:45:22 -08:00
Gregory Becker
6fa0bb991a Removed cflags from default format string and made them an option within the compiler string. Added -f option to find command; with -f, find prints flags 2015-11-10 15:45:22 -08:00
Gregory Becker
5a9394c65f added default compilers into spec and fixed constrain/concretize bug 2015-11-10 15:45:22 -08:00
Gregory Becker
cb5bc242db Changed flag default to "". Updated printing and other logic to match. Seems to have solved error in normalize. 2015-11-10 15:45:22 -08:00
Gregory Becker
2d77173dfa partial commit of cflags for debugging 2015-11-10 15:45:22 -08:00
Gregory Becker
42b5b7d2dd Commit of compiler flags addition: 
Flags are passed from the command line all the way through
build environments to environment variables.
      Flags are specified using +name=value and values are quoted
using escaped quotes when necessary.

Future work includes using the flags in the compiler wrapper script
and hopefully updating the parser for a gentler user experience of
the spec language.
 2015-11-10 15:45:22 -08:00
Gregory Becker
db1b21b9aa reclaimed the = sign. Architectures now specified by +arch= instead. Decided to prepend flag names with + for clarity in spec names and ease of parsing. Also generalized variants, although there is not yet a way to specify a generalized (name=value) variant. 2015-11-10 15:45:22 -08:00
Gregory Becker
7989a7f903 Headless specs and /hash spec specification 2015-11-10 15:42:10 -08:00
Gregory Becker
f9c8c4d216 partial commit to merge database 2015-11-10 15:40:00 -08:00
Gregory Becker
4ed22ad932 partial commit of cflags work 2015-11-10 15:39:27 -08:00
Gregory Becker
08729315c6 Added bgq architecture subclass: may require additional functionality to support 2015-11-05 09:21:56 -08:00
Todd Gamblin
0d993947ee Fix SPACK-93, SPACK-94, GitHub #150 
- `remove_prefix` was modified to remove from the DB, but the package
  may not have been added to the DB yet when `remove_prefix` is called
  from `cleanup`.

- Made `remove_prefix` a pure utility function (it just removes the prefix)

- Added `installed_db.remove()` call only after the `remove_prefix` in
  `uninstall`.
 2015-11-04 16:44:33 -08:00
Mario Melara
b61d554dc8 Fixed architecture.py file so it doesn't have the weird merge changes i.e <<<HEAD etc. Also added target method of recognizing it's own architecture 2015-11-04 13:28:12 -08:00
Mario Melara
2bc73b4a27 Merge branch 'features/spackathon' of https://github.com/NERSC/spack into features/spackathon 2015-11-04 13:21:33 -08:00
Gregory Becker
3a73ae1683 Fixed the previous commit 2015-11-04 13:12:11 -08:00
Mario Melara
9bf8e8573c generic linux architecture subclass 2015-11-04 13:08:48 -08:00
Mario Melara
e3a02ea5c7 Merge branch 'features/spackathon' of https://github.com/NERSC/spack into features/spackathon 
Conflicts:
	lib/spack/spack/architecture.py
 2015-11-04 13:06:56 -08:00
Gregory Becker
35532d6b0a Changed cray architecture subclass to add proper targets for front at back end nodes 2015-11-04 13:00:35 -08:00
Mario Melara
37260962e5 changed some potential syntax errors and added a way for target to recognize class 2015-11-04 12:57:29 -08:00
Gregory Becker
5ac974c9b2 Enforced that the architecture subclass cannot add a target that shares a name with a target alias 2015-11-04 12:50:22 -08:00
Todd Gamblin
339da1da3d Make architecture reflect OS *and* machine. Use Python's platform module. 2015-11-04 07:46:17 -08:00
Mario Melara
058e72d29c Added default target and also front end and back end targets 2015-11-02 11:08:55 -08:00
Mario Melara
eb2cf1698f Fixed some issues with naming 2015-11-02 11:08:08 -08:00
Mario Melara
9a91da9ccd Added arch_path to list of modules path 2015-10-30 15:59:28 -07:00
Mario Melara
e39586c81d Reverted back all architecture way of finding modules 2015-10-30 15:58:49 -07:00
Mario Melara
2d87bb92ed Changed front-end and back-end to front_end and back_end to avoid error 2015-10-30 15:58:20 -07:00
Mario Melara
db0695e46a architectures folder where the Architecture subclasses reside 2015-10-30 15:57:00 -07:00
Gregory Becker
6e56070386 improved aliasing in target method in architecture.py 2015-10-30 15:15:36 -07:00
Gregory Becker
d177184777 added target method to architecture.py and minor cleanup 2015-10-30 15:12:29 -07:00
Mario Melara
0a004e0fce Merge branch 'features/spackathon' of https://github.com/NERSC/spack into features/spackathon 
Merge Gregory Becker's commits with my own architecture commits
 2015-10-30 14:53:49 -07:00
Mario Melara
7ab921ff02 Changed architecture class and added class Target 2015-10-30 14:46:26 -07:00
Gregory Becker
382d8478bf prototype of new architecture concretization saved as new_concretize_architecture in concretize.py 2015-10-30 10:23:33 -07:00
Gregory Becker
09597fe8dc updated the executible to return stderr when specified. Added load_module to build_environment.py, loads target 2015-10-30 09:44:28 -07:00
Massimiliano Culpo
6db2a58008 Fixed typo : write_lock() -> write_transaction() 2015-10-30 07:42:58 +01:00
Todd Gamblin
a1c42c2753 Merge Greg Becker's build database support into develop. 2015-10-29 02:22:20 -07:00
Todd Gamblin
671faa4b99 Merge pull request #124 from scheibelp/features/testinstall-cmd 
Features/testinstall cmd
 2015-10-29 00:16:52 -07:00
Todd Gamblin
ed0c80e1cb Merge pull request #145 from markcmiller86/v-char-bugfix 
generalize one of the cases to handle many more kinds of formatting o…
 2015-10-29 00:02:01 -07:00
Todd Gamblin
fec6674597 Merge pull request #139 from mathstuf/use-uname-for-arch 
architecture: use uname if available
 2015-10-29 00:01:30 -07:00
miller86
21e9d5c5c3 generalize one of the cases to handle many more kinds of formatting options 2015-10-27 17:05:27 -07:00
Todd Gamblin
a58ae0c5d0 Build database working with simple transaction support; all tests passing. 2015-10-27 16:36:44 -07:00
Todd Gamblin
bf8479bec6 Fix stupid lock bug. 
- Code simplification ignored case where exception was raised.
- If LockError was raised, read and write counts were incremented erroneously.
- updated lock test.
 2015-10-27 16:34:26 -07:00
Peter Scheibel
50d0a2643b Not all package stages have an archive file (e.g. source code repos) but all of 
them do have a source_path: use this instead to check whether the package
resources were successfully retrieved.
 2015-10-27 13:34:46 -07:00
Todd Gamblin
af7b96c14a Lock acquires return True/False depending on whether they got POSIX lock. 2015-10-27 00:35:06 -07:00
Ben Boeckel
6c9b10f73d architecture: remove custom mac_type method 2015-10-26 18:55:10 -04:00
Ben Boeckel
17a58ee0a9 architecture: use uname if available 2015-10-26 18:54:04 -04:00
Peter Scheibel
3b554c709b Fetch errors were also terminating runs of test-install with system exit, so 
stage.fetch() was updated to raise a FetchError instead of calling tty.die().
Output is the same for spack install in case of a fetch error.
 2015-10-26 15:26:08 -07:00
Ben Boeckel
9d90cb6962 python: use the setdefault method on dict 
It allows more concise code and skips some key lookups.
 2015-10-26 17:38:19 -04:00
Peter Scheibel
9576860f8c Making SpackError reference consistent. 2015-10-26 14:27:44 -07:00
Peter Scheibel
6a16040462 Automatically create a 'test-output' directory in the current directory if no 
output path is specified. Test output files are placed in this directory.
Furthermore the filenames now have the prefix "test" (but otherwise are the
string representation of the spec ID as before).
 2015-10-26 11:58:52 -07:00
Todd Gamblin
ead8ac58c6 Working Lock class, now uses POSIX fcntl locks, extensive unit test. 
- llnl.util.lock now uses fcntl.lockf instead of flock
  - purported to have more NFS compatibility.

- Added an extensive test case for locks.
  - tests acquiring, releasing, upgrading, timeouts, shared, & exclusive cases.
 2015-10-24 19:57:43 -07:00
Todd Gamblin
908a93a470 Add a multiprocess Barrier class to use for testing parallel code. 2015-10-24 19:54:52 -07:00
Peter Scheibel
cc0ee3dc29 The HTML number conversion regex operating against a byte string will only 
convert individual bytes, so therefore incorrectly converts utf-8 encoded
characters. Decoding byte strings to unicode objects results in correct HTML
number encodings.
 2015-10-23 20:56:06 -07:00
Peter Scheibel
d76c923623 1. Rename CommandError -> InstallError 
2. InstallError now subclasses SpackError vs. StandardError (so it is now
handled by the spack shell script)
 2015-10-23 16:18:06 -07:00
Peter Scheibel
ea872f8098 1. Added CommandError exception to build_environment 
2. The parent of a failed child process in build_environment.fork no longer
calls sys.exit - instead it raises a CommandError (from [1])
3. test-install command now attempts to install all packages even if one fails
 2015-10-22 17:44:16 -07:00