Commit graph

2386 commits

Author SHA1 Message Date
Denis Davydov
c9da92dc97 doc: start with known issues section (#3722) 2017-04-10 08:20:57 -05:00
Adam J. Stewart
8e4ada5e99 Make it more clear that version ranges are inclusive (#3759) 2017-04-10 07:25:48 -05:00
Massimiliano Culpo
10c395b2f5 Abinit: a few modernizations for the package and its dependencies (#3699)
* libxc: added libs interface

* hdf5: added libs interface, added conflicts

* abinit: modernized package to use build interface

* netcdf-fortran: added libs interface

* abinit: added version 8.2.2
2017-04-07 11:18:34 +02:00
George Hartzell
84208523f9 set_executable can set S_IX{GRP,OTH} (#3742)
`set_executable` now checks if a user/group.other had read permission
on a file and if it does then it sets the corresponding executable
bit.

See #1483.
2017-04-06 16:25:13 -05:00
Max Katz
a526bcaf11 Correct uninstall -d to uninstall -R in tutorial (#3740)
-d was changed to -R as of PR #1917
2017-04-06 15:44:43 -05:00
Todd Gamblin
715ac8b7e6 Update copyright on the docs. (#3718) 2017-04-05 18:34:17 -07:00
Adam J. Stewart
df150b3e92 spack create MakefilePackage (#3710)
* spack create MakefilePackage

* Change default Perl template to match other build systems
2017-04-05 18:00:42 -05:00
scheibelp
9e50d16f13 Merge pull request #2789 from paulhopkins/features/allow_package_installation_directory_to_be_configured
Allow installation directory layout to be configured
2017-04-05 10:45:11 -07:00
Paul Hopkins
95c3cff6be Allow installation directory layout to be configured using either hash
length parameter or spec formatting.
2017-04-05 12:08:56 +01:00
scheibelp
0038ea84a1 Avoid null reference in arch concretization (#2596)
Fixes #2587

The concretizer falls back on using the root architecture (followed
by the default system architecture) to fill in unspecified arch
properties for a spec. It failed to check cases where the root could
be None.
2017-04-04 15:03:52 -07:00
Massimiliano Culpo
20f388cebd Fixes #3675: Abinit: invalid spec on concretization #(3686) 2017-04-04 07:03:11 -07:00
Justin Cook
508c96d046 Update getting_started.rst (#3685)
Fixed typo referring to .bashrc file
2017-04-04 06:50:44 -05:00
Adam J. Stewart
50df071ad9 Overhaul Spack's URL parsing (#2972)
* Remove fake URLs from Spack

* Ignore long lines for URLs that start with ftp:

* Preliminary changes to version regexes

* New redesign of version regexes

* Allow letters in version-only

* Fix detection of versions that end in Final

* Rearrange a few regexes and add examples

* Add tests for common download repositories

* Add test cases for common tarball naming schemes

* Finalize version regexes

* spack url test -> spack url summary

* Clean up comments

* Rearrange suffix checks

* Use query strings for name detection

* Remove no longer necessary url_for_version functions

* Strip off extraneous information after package name

* Add one more test

* Dot in square brackets does not need to be escaped

* Move renaming outside of parse_name_offset

* Fix versions for a couple more packages

* Fix flake8 and doc tests

* Correctly parse Python, Lua, and Bio++ package names

* Use effective URLs for mfem

* Add checksummed version to mitos

* Remove url_for_version from STAR-CCM+ package

* Revert changes to version numbers with underscores and dashes

* Fix name detection for tbb

* Correctly parse Ruby gems

* Reverted mfem back to shortened URLs.

* Updated instructions for better security

* Remove preferred=True from newest version

* Add tests for new `spack url list` flags

* Add tests for strip_name_suffixes

* Add unit tests for version separators

* Fix bugs related to parseable name but in parseable version

* Remove dead code, update docstring

* Ignore 'binary' at end of version string

* Remove platform from version

* Flip libedit version numbers

* Re-support weird NCO alpha/beta versions

* Rebase and remove one new fake URL

* Add / to beginning of regex to avoid picking up similarly named packages

* Ignore weird tar versions

* Fix bug in url parse --spider when no versions found

* Less strict version matching for spack versions

* Don't rename Python packages

* Be a little more selective, version must begin with a digit

* Re-add fake URLs

* Fix up several other packages

* Ignore more file endings

* Add parsing support for Miniconda

* Update tab completion

* XFAILS are now PASSES for 2 web tests
2017-04-03 15:34:16 -07:00
Massimiliano Culpo
ffef681377 new directive: conflicts() (#3125)
* Add conflicts(<spec>) directive
* openblas: added conflicts for intel@16 refs #3119
* added brief docs and unit tests
2017-04-02 11:40:09 -07:00
Todd Gamblin
0e785e0500 Add epigraph in docs. 2017-04-01 15:32:06 -07:00
Todd Gamblin
221f179716 Add better tests for web.py; fix some bugs found with spidering.
- _spider in web.py was actually failing to spider deeper than a certain
  point.
- Fixed multiprocessing pools to not use daemons and to allow recursive
  spawning.
- Added detailed tests for spidering and for finding archive versions.
- left some xfail URL finding exercises for the reader.
- Fix noqa annotations for some @when decorators
2017-04-01 15:10:45 -07:00
Adam J. Stewart
28d6d375b4 Fix Python 3 support in spack versions
- Add missing import, fixes spack versions in Python 2
- Fix spack versions in Python 3
2017-04-01 15:10:45 -07:00
Kenneth Hoste
fb98f9ee19 fix annoying 'fatal: Not a git repository' error message (#3657)
fix annoying 'fatal: Not a git repository' error message produced by 'spack list' when Spack is not run from a Git repository (#3657)
2017-04-01 03:29:06 -07:00
Gregory Lee
12c1216c34 fix flake8 ignore syntax (use E instead of ignore) (#3651) 2017-03-31 17:51:09 -07:00
Massimiliano Culpo
0c44dd28bb test/package_sanity.py: ported to pytest (#3474) 2017-03-31 13:42:04 -07:00
Massimiliano Culpo
ee7753a597 test/versions.py: ported to pytest (#3635) 2017-03-31 13:41:48 -07:00
Todd Gamblin
a8bcc23fe7 Spack works with Python 3
- Update version guard in spack script to allow python 3
- Update min required version in the docs
2017-03-31 13:40:41 -07:00
Todd Gamblin
3f21f2b088 Clean up tests and add Python3 to Travis.
- Clean up spec_syntax tests: don't dependend on DB order.
- spec_syntax hash parsing tests were strongly dependent on the order the
  DB was traversed.
- Tests now specifically grab the specs they want from the mock DB.
- Tests are more readable as a result.

- Add Python3 versions to Travis tests.
2017-03-31 13:40:41 -07:00
Todd Gamblin
7f3f493024 Fix concretization bugs with virtuals and deptypes.
1. Fix #2807: Can't depend on virtual and non-virtual package

- This is tested by test_my_dep_depends_on_provider_of_my_virtual_dep in
  the concretize.py test.

- This was actually working in the test suite, but it depended on the
  order the dependencies were resolved in. Resolving non-virtual then
  virtual worked, but virtual, then non-virtual did not.

- Problem was that an unnecessary copy was made of a spec that already
  had some dependencies set up, and the copy lost half of some of the
  dependency relationships.  This caused the "can'd depend on X twice
  error".

- Fix by eliminating unnecessary copy and ensuring that dep parameter of
  _merge_dependency is always safe to own -- i.e. it's a defensive copy
  from somewhere else.

2. Fix bug and simplify concretization of deptypes.

- deptypes weren't being accumulated; they were being set on each
  DependencySpec. This could cause concretization to get into an infinite
  loop.

- Fixed by accumulating deptypes in DependencySpec.update_deptypes()

- Also simplified deptype normalization logic: deptypes are now merged in
  constrain() like everything else -- there is no need to merge them
  specially or to look at dpeendents in _merge_dependency().

- Add some docstrings to deptype tests.
2017-03-31 13:40:41 -07:00
Todd Gamblin
fe6f39b662 Use key sorting instead of cmp()
- Get rid of pkgsort() usage for preferred variants.
- Concretization is now entirely based on key-based sorting.
- Remove PreferredPackages class and various spec cmp() methods.
- Replace with PackagePrefs class that implements a key function for
  sorting according to packages.yaml.
- Clear package pref caches on config test.
- Explicit compare methods instead of total_ordering in Version.
- Our total_ordering backport wasn't making Python 3 happy for some
  reason.
- Python 3's functools.total_ordering and spelling the operators out
  fixes the problem.
- Fix unicode issues with spec hashes, json, & YAML
- Try to use str everywhere and avoid unicode objects in python 2.
2017-03-31 13:40:41 -07:00
Todd Gamblin
0cd6555388 Resolve Python2/Python3 unicode issues by using str()
- Remove ascii encoding assumption from spack_yaml
- proc.communicate() returns bytes; convert to str before adding.
- Fix various byte string/unicode issues for Python 2/3 support
- Need to decode subprocess output as utf-8 in from_sourcing_files.
- Fix comments in strify()
2017-03-31 13:40:41 -07:00
Todd Gamblin
1d1a14dbe9 Convert Python 2 idioms to Python 2/3-compatible ones.
- convert print, StringIO, except as, octals, izip
- convert print statement to print function
- convert StringIO to six.StringIO
  - remove usage of csv reader in Spec, in favor of simple regex
  - csv reader only does byte strings
- convert 0755 octal literals to 0o755
- convert `except Foo, e` to `except Foo as e`
- fix a few places `str` is used.
  - may need to switch everything to str later.
- convert iteritems usages to use six.iteritems
- fix urllib and HTMLParser
- port metaclasses to use six.with_metaclass
- More octal literal conversions for Python 2/3
- Fix a new octal literal.
- Convert `basestring` to `six.string_types`
- Convert xrange -> range
- Fix various issues with encoding, iteritems, and Python3 semantics.
- Convert contextlib.nested to explicitly nexted context managers.
- Convert use of filter() to list comprehensions.
- Replace reduce() with list comprehensions.
-  Clean up composite: replace inspect.ismethod() with callable()
- Python 3 doesn't have "method" objects; inspect.ismethod returns False.
- Need to use callable in Composite to make it work.
- Update colify to use future division.
- Fix zip() usages that need to be lists.
- Python3: Use line-buffered logging instead of unbuffered.
- Python3 raises an error with unbuffered I/O
  - See https://bugs.python.org/issue17404
2017-03-31 13:40:41 -07:00
Todd Gamblin
0331b08c64 Update externals to work with Python 3
- Update YAML version to support Python 3
- Python 3 support for ordereddict backport
- Exclude Python3 YAML from version tests.
- Vendor six into Spack.
- Make Python version-check tests work with Python 3
- Add ability to add version check exceptions with '# nopyqver' line
  comments.
2017-03-31 13:40:41 -07:00
Adam J. Stewart
bc404532ea PythonPackage: Let There Be Tests! (#2869)
* Run python setup.py test if --run-tests
* Attempt to import the Python module after installation
* Add testing support to numpy and scipy
* Remove duplicated comments
* Update to new run-tests callback methodology
* Remove unrelated changes for another PR
2017-03-31 13:39:07 -07:00
Michael Kuhn
0d421137f4 Fix mxml (#3639)
mxml is now hosted on GitHub.
2017-03-31 13:41:08 -05:00
Milton Woods
9e43ff821c Extendable Perl (#3614)
* perl: make extendable and add Module::Build package
* perl: allow 'spack create' to identify perl packages from their contents
* perl-module-build: fix indenting of package docstring
* perl: split install() method for extensions into phases
* perl: auto-detect build method (Makefile.PL vs Build.PL) and define a 'check' method
* PerlPackage: use import statements similar to those in AutotoolsPackage
* PerlModule: fix detection of Build.PL
* PerlPackageTemplate: remove extraneous lines to avoid flake8 warnings
* PerlPackageTemplate: split into separate templates for Makefile.PL and Build.PL
* PerlPackage: add cross-references to docstrings
* AutotoolsPackage: fix ambiguous cross-references to avoid errors in doc tests
* PerlbuildPackageTemplate: depend on perl-module-build if Build.PL exists
2017-03-30 16:38:58 -07:00
Todd Gamblin
782f29bc4b Fix bug in spack find for installations with unknown namespaces. (#3573)
- Spack find would fail with "unknown namespace" for some queries when a
  package from an unknown namespace was installed.

- Solve by being conservative: assume unknown packages are NOT providers
  of virtual dependencies.
2017-03-28 08:43:43 -07:00
Todd Gamblin
e549daa8ce Bugfix: allow deactivating installs that don't have packages anymore. (#3572)
- deactivate -a wouldn't work if the installation's package was no longer
  available.

- Fix installed_extensions_for so that it doesn't need to look at the
  package.py file.
2017-03-28 08:43:15 -07:00
Denis Davydov
61d01a71dd make git fetching quite (#3180)
also output repo and branch/commit/tag in one line
2017-03-24 17:37:47 -07:00
Adam J. Stewart
4ecfc39e1e Add Expect package (#3517)
* Add Expect package

* Ignore patches during flake8 tests for package.py files

* Remove controversial changes
2017-03-23 14:35:11 -05:00
George Hartzell
9b5f5fccf0 Fix for find --explicit #3374 (#3492)
This fixes the problem described in #3374, which describes `spack find` ignore explicit/implicit.

I believe that this was broken in #2626.

This restores the behavior of implicit/explicit for me.

I believe that it does not screw anything else up, but ....
2017-03-21 15:30:38 -07:00
scheibelp
19dca26f3a Improve output for compiler commands (#3480)
* Order listed compiler sections

"spack compiler list" output compiler sections in an arbitrary order.
With this commit compiler sections are ordered primarily by compiler
name and then by operating system and target.

* Compiler search lists config files with compilers

If a compiler entry is already defined in a configuration file that
the user does not know about, they may be confused when that compiler
is not added by "spack compiler find". This commit adds a message at
the end of "spack compiler find" to inform the user of the locations
of all config files where compilers are defined.
2017-03-20 12:36:44 -07:00
Matthew LeGendre
5ac6421f14 Fix issue with config.guess patching when the overwritten config.guess did not have write permissions. (#3494) 2017-03-20 12:35:38 -07:00
Todd Gamblin
1297e47463 Use byte-encoded UTF-8 for sourced environment in Python 2 (#3489)
- Fixes recurring errors on develop with unicode commit characters.
- still Python3-proof: python3 will use str instead of bytestrings.
2017-03-18 22:48:20 -07:00
Adam J. Stewart
10b767b93f Don't use @system in packages.yaml (#3472) 2017-03-17 07:55:55 -07:00
scheibelp
0b27a7e13d Detect when OS updates affect compiler selection (#3410)
Fixes #1476

Concretization uses compilers defined in config files and if those
are not available defaults to searching typical paths where the
detected operating system would have a compiler. If there is an OS
update, the detected OS can change; in this case all compilers
defined in the config files would no longer match (because they would
be associated with the previous OS version). The error message in
this case was too vague. This commit adds logic for detecting when it
is likely that the OS has been updated (in particular when that
affects compiler concretization) and improves the information provided
to the user in the error message.
2017-03-16 21:08:13 -07:00
scheibelp
5936ad2ca7 Dont propagate flags between different compilers (#3379)
* Dont propagate flags between different compilers

Fixes #2786

Previously when a spec had no parents with an equivalent compiler,
Spack would default to adding the compiler flags associated with the
root of the DAG. This eliminates that default.

* added test for compiler flag propagation

* simplify compiler flag propagation logic
2017-03-16 21:07:35 -07:00
scheibelp
9a27dec8e8 Dont auto-init compiler conf for 'compiler find' (#3439)
Fixes #3428

Users can run 'spack compiler find' to automatically initialize their
compilers.yaml configuration file. It also turns out that Spack will
implicitly initialize the compilers configuration file as part of
detecting compilers if none are found (so if a user were to attempt to
concretize a spec without running 'spack compiler find' it would not
fail). However, in this case Spack was overlooking its own implicit
initialization of the config files and would report that no new
compilers were found. This commit removes implicit initialization when
the user calls 'spack compiler find'.

This did not surface until #2999 because the 'spack compiler' command
defaulted to using a scope 'user/platform' that was not accounted for
in get_compiler_config (where the implicit initialization logic
predates the addition of this new scope); #2999 removed the scope
specification when checking through config files, leading to the
implicit initialization.
2017-03-16 11:24:34 -07:00
Adam J. Stewart
dca4d2b15e Consistent docs and usage of env mod methods (#3351) 2017-03-14 22:26:44 -07:00
Denis Davydov
560d28ac7f fix automatic mixing of clang with gfortran 6.3.0 on macOS (#3427)
* fix automatic mixing of clang with gfortran 6.3.0 on macOS
* automatically mix any gfortran with any clang on macOS
* adjust the unit test
2017-03-14 09:48:27 -07:00
Massimiliano Culpo
8c3edfd36f test/file_cache.py: ported to pytest (#3429) 2017-03-14 09:07:04 -07:00
Todd Gamblin
d11e262b36 Fix from_sourcing_files(): decode json input as utf-8 (#3433) 2017-03-13 15:52:36 -07:00
Erik Schnetter
63a8f79980 Correct inconsistency in comment (#3414) 2017-03-11 12:52:08 -06:00
Todd Gamblin
e3101808ae Make multimethods work with inheritance. (#3411)
Previously, this would fail with a NoSuchMethodError:

    class Package(object):
        # this is the default implementation
        def some_method(self):
            ...

    class Foo(Package):
        @when('platform=cray')
        def some_method(self):
            ...

        @when('platform=linux')
        def some_method(self):
            ...

This fixes the implementation of `@when` so that the superclass method
will be invoked when no subclass method matches.

Adds tests to ensure this works, as well.
2017-03-11 05:48:36 -08:00
scheibelp
e727f56d89 Features/compiler config consistency (#2999)
* default scope for config command is made consistent with cmd/__init__ default

* dont specify a scope when looking for compilers with a matching spec (since compiler concretization is scope-independent)

* config edit should default to platform-specific file only for compilers

* when duplicate compiler specs are detected, the exception raised now points the user to the files where the duplicates appear

* updated error message to emphasize that a spec is duplicated (since multiple specs can reference the same compiler)

* 'spack compilers' is now also broken down into sections by os and target

* Added tests for new compiler methods
2017-03-10 13:58:48 -08:00
Gregory Lee
604b75c1f9 created elf virtual package and updated dependent packages (#3317)
* created elf virtual package and updated dependent packages
* added `hide_files` context manager to handle moving files.
2017-03-09 10:36:32 -08:00
Massimiliano Culpo
f1d66467e3 package.py: packages dump build dependencies in prefix (#3373)
Modifications:
- `dump_packages` copies build dependencies into `$prefix/.spack`, as well as the link/run dependencies that we already copied there.
- fake installs copy dependency packages into `$prefix/.spack` as well
2017-03-07 07:05:45 -08:00
Massimiliano Culpo
c6d9a45f18 test/environment.py: ported to pytest, added a test on separators (#3375) 2017-03-07 07:03:12 -08:00
Adam J. Stewart
a2d70a45fb Allow find_libraries to accept lists or strings (#3363)
* Allow find_libraries to accept lists or strings

* Convert one more example from list to string
2017-03-04 11:23:57 -06:00
Joschka Lingemann
7ef95767ee Fix error raised for multiple virtual packages. (#3306) 2017-03-03 16:30:32 -06:00
Massimiliano Culpo
ed582cef68 New interface for passing build information among specs (#1875)
- Added a new interface for Specs to pass build information
  - Calls forwarded from Spec to Package are now explicit
  - Added descriptor within Spec to manage forwarding
  - Added state in Spec to maintain query information
  - Modified a few packages (the one involved in spack install pexsi) to showcase changes

- This uses an object wrapper to `spec` to implement the `libs` sub-calls.
  - wrapper is returned from `__getitem__` only if spec is concrete
  - allows packagers to access build information easily
2017-03-02 10:01:29 -08:00
Massimiliano Culpo
5ce926d2d1 test/packages: fixed test suite (#3236)
It seems the tests in `packages.py` were running just because we had a specific order of execution. This should fix the problem, and make the test_suite more resilient to running order.
2017-03-02 09:29:23 -08:00
Javier
d49cb2734f Fixing issue wheh overwriting build_args in PythonPackage (#3200) 2017-02-21 08:04:32 -06:00
Adam J. Stewart
6d3f649382 Fix readline support in spack python (#3132)
* Fix readline support in `spack python`
* Add documentation on `spack python`
* Add unit tests for `spack python`
2017-02-17 15:58:06 -08:00
Adam J. Stewart
524303db06 Set default module type based on modules.yaml (#3173) 2017-02-17 13:46:31 -08:00
Adam J. Stewart
e492aff4f7 More consistent yes/no prompts (#3174)
* More consistent yes/no prompts
* Add ==> prefix to yes/no and number prompts
2017-02-17 13:45:02 -08:00
Elizabeth Fischer
29d070e50c On uninstall, change shortcut flag for --dependents to -R, as per Spack convention. (-r = --dependencies, -R = --dependents). (#1917) 2017-02-17 12:08:17 -08:00
Adam J. Stewart
38dd01f6b5 Fix missing space in error message output (#3169) 2017-02-17 12:02:48 -08:00
Elizabeth Fischer
682d5cf164 py-git2: New Package (#1971) 2017-02-17 11:55:22 -08:00
Adam J. Stewart
a9cf99b3e2 Fix typos in External Packages documentation (#3168) 2017-02-17 11:49:55 -08:00
Massimiliano Culpo
e24fdb49ea fix: don't call setup_environment when not needed (#3060)
* Don't call setup_environment when not needed. fixes #3059

* setup_environment and modules: added unit tests
2017-02-10 16:09:43 -08:00
becker33
f9e3b58d7e Make distro more robust to unreadable files (#3110)
* Make distro more robust to unreadable files. Will upstream

* Comment for clarify
2017-02-10 10:23:04 -08:00
Denis Davydov
b1eb921f28 make svn fetch quiet (#3070) 2017-02-09 13:55:18 -08:00
Todd Gamblin
b192011add Revert "db and concretization of packages modified after installation: fixes #2911" (#3078) 2017-02-08 13:12:10 -08:00
Massimiliano Culpo
79c1f7885a db and concretization of modified packages: fixes #2911 (#2920) 2017-02-07 18:28:01 -08:00
becker33
968199de7d Fix spec hash printing (#2941)
- Fix format printing to match command line for hashes and full name formats
- Update spack graph to use new format
- Changed format string signifier for hashes from `$#` to `$/`
2017-02-03 18:27:34 -08:00
George Hartzell
c456dfb60f Make module autoload warnings configurable (#2763)
Modules generated by the module creation machinery currently print out
a notice that warnts the user that things are being autoloaded.  In
some situations those warnings are problematic.  See #2754 for
discussion.

This is a first cut at optionally disabling the warning messages:

- adds a helper tothe EnvModule base class that encapsulates the
  config file variable;
- adds a method to the base class that provides a default (empty)
  code fragment for generating a warning message;
- passes the warning fragment into the bit that formats the autoload
  string;
- adds specialized autload_warner() methods in the tcl and lmod
  subclasses;; and finally
- touches up the autoload_format strings in the specialized classes.
2017-02-03 15:53:30 -08:00
George Hartzell
5a836fd06e Blacklist implicit packages for modulefile generation (#2603)
Add the ability to the modules generation process to blacklist
packages that were installed implicitly.  One can still whitelist
modules that were installed implicitly.

This changes adds a `blacklist_implicts` boolean as a peer to the
`whitelist` and `blacklist` arrays, e.g.:

```
modules:
  enable::
    - lmod
  lmod:
    whitelist:
      - 'lua'
      - 'py-setuptools'
    blacklist:
      -  '%gcc@4.8.3'
    blacklist_implicits: True
```

It adds a small helper in `spec.py` and then touches up the package
filtering code in `modules.py`.
2017-02-03 15:42:46 -08:00
serbanmaerean
8275b44715 Change default fflags for XL compiler to only "-qzerosize" (#2966) 2017-01-31 14:51:05 -05:00
Adam J. Stewart
41c77d7429 Add installcheck phase to AutotoolsPackage (#2863)
* Add installcheck phase to AutotoolsPackage
* Update installcheck phase with new callbacks API
* build_directory has been converted to a property
2017-01-31 14:35:38 -05:00
Henrik Bengtsson
6e729bc333 DOCS: Examplify what file types are automatically extracted (#2955) 2017-01-31 08:26:19 -08:00
Adam J. Stewart
123f057089 Refactor Spack's URL parsing commands (#2938)
* Replace `spack urls` and `spack url-parse` with `spack url`
* Allow spack url list to only list incorrect parsings
* Add spack url test reporting
* Add unit tests for new URL commands
2017-01-31 11:14:52 -05:00
Massimiliano Culpo
2babe4e4ff removed call syntax from property fixes #2944 (#2945) 2017-01-31 10:48:57 -05:00
Adam J. Stewart
0c0a1fdedc Add several new R packages (#2952)
* Add several new R packages

* Add a few more R packages

* Update more versions

* Convert Package to RPackage

* Add a few more packages

* Add missing dependencies
2017-01-31 09:54:34 -05:00
becker33
58a4c5c14d Fix ambiguous hash message generation. (#2940)
* Fix ambiguous hash message generation. Engineering fix
2017-01-27 11:37:21 -08:00
Adam J. Stewart
6158115ca8 Standardize argparse help messages (#2847) 2017-01-26 02:33:01 -08:00
Todd Gamblin
ffb8aaa3f3 Packaging docs for MPI (#2838)
* Add MPI docs to packaging guide and simplify packaging guide TOC a bit.
2017-01-26 02:29:31 -08:00
Massimiliano Culpo
81a5146b1d AutotoolsPackage: minor improvements (#2859)
* AutotoolsPackage: added configure_directory to permit build out of source. The configure script executable is now invoked with an absolute path. Modified a few packages accordingly.

* build_systems: functions returning directories are now properties

* build_systems: fixed issues with tcl and tk

* AutotoolsPackage: reworked recipe for autoreconf
2017-01-26 02:27:15 -08:00
Todd Gamblin
a5700a8888 Use codecov for coverage instead of coveralls (#2933)
* Switch from coveralls to codecov

- Add .codecov.yml, simplify .travis.yml
- Add codecov badge to README.md

* Add tests for spack graph.
2017-01-26 02:19:35 -08:00
scheibelp
e4d2d747ce Spec.satisfies accesses Spec.concrete as property (#2928)
* Spec.satisfies accesses Spec.concrete as property

Fixes #2760

When copying a spec, _concrete is always set to False for each
dependency. "Spec.satisfies" was accessing the member "_concrete"
directly instead of using the property "concrete". This means that
if you copy a spec, the dependencies will be considered equal, but
did not necessarily satisfy one another. Spec.satisfies is a
prerequisite for a package to be considered an extension; as a
consequence, an extension with run-time dependencies that were also
extensions did not activate those extensions. This updates
Spec.satisfies to avoid checking the cached member "_concrete"
directly.

* Added test to check for activation of dependency extension

* Added test to check for transitive satisfiability between a spec and its copy
2017-01-25 21:43:12 -07:00
becker33
8ae380fb71 Fixes for parsing specs with hashes (#2889)
- Allows hashes to be specified after other parts of the spec
- Does not allow other parts of the spec to be specified after the hash
- The hash must either end input or be followed by another separate spec
- The next spec cannot be an anonymous spec (it must start with a package name or a hash)

See #2769 (after it was merged) for further discussion of this interface addition. That discussion resulted in these requirements:

```
python                     # 1 spec
/abc123                    # 1 spec
python /abc123             # 1 spec
/456789                    # 1 spec
python /abc123 /456789     # 2 specs
python /456789 /abc123     # 2 specs
/abc123 /456789            # 2 specs
/456789 /abc123            # 2 specs
/456789 /abc123 python     # 3 specs
```

assuming `abc123` and `456789` are both hashes of different python specs.
2017-01-25 21:38:10 -07:00
serbanmaerean
5e2a96574b Add support for IBM threaded compilers: xl*_r (#2894)
* Add support for IBM threaded compilers, xl*_r

Added new compiler class, xl_r; added default flags to the compilers.yaml file.

* Add cppflags to the set of default flags to be added to the compilers stanza in compiler.yaml.

These flags are optional. Only defined flags will be listed in the compilers.yaml file.

* Fix scripting warnings revealed by flake8.

Updated __init__.py and xl_r.py to conform with flake8 rules.

* Add justification to the definition of the XL default compiler flags.
2017-01-25 16:26:17 -07:00
Massimiliano Culpo
fc866ae0fe build systems: simpler, clearer decorators: run_after, run_before (#2860)
* PackageMeta: `run_before` is an alias of `precondition`, `run_after` an alias of `sanity_check`

* PackageMeta: removed `precondition` and `sanity_check`

* PackageMeta: decorators are now free-standing

* package: modified/added docstrings. Fixed the semantics of `on_package_attributes`.

* package: added unit test assertion as side effects of install

* build_systems: factored build-time test running into base class

* r: updated decorators in package.py

* docs: updated decorator names
2017-01-25 08:57:01 -07:00
Adam J. Stewart
7d3da2ebdc Grammar and broken link fixes in modules tutorial (#2912) 2017-01-24 10:25:44 -07:00
Massimiliano Culpo
a8e1d78881 documentation: build-system phases + build-time tests (#2780)
* documentation: reworked packaging guide to add build-system phases

* documentation: improvements to AutotoolsPackage autodocs

* build_systems: updated autodocs

* run-tests: added a few information on how to run tests fixes #2606 fixes#2605

* documentation: fixed items brought up by @davydden

    * typos in docs
    * consistent use of 'build system' (i.e. removed 'build-system' from docs)
    * added a note on possible default implementations for build-time tests

* documentation: fixed items brought up by @citibeth

    * added note to explain the difference between build system and language used in a package
    * capitalized bullet items
    * added link to API docs

* documentation: fixed multiple cross-references after rebase

* documentation: fixed minor issues raised by @tgamblin

* documentation: added entry in table for the `PythonPackage` class

* docs: fixed issues brought up by @citybeth in the second review
2017-01-23 13:55:39 -08:00
Adam J. Stewart
dae353374e Simplify unit tests listed in Contribution Guide (#2904)
* Simplify unit tests listed in Contribution Guide

* Use long name for option flags
2017-01-23 09:48:49 -08:00
Erik Schnetter
cade0181fd Spack: Correct fix_darwin_install_name (#2886)
Previously, fix_darwin_install_name would only handle dependencies that have no path set, and it ignore dependencies that have the build directory as path baked in. Catch this, and replace it by the install directory.
2017-01-20 10:22:59 -08:00
Adam J. Stewart
4dad5aab84 Add spack edit option for build systems (#2865) 2017-01-18 18:49:48 -08:00
Adam J. Stewart
02f92fc7f8 Massive conversion from Package to AutotoolsPackage (#2845)
* Massive conversion from Package to AutotoolsPackage

* Forgot to convert p4est to AutotoolsPackage

* Fix typo

* Fix broken link in docs
2017-01-18 10:34:09 -08:00
Adam J. Stewart
f0f230d480 Allow spack create to detect packages that need to run autoreconf (#2848) 2017-01-17 17:42:45 -08:00
Jason Sarich
a1f7006400 check if node is already deleted (#2799)
* check if node is already deleted

* fix variable name
2017-01-17 14:23:32 -08:00
Adam J. Stewart
061076aea9 Support scionlib version detection (#2818) 2017-01-17 10:54:36 -08:00
Todd Gamblin
1856317777 Bump version to 0.10.0 2017-01-16 20:01:07 -08:00
Adam J. Stewart
c0aaa8fcea Add PythonPackage base class
- Add a PythonPackage class with build system support.
  - Support build phases in PythonPackage
  - Add a custom sanity check for PythonPackages
  - Get rid of nolink dependencies in python packages

- Update spack create to use new PythonPackage class

- Port most of Python packages to new PythonPackage class

- Conducted a massive install and activate of Python packages.
  - Fixed bugs introduced by install and activate.

- Update API docs on PythonPackage
2017-01-16 20:00:31 -08:00
Adam J. Stewart
1f49493fee Major improvements to spack create (#2707)
* Initial changes to spack create command

* Get 'spack create <url>' working again

* Simplify call to BuildSystemGuesser

* More verbose output of spack create

* Remove duplicated code from spack create and spack checksum

* Add better documentation to spack create docstrings

* Fix pluralization bug

* Flake8

* Update documentation on spack create and deprecate spack edit --force

* Make it more obvious when we are renaming a package

* Further deprecate spack edit --force

* Fix unit tests

* Rename default template to generic template

* Don't add automake/autoconf deps to Autotools packages

* Remove changes to default $EDITOR

* Completely remove all traces of spack edit --force

* Remove grammar changes to make the PR easier to review
2017-01-16 17:13:12 -08:00
Adam J. Stewart
6eed279fce Don't duplicate spack test in spack help (#2837) 2017-01-16 14:56:38 -08:00
George Hartzell
aa3cc0ea81 Add lmod support for spack module loads (#2758)
The `spack module loads` command only supported tcl and dotkit.
This adds support for lmod.
2017-01-16 03:25:39 -08:00
becker33
a091eeceab Parser fix (#2769)
* Fixed parser to eliminate need for escape quotes. TODO: Fix double call to shlex, fix spaces in spec __str__

* Fixed double shlex

* cleanup

* rebased on develop

* Fixed parsing for multiple specs; broken since #360

* Revoked elimination of the `-` sigil in the syntax, and added it back into tests

* flake8

* more flake8

* Cleaned up dead code and added comments to parsing code

* bugfix for spaces in arguments; new bug found in testing

* Added unit tests for kv pairs in parsing/lexing

* Even more flake8

* ... yet another flake8

* Allow multiple specs in install

* unfathomable levels of flake8

* Updated documentation to match parser fix
2017-01-15 19:17:54 -08:00
Adam J. Stewart
f480e3449e Added customization for make targets in 'build' and 'install' phases for CMakePackage (#2742)
* Added customization for make targets in 'build' and 'install' phases for CMakePackage

* Use rst in build system docs so that Sphinx generates nice API docs

* Allow AutotoolsPackages to be built in a different directory

* Flake8

* Fix missing import

* Allow configure to be located in different directory

* Update espressopp to use build targets

* Flake8

* Sphinx fix, lists must be a new paragraph

* Back out change that allowed a configure script in a different directory than build_directory

* Add missing deps, build in parallel

* Missing space for rst list
2017-01-15 16:23:16 -08:00
Adam J. Stewart
f2d4c15395 Remove cmake build dep from CMakePackages (#2684) 2017-01-15 03:18:52 -08:00
Todd Gamblin
0d367bbd20 Add a test to exercise non-buildable external packages. (#2833) 2017-01-15 02:37:20 -08:00
becker33
677623962e Fixed find -lp command (coloring was broken) (#2820) 2017-01-15 02:36:36 -08:00
Massimiliano Culpo
436f6a4ab6 get_std_cmake_args delegates to CMakePackage._std_args fixes #2665 (#2805) 2017-01-15 02:34:15 -08:00
Massimiliano Culpo
cd511791d1 configure and build command: fix the stop iteration semantics
- broke somewhere between bdf4832269 (#2810) and 17b13b161b
2017-01-12 09:11:48 -08:00
Massimiliano Culpo
de2cb864fa mirror: fix error for some specs during concretization (#2806) 2017-01-11 07:38:22 -08:00
Todd Gamblin
f5557b5723 Change nargs='*' to argparse.REMAINDER for constraint options. (#2798) 2017-01-10 12:25:41 -08:00
Elizabeth Fischer
402dfe30f9 Get Rid of nobuild, nolink, and alldeps (#2765)
* Removing the nobuild, nolink, and alldeps dependency types in favor of being explicit.
* This will help with maintenance going forward, as adding more dependency types won't affect existing declared dependencies in weird ways.
* default deptype is still `('build', 'link')`
2017-01-07 19:59:02 -08:00
Adam J. Stewart
daff3c0908 Preliminary RPackage class (#2761) 2017-01-07 16:28:52 -08:00
Axel Huebl
05d77917fd GitLab: Tarball Version Test (#2296)
* GitLab: Tarball Version Test

Upload a test demonstrating #2290

* Add GitLab parsing
2017-01-06 08:53:19 -08:00
Mario Melara
3d54c3c689 Correct env vars for modulecmd; Fixes #1481 (#2745)
Don't override MODULEPATH with PATH, PATH has no modulefiles

PEP8 compliance
2017-01-05 11:15:38 -08:00
Massimiliano Culpo
9271c69b5d test: fixed 'refresh_builtin_mock' fixture (#2748) 2017-01-05 10:25:24 -08:00
Elizabeth Fischer
3dd4a01a5e Standardize package names: lower-case, not Mixed_CASE (#2475)
* Rename packages

* Upcasing depends_on() in packages.

* Downcased extends('r')

* Fixed erroneously changed URL that had slipped through.

* Fixed typo

* Fixed link from documentation into package source code.

* Fixed another doc problem.

* Changed underscores to dashes in package names.

* Added test to enforce lowercase, no-underscore naming convention.

* Fix r-xgboost

* Downcase more instances of 'R' in package auto-creation.

* Fix test.

* Converted unit test packages to use dashes not underscores

* Downcase `r` in the docs.

* Update module_file_support.rst

Fix r->R for class R.
2017-01-04 18:24:07 -08:00
Todd Gamblin
d32d5e45fb Fix issues when a package provides the same vdep twice. (#2710)
* Fix issues when a package provides the same vdep twice.

- provides() now adds to a set of provided vdeps instead of a single one.

* flake8
2017-01-02 18:40:57 -08:00
Todd Gamblin
b9ec69dce1 Disallow vdeps in packages.yaml (#2699)
* Consolidate packages.yaml code to preferred_packages

* Add validation check and a test for packages.py parsing.

* flake8
2016-12-31 08:12:38 -08:00
Adam J. Stewart
8cb54a9a67 Fix missing debug parameter for parse_version() (#2696) 2016-12-30 22:52:49 -08:00
Todd Gamblin
c88df5819d Fix test command arg handling. 2016-12-30 17:18:19 -08:00
Michael Kuhn
a42f340d37 Allow setting default variants (#2644) 2016-12-30 14:11:39 -08:00
Todd Gamblin
972a277956 Fix Python issue with build failures; Add test case for build failures. (#2687) 2016-12-30 11:37:50 -08:00
Todd Gamblin
5fbab1f4b5 Concretize preserves deptypes (#2681)
Concretization preserves deptypes
2016-12-29 14:43:59 -08:00
Todd Gamblin
d6390c159f Fix issues related to the switchover to pytest. (#2685)
- Remove stale reference to `import nose` from `bin/spack` script.
- Add `py` to externals (missing dependency for `pytest`)
2016-12-29 11:59:45 -08:00
Massimiliano Culpo
7ea10e768e unit tests: replace nose with pytest (#2502)
* Porting: substitute nose with ytest

This huge commit substitutes nose with pytest as a testing system. Things done here:

* deleted external/nose as it is no longer used
* moved mock resources in their own directory 'test/mock/'
* ported two tests (cmd/find, build_system) to pytest native syntax as an example
* build_environment, log: used monkeypatch instead of try/catch
* moved global mocking of fetch_cache to an auto-used fixture
* moved global mocking from test/__init__.py to conftest.py
* made `spack test` a wrapper around pytest
* run-unit-tests: avoid running python 2.6 tests under coverage to speed them up
* use `pytest --cov` instead of coverage run to cut down testing time

* mock/packages_test: moved mock yaml configuration to files instead of leaving it in the code as string literals

* concretize.py: ported tests to native pytest, reverted multiprocessing in pytest.ini as it was creating the wrong report for coveralls

* conftest.py, fixtures: added docstrings

* concretize_preferences.py: uses fixtures instead of subclassing MockPackagesTest

* directory_layout.py: uses fixtures instead of subclassing MockPackagesTest

* install.py: uses fixtures instead of subclassing MockPackagesTest

* optional_deps.py: uses fixtures instead of subclassing MockPackagesTest

optional_deps.py: uses fixtures instead of subclassing MockPackagesTest

* packages.py: uses fixtures instead of subclassing MockPackagesTest

* provider_index.py: uses fixtures instead of subclassing MockPackagesTest

* spec_yaml.py: uses fixtures instead of subclassing MockPackagesTest

* multimethod.py: uses fixtures instead of subclassing MockPackagesTest

* install.py: now uses mock_archive_url

* git_fetch.py: uses fixtures instead of subclassing MockPackagesTest

* hg_fetch.py: uses fixtures instead of subclassing MockPackagesTest

* svn_fetch.py, mirror.py: uses fixtures instead of subclassing MockPackagesTest
repo.py: deleted

* test_compiler_cmd.py: uses fixtures instead of subclassing MockPackagesTest

* cmd/module.py, cmd/uninstall.py: uses fixtures instead of subclassing MockDatabase

* database.py: uses fixtures instead of subclassing MockDatabase, removed mock/database

* pytest: uncluttering fixture implementations

* database: changing the scope to 'module'

* config.py: uses fixtures instead of subclassing MockPackagesTest

* spec_dag.py, spec_semantics.py: uses fixtures instead of subclassing MockPackagesTest

* stage.py: uses fixtures instead of subclassing MockPackagesTest. Removed mock directory

* pytest: added docstrings to all the fixtures

* pytest: final cleanup

* build_system_guess.py: fixed naming and docstrings as suggested by @scheibelp

* spec_syntax.py: added expected failure on parsing multiple specs closes #1976

* Add pytest and pytest-cov to Spack externals.

* Make `spack flake8` ignore externals.

* run-unit-tests runs spack test and not pytest.

* Remove all the special stuff for `spack test`

- Remove `conftest.py` magic and all the special case stuff in `bin/spack`

- Spack commands can optionally take unknown arguments, if they want to
  handle them.

- `spack test` is now a command like the others.

- `spack test` now just delegates its arguments to `pytest`, but it does
  it by receiving unknown arguments and NOT taking an explicit
  help argument.

* Fix error in fixtures.

* Improve `spack test` command a bit.

- Now supports an approximation of the old simple interface
- Also supports full pytest options if you want them.

* Use external coverage instead of pytest-cov

* Make coverage use parallel-mode.

* change __init__.py docs to include pytest
2016-12-29 07:48:48 -08:00
Elizabeth Fischer
88f57d7543 Allow exclusion of packages from spack module loads (#2667)
* Allow exclusion of packages from `spack module loads`

* Comment out excluded packages instead of not showing them at all.
2016-12-28 13:31:18 -08:00
Massimiliano Culpo
17b13b161b Directive inheritance: laziness for the win (#2623)
* inheritance of directives: using meta-classes to inject attributes coming from directives into packages + lazy directives

* _dep_types -> dependency_types
* using a meta-class to inject directives into packages
* directives are lazy

fixes #2466

* directives.py: allows for multiple inheritance. Added blank lines as suggested by @tgamblin

* directives.py: added a test for simple inheritance of directives

* Minor improvement requested by @tgamblin

CMakePackage: importing names from spack.directives
directives: wrap __new__ to respect pep8

* Refactoring requested by @tgamblin

directives: removed global variables in favor of class variables. Simplified the interface for directives (they return a callable on a package or a list of them).
2016-12-28 12:37:02 -08:00
Adam J. Stewart
29bac34c1d Ensure that every file in Spack has a license (#2659)
* Ensure that every package has a license

Also fixes URLs with http://http:// doubled.

This is a continuation of #2656.

* Add license to every file in Spack

* Make sure Todd is the author of all packages

* Fix flake8 tests

* Don't license external Sphinx docs

* Don't display licenses in tutorial example packages

Also fixes typos and converts command-line examples
from tcsh to bash, which is more common
2016-12-27 00:17:12 -08:00
Adam J. Stewart
f34c5367d5 Better tabulation for spack checksum (#2673) 2016-12-25 19:25:53 -08:00
Denis Davydov
91bfebbed8 filter_system_path: additionally filter system prefixes (#2672)
That's because in set_build_environment_variables()
the funciton filter_system_paths() is actually applied to
package prefixes, and not to prefix/lib or prefix/include.
2016-12-25 19:24:56 -08:00
Todd Gamblin
c4afaabea9 Add argument to spack spec to show deptypes. (#2680)
- `-t` | `--types` argument now shows deptypes in `spack spec`
2016-12-25 14:15:58 -08:00
Adam J. Stewart
7e8767e3fd Remove usernames from paths in docs (#2675)
* Remove usernames from paths in docs

* Fix path export
2016-12-25 12:36:35 -08:00
Adam J. Stewart
c3cd948537 Modify github archive regex to support luaposix (#2677) 2016-12-25 12:32:19 -08:00
scheibelp
3f1cdbf5ef Revert #2292: use frontend compiler for build deps (#2549)
The primary goal of #2292 was to use the frontend compiler to make
build dependencies like cmake on HPC platforms. It turns out that
while this works in some cases, it did not handle cases where a
package was a link dependency of the root and of a build dependency
(and could produce incorrect concretizations which would not build).
2016-12-20 00:21:25 -08:00
Todd Gamblin
c89de04f40 Support Yorick versions (#2640) 2016-12-19 14:55:23 -08:00
Todd Gamblin
4b7b6768e6 Modify github regex to support NCO. (#2633) 2016-12-19 14:17:18 -08:00
Todd Gamblin
a2b4146e10 Fixes for various hash issues (#2626)
* Better output for disambiguate_specs()

* Fix wrong exception name.

* Fix satsifies(): concrete specs require matching by hash.

- Fixes uninstall by hash and other places where we need to match a
  specific spec.

- Fix an error in provider_index (satisfies() call was backwards)

- Fix an error in satisfies_dependencies(): checks were too shallow.

* Fix default args in Spec.tree()

* Move installed_dependents() to DB to avoid unknown package error.

* Make `spack find` and `sapck.store.db.query()` faster for hashes.

* Add a test to ensure satisfies() respects concrete Specs' hashes.
2016-12-19 09:09:53 -08:00
Alfredo Adolfo Gimenez
68e9a2ed8d Added customization for make targets in 'build' and 'install' phases for AutotoolsPackage (#2464)
* Customization for make targets in build and test phases for AutotoolsPackage

* Updated Blitz++ to use customized make build and test targets

* Removed flake8 error

* Removed make test customization, added make install customization, need to figure out issues with multiple make targets

* Changed build_targets and install_targets to normal attributes
2016-12-18 16:27:47 -08:00
Massimiliano Culpo
87c9b01033 openblas: derives from MakefilePackage (#2488)
* MakefilePackage: changed build_args and install_args for consistency with #2464
openblas: derives from MakefilePackage

* MakefilePackage: changed default edit behavior
2016-12-15 10:15:52 -08:00
Mario Melara
32cecf55ee Fix target for compiler on CNL operating systems (#2580)
Changes any to a string to avoid <built-in function any> being
incorrectly added to target in compilers.yaml.
2016-12-14 14:37:14 -08:00
Adam J. Stewart
1ac4ae0b41 Make unzip quiet (#2593) 2016-12-14 14:24:02 -08:00
Mario Melara
01230a62d4 Change attribute to match ArchSpec refactor (#2579)
Fixes issue #2578
2016-12-13 13:58:03 -08:00
Ben Morgan
fb809189d3 Better cxx11/14/17 flags for GNU/Clang/Intel (#2539)
* Better cxx11/14 flags for GNU/Clang/Intel

- GCC 4.8 only supports -std=c++1y for C++14
- Use CMake's rules for AppleClang to set cxx11 and cxx14 flags based on
  Apple Xcode/LLVM version
- Use CMake's rules for Intel to add support for cxx14 flags based on
  Intel version.

* Add cxx17_flag property

Implement property in compiler for c++17 as per those for c++11/14.
Add concrete support for GNU/Clang:

- Return -std=c++1z for GCC 5 and above per GCC documentation
- Return -std=c++1z for Clang 3.5 and above per Clang documentation
- Return -std=c++1z for Apple LLVM 6.1 and above per CMake's rules
2016-12-13 13:15:10 -08:00
Elizabeth Fischer
4f9e548bf6 Update spack setup and spack graph to be consistent with recent refactoring (#2317)
* Update `spack setup` and `spack graph` to be consistent with c557e765 and 9347f869.  Fixes #2316.

* Added another "fix" necessary to make `spack setup` work.

* Added another "fix" necessary to make `spack setup` work. (reverted from commit 7f0d3ecb38c97ec00491d7cd66b4266e3018b1ca)
2016-12-13 10:48:40 -08:00
Todd Gamblin
8496d8ff77 Add a test to ensure package names have the right case. (#2562) 2016-12-13 01:23:40 -08:00
Adam J. Stewart
9f71382912 Add missing bzip2 dependency to freetype (#2536)
* Add freetype dependency on bzip2

* Switch package with an install method for the docs
2016-12-12 14:20:36 -08:00
Sergey Kosukhin
8f28c9009e Set parent process's input stream inside the build process. (#2535) 2016-12-12 11:55:13 -08:00
Todd Gamblin
d3d87ea190 Add documentation for repositories and namespaces. (#2474)
* Add documentation for repositories and namespaces.

* Update and extend repository documentation per review.

- Also add `-N` argument for `spack spec`
2016-12-12 00:54:20 -08:00
Stephen McDowell
7e96b09e63 [docs] missin .. code-block:: console in Spack 101 (#2530)
The advanced [Uninstalling Packages](spack.readthedocs.io/en/latest/tutorial_sc16_spack_basics.html#uninstalling-packages) via hash has a couple missing `.. code-block:: console` directives ;)

I have no idea what branch to direct this to though...
2016-12-11 11:40:36 -08:00