Commit graph

2288 commits

Author SHA1 Message Date
Mario Melara
0405505e21 Quick fix for #3153 (#3822) 2017-04-14 07:56:51 -07:00
Adam J. Stewart
e12e9996e4 Document commands that no longer work (#3781) 2017-04-12 10:46:00 -05:00
Denis Davydov
c9da92dc97 doc: start with known issues section (#3722) 2017-04-10 08:20:57 -05:00
Adam J. Stewart
8e4ada5e99 Make it more clear that version ranges are inclusive (#3759) 2017-04-10 07:25:48 -05:00
Massimiliano Culpo
10c395b2f5 Abinit: a few modernizations for the package and its dependencies (#3699)
* libxc: added libs interface

* hdf5: added libs interface, added conflicts

* abinit: modernized package to use build interface

* netcdf-fortran: added libs interface

* abinit: added version 8.2.2
2017-04-07 11:18:34 +02:00
George Hartzell
84208523f9 set_executable can set S_IX{GRP,OTH} (#3742)
`set_executable` now checks if a user/group.other had read permission
on a file and if it does then it sets the corresponding executable
bit.

See #1483.
2017-04-06 16:25:13 -05:00
Max Katz
a526bcaf11 Correct uninstall -d to uninstall -R in tutorial (#3740)
-d was changed to -R as of PR #1917
2017-04-06 15:44:43 -05:00
Todd Gamblin
715ac8b7e6 Update copyright on the docs. (#3718) 2017-04-05 18:34:17 -07:00
Adam J. Stewart
df150b3e92 spack create MakefilePackage (#3710)
* spack create MakefilePackage

* Change default Perl template to match other build systems
2017-04-05 18:00:42 -05:00
scheibelp
9e50d16f13 Merge pull request #2789 from paulhopkins/features/allow_package_installation_directory_to_be_configured
Allow installation directory layout to be configured
2017-04-05 10:45:11 -07:00
Paul Hopkins
95c3cff6be Allow installation directory layout to be configured using either hash
length parameter or spec formatting.
2017-04-05 12:08:56 +01:00
scheibelp
0038ea84a1 Avoid null reference in arch concretization (#2596)
Fixes #2587

The concretizer falls back on using the root architecture (followed
by the default system architecture) to fill in unspecified arch
properties for a spec. It failed to check cases where the root could
be None.
2017-04-04 15:03:52 -07:00
Massimiliano Culpo
20f388cebd Fixes #3675: Abinit: invalid spec on concretization #(3686) 2017-04-04 07:03:11 -07:00
Justin Cook
508c96d046 Update getting_started.rst (#3685)
Fixed typo referring to .bashrc file
2017-04-04 06:50:44 -05:00
Adam J. Stewart
50df071ad9 Overhaul Spack's URL parsing (#2972)
* Remove fake URLs from Spack

* Ignore long lines for URLs that start with ftp:

* Preliminary changes to version regexes

* New redesign of version regexes

* Allow letters in version-only

* Fix detection of versions that end in Final

* Rearrange a few regexes and add examples

* Add tests for common download repositories

* Add test cases for common tarball naming schemes

* Finalize version regexes

* spack url test -> spack url summary

* Clean up comments

* Rearrange suffix checks

* Use query strings for name detection

* Remove no longer necessary url_for_version functions

* Strip off extraneous information after package name

* Add one more test

* Dot in square brackets does not need to be escaped

* Move renaming outside of parse_name_offset

* Fix versions for a couple more packages

* Fix flake8 and doc tests

* Correctly parse Python, Lua, and Bio++ package names

* Use effective URLs for mfem

* Add checksummed version to mitos

* Remove url_for_version from STAR-CCM+ package

* Revert changes to version numbers with underscores and dashes

* Fix name detection for tbb

* Correctly parse Ruby gems

* Reverted mfem back to shortened URLs.

* Updated instructions for better security

* Remove preferred=True from newest version

* Add tests for new `spack url list` flags

* Add tests for strip_name_suffixes

* Add unit tests for version separators

* Fix bugs related to parseable name but in parseable version

* Remove dead code, update docstring

* Ignore 'binary' at end of version string

* Remove platform from version

* Flip libedit version numbers

* Re-support weird NCO alpha/beta versions

* Rebase and remove one new fake URL

* Add / to beginning of regex to avoid picking up similarly named packages

* Ignore weird tar versions

* Fix bug in url parse --spider when no versions found

* Less strict version matching for spack versions

* Don't rename Python packages

* Be a little more selective, version must begin with a digit

* Re-add fake URLs

* Fix up several other packages

* Ignore more file endings

* Add parsing support for Miniconda

* Update tab completion

* XFAILS are now PASSES for 2 web tests
2017-04-03 15:34:16 -07:00
Massimiliano Culpo
ffef681377 new directive: conflicts() (#3125)
* Add conflicts(<spec>) directive
* openblas: added conflicts for intel@16 refs #3119
* added brief docs and unit tests
2017-04-02 11:40:09 -07:00
Todd Gamblin
0e785e0500 Add epigraph in docs. 2017-04-01 15:32:06 -07:00
Todd Gamblin
221f179716 Add better tests for web.py; fix some bugs found with spidering.
- _spider in web.py was actually failing to spider deeper than a certain
  point.
- Fixed multiprocessing pools to not use daemons and to allow recursive
  spawning.
- Added detailed tests for spidering and for finding archive versions.
- left some xfail URL finding exercises for the reader.
- Fix noqa annotations for some @when decorators
2017-04-01 15:10:45 -07:00
Adam J. Stewart
28d6d375b4 Fix Python 3 support in spack versions
- Add missing import, fixes spack versions in Python 2
- Fix spack versions in Python 3
2017-04-01 15:10:45 -07:00
Kenneth Hoste
fb98f9ee19 fix annoying 'fatal: Not a git repository' error message (#3657)
fix annoying 'fatal: Not a git repository' error message produced by 'spack list' when Spack is not run from a Git repository (#3657)
2017-04-01 03:29:06 -07:00
Gregory Lee
12c1216c34 fix flake8 ignore syntax (use E instead of ignore) (#3651) 2017-03-31 17:51:09 -07:00
Massimiliano Culpo
0c44dd28bb test/package_sanity.py: ported to pytest (#3474) 2017-03-31 13:42:04 -07:00
Massimiliano Culpo
ee7753a597 test/versions.py: ported to pytest (#3635) 2017-03-31 13:41:48 -07:00
Todd Gamblin
a8bcc23fe7 Spack works with Python 3
- Update version guard in spack script to allow python 3
- Update min required version in the docs
2017-03-31 13:40:41 -07:00
Todd Gamblin
3f21f2b088 Clean up tests and add Python3 to Travis.
- Clean up spec_syntax tests: don't dependend on DB order.
- spec_syntax hash parsing tests were strongly dependent on the order the
  DB was traversed.
- Tests now specifically grab the specs they want from the mock DB.
- Tests are more readable as a result.

- Add Python3 versions to Travis tests.
2017-03-31 13:40:41 -07:00
Todd Gamblin
7f3f493024 Fix concretization bugs with virtuals and deptypes.
1. Fix #2807: Can't depend on virtual and non-virtual package

- This is tested by test_my_dep_depends_on_provider_of_my_virtual_dep in
  the concretize.py test.

- This was actually working in the test suite, but it depended on the
  order the dependencies were resolved in. Resolving non-virtual then
  virtual worked, but virtual, then non-virtual did not.

- Problem was that an unnecessary copy was made of a spec that already
  had some dependencies set up, and the copy lost half of some of the
  dependency relationships.  This caused the "can'd depend on X twice
  error".

- Fix by eliminating unnecessary copy and ensuring that dep parameter of
  _merge_dependency is always safe to own -- i.e. it's a defensive copy
  from somewhere else.

2. Fix bug and simplify concretization of deptypes.

- deptypes weren't being accumulated; they were being set on each
  DependencySpec. This could cause concretization to get into an infinite
  loop.

- Fixed by accumulating deptypes in DependencySpec.update_deptypes()

- Also simplified deptype normalization logic: deptypes are now merged in
  constrain() like everything else -- there is no need to merge them
  specially or to look at dpeendents in _merge_dependency().

- Add some docstrings to deptype tests.
2017-03-31 13:40:41 -07:00
Todd Gamblin
fe6f39b662 Use key sorting instead of cmp()
- Get rid of pkgsort() usage for preferred variants.
- Concretization is now entirely based on key-based sorting.
- Remove PreferredPackages class and various spec cmp() methods.
- Replace with PackagePrefs class that implements a key function for
  sorting according to packages.yaml.
- Clear package pref caches on config test.
- Explicit compare methods instead of total_ordering in Version.
- Our total_ordering backport wasn't making Python 3 happy for some
  reason.
- Python 3's functools.total_ordering and spelling the operators out
  fixes the problem.
- Fix unicode issues with spec hashes, json, & YAML
- Try to use str everywhere and avoid unicode objects in python 2.
2017-03-31 13:40:41 -07:00
Todd Gamblin
0cd6555388 Resolve Python2/Python3 unicode issues by using str()
- Remove ascii encoding assumption from spack_yaml
- proc.communicate() returns bytes; convert to str before adding.
- Fix various byte string/unicode issues for Python 2/3 support
- Need to decode subprocess output as utf-8 in from_sourcing_files.
- Fix comments in strify()
2017-03-31 13:40:41 -07:00
Todd Gamblin
1d1a14dbe9 Convert Python 2 idioms to Python 2/3-compatible ones.
- convert print, StringIO, except as, octals, izip
- convert print statement to print function
- convert StringIO to six.StringIO
  - remove usage of csv reader in Spec, in favor of simple regex
  - csv reader only does byte strings
- convert 0755 octal literals to 0o755
- convert `except Foo, e` to `except Foo as e`
- fix a few places `str` is used.
  - may need to switch everything to str later.
- convert iteritems usages to use six.iteritems
- fix urllib and HTMLParser
- port metaclasses to use six.with_metaclass
- More octal literal conversions for Python 2/3
- Fix a new octal literal.
- Convert `basestring` to `six.string_types`
- Convert xrange -> range
- Fix various issues with encoding, iteritems, and Python3 semantics.
- Convert contextlib.nested to explicitly nexted context managers.
- Convert use of filter() to list comprehensions.
- Replace reduce() with list comprehensions.
-  Clean up composite: replace inspect.ismethod() with callable()
- Python 3 doesn't have "method" objects; inspect.ismethod returns False.
- Need to use callable in Composite to make it work.
- Update colify to use future division.
- Fix zip() usages that need to be lists.
- Python3: Use line-buffered logging instead of unbuffered.
- Python3 raises an error with unbuffered I/O
  - See https://bugs.python.org/issue17404
2017-03-31 13:40:41 -07:00
Todd Gamblin
0331b08c64 Update externals to work with Python 3
- Update YAML version to support Python 3
- Python 3 support for ordereddict backport
- Exclude Python3 YAML from version tests.
- Vendor six into Spack.
- Make Python version-check tests work with Python 3
- Add ability to add version check exceptions with '# nopyqver' line
  comments.
2017-03-31 13:40:41 -07:00
Adam J. Stewart
bc404532ea PythonPackage: Let There Be Tests! (#2869)
* Run python setup.py test if --run-tests
* Attempt to import the Python module after installation
* Add testing support to numpy and scipy
* Remove duplicated comments
* Update to new run-tests callback methodology
* Remove unrelated changes for another PR
2017-03-31 13:39:07 -07:00
Michael Kuhn
0d421137f4 Fix mxml (#3639)
mxml is now hosted on GitHub.
2017-03-31 13:41:08 -05:00
Milton Woods
9e43ff821c Extendable Perl (#3614)
* perl: make extendable and add Module::Build package
* perl: allow 'spack create' to identify perl packages from their contents
* perl-module-build: fix indenting of package docstring
* perl: split install() method for extensions into phases
* perl: auto-detect build method (Makefile.PL vs Build.PL) and define a 'check' method
* PerlPackage: use import statements similar to those in AutotoolsPackage
* PerlModule: fix detection of Build.PL
* PerlPackageTemplate: remove extraneous lines to avoid flake8 warnings
* PerlPackageTemplate: split into separate templates for Makefile.PL and Build.PL
* PerlPackage: add cross-references to docstrings
* AutotoolsPackage: fix ambiguous cross-references to avoid errors in doc tests
* PerlbuildPackageTemplate: depend on perl-module-build if Build.PL exists
2017-03-30 16:38:58 -07:00
Todd Gamblin
782f29bc4b Fix bug in spack find for installations with unknown namespaces. (#3573)
- Spack find would fail with "unknown namespace" for some queries when a
  package from an unknown namespace was installed.

- Solve by being conservative: assume unknown packages are NOT providers
  of virtual dependencies.
2017-03-28 08:43:43 -07:00
Todd Gamblin
e549daa8ce Bugfix: allow deactivating installs that don't have packages anymore. (#3572)
- deactivate -a wouldn't work if the installation's package was no longer
  available.

- Fix installed_extensions_for so that it doesn't need to look at the
  package.py file.
2017-03-28 08:43:15 -07:00
Denis Davydov
61d01a71dd make git fetching quite (#3180)
also output repo and branch/commit/tag in one line
2017-03-24 17:37:47 -07:00
Adam J. Stewart
4ecfc39e1e Add Expect package (#3517)
* Add Expect package

* Ignore patches during flake8 tests for package.py files

* Remove controversial changes
2017-03-23 14:35:11 -05:00
George Hartzell
9b5f5fccf0 Fix for find --explicit #3374 (#3492)
This fixes the problem described in #3374, which describes `spack find` ignore explicit/implicit.

I believe that this was broken in #2626.

This restores the behavior of implicit/explicit for me.

I believe that it does not screw anything else up, but ....
2017-03-21 15:30:38 -07:00
scheibelp
19dca26f3a Improve output for compiler commands (#3480)
* Order listed compiler sections

"spack compiler list" output compiler sections in an arbitrary order.
With this commit compiler sections are ordered primarily by compiler
name and then by operating system and target.

* Compiler search lists config files with compilers

If a compiler entry is already defined in a configuration file that
the user does not know about, they may be confused when that compiler
is not added by "spack compiler find". This commit adds a message at
the end of "spack compiler find" to inform the user of the locations
of all config files where compilers are defined.
2017-03-20 12:36:44 -07:00
Matthew LeGendre
5ac6421f14 Fix issue with config.guess patching when the overwritten config.guess did not have write permissions. (#3494) 2017-03-20 12:35:38 -07:00
Todd Gamblin
1297e47463 Use byte-encoded UTF-8 for sourced environment in Python 2 (#3489)
- Fixes recurring errors on develop with unicode commit characters.
- still Python3-proof: python3 will use str instead of bytestrings.
2017-03-18 22:48:20 -07:00
Adam J. Stewart
10b767b93f Don't use @system in packages.yaml (#3472) 2017-03-17 07:55:55 -07:00
scheibelp
0b27a7e13d Detect when OS updates affect compiler selection (#3410)
Fixes #1476

Concretization uses compilers defined in config files and if those
are not available defaults to searching typical paths where the
detected operating system would have a compiler. If there is an OS
update, the detected OS can change; in this case all compilers
defined in the config files would no longer match (because they would
be associated with the previous OS version). The error message in
this case was too vague. This commit adds logic for detecting when it
is likely that the OS has been updated (in particular when that
affects compiler concretization) and improves the information provided
to the user in the error message.
2017-03-16 21:08:13 -07:00
scheibelp
5936ad2ca7 Dont propagate flags between different compilers (#3379)
* Dont propagate flags between different compilers

Fixes #2786

Previously when a spec had no parents with an equivalent compiler,
Spack would default to adding the compiler flags associated with the
root of the DAG. This eliminates that default.

* added test for compiler flag propagation

* simplify compiler flag propagation logic
2017-03-16 21:07:35 -07:00
scheibelp
9a27dec8e8 Dont auto-init compiler conf for 'compiler find' (#3439)
Fixes #3428

Users can run 'spack compiler find' to automatically initialize their
compilers.yaml configuration file. It also turns out that Spack will
implicitly initialize the compilers configuration file as part of
detecting compilers if none are found (so if a user were to attempt to
concretize a spec without running 'spack compiler find' it would not
fail). However, in this case Spack was overlooking its own implicit
initialization of the config files and would report that no new
compilers were found. This commit removes implicit initialization when
the user calls 'spack compiler find'.

This did not surface until #2999 because the 'spack compiler' command
defaulted to using a scope 'user/platform' that was not accounted for
in get_compiler_config (where the implicit initialization logic
predates the addition of this new scope); #2999 removed the scope
specification when checking through config files, leading to the
implicit initialization.
2017-03-16 11:24:34 -07:00
Adam J. Stewart
dca4d2b15e Consistent docs and usage of env mod methods (#3351) 2017-03-14 22:26:44 -07:00
Denis Davydov
560d28ac7f fix automatic mixing of clang with gfortran 6.3.0 on macOS (#3427)
* fix automatic mixing of clang with gfortran 6.3.0 on macOS
* automatically mix any gfortran with any clang on macOS
* adjust the unit test
2017-03-14 09:48:27 -07:00
Massimiliano Culpo
8c3edfd36f test/file_cache.py: ported to pytest (#3429) 2017-03-14 09:07:04 -07:00
Todd Gamblin
d11e262b36 Fix from_sourcing_files(): decode json input as utf-8 (#3433) 2017-03-13 15:52:36 -07:00
Erik Schnetter
63a8f79980 Correct inconsistency in comment (#3414) 2017-03-11 12:52:08 -06:00