Commit graph

5849 commits

Author SHA1 Message Date
Justin Cook
508c96d046 Update getting_started.rst (#3685)
Fixed typo referring to .bashrc file
2017-04-04 06:50:44 -05:00
Adam J. Stewart
d6a28a7401 Add IOzone package (#3679) 2017-04-03 18:05:45 -05:00
Adam J. Stewart
50df071ad9 Overhaul Spack's URL parsing (#2972)
* Remove fake URLs from Spack

* Ignore long lines for URLs that start with ftp:

* Preliminary changes to version regexes

* New redesign of version regexes

* Allow letters in version-only

* Fix detection of versions that end in Final

* Rearrange a few regexes and add examples

* Add tests for common download repositories

* Add test cases for common tarball naming schemes

* Finalize version regexes

* spack url test -> spack url summary

* Clean up comments

* Rearrange suffix checks

* Use query strings for name detection

* Remove no longer necessary url_for_version functions

* Strip off extraneous information after package name

* Add one more test

* Dot in square brackets does not need to be escaped

* Move renaming outside of parse_name_offset

* Fix versions for a couple more packages

* Fix flake8 and doc tests

* Correctly parse Python, Lua, and Bio++ package names

* Use effective URLs for mfem

* Add checksummed version to mitos

* Remove url_for_version from STAR-CCM+ package

* Revert changes to version numbers with underscores and dashes

* Fix name detection for tbb

* Correctly parse Ruby gems

* Reverted mfem back to shortened URLs.

* Updated instructions for better security

* Remove preferred=True from newest version

* Add tests for new `spack url list` flags

* Add tests for strip_name_suffixes

* Add unit tests for version separators

* Fix bugs related to parseable name but in parseable version

* Remove dead code, update docstring

* Ignore 'binary' at end of version string

* Remove platform from version

* Flip libedit version numbers

* Re-support weird NCO alpha/beta versions

* Rebase and remove one new fake URL

* Add / to beginning of regex to avoid picking up similarly named packages

* Ignore weird tar versions

* Fix bug in url parse --spider when no versions found

* Less strict version matching for spack versions

* Don't rename Python packages

* Be a little more selective, version must begin with a digit

* Re-add fake URLs

* Fix up several other packages

* Ignore more file endings

* Add parsing support for Miniconda

* Update tab completion

* XFAILS are now PASSES for 2 web tests
2017-04-03 15:34:16 -07:00
Adam J. Stewart
7e9777f294 Add latest version of pkg-config (#3672) 2017-04-03 14:09:44 -05:00
健美猫
734f90d11f openfst conflicts with intel@16: and gcc@6: . (#3671) 2017-04-03 13:35:04 -05:00
健美猫
74bd894427 Fix conflict directives for libpfm4. (#3673) 2017-04-03 13:05:03 -05:00
Adam J. Stewart
6cf87e3637 Add latest alpha release of MVAPICH2 (#3674) 2017-04-03 13:04:42 -05:00
George Hartzell
11fe0b9b26 Add depends_on() new-ish bison (#3661)
* Add depends_on() new-ish bison

* Refine dependency, only needed to build.
2017-04-03 10:59:44 -05:00
健美猫
57b3961778 Add libpfm4 package. (#3667) 2017-04-03 10:59:24 -05:00
Jeffrey Salmond
9b19968bac add CUDA_HOME arg to nccl makefile (#3666) 2017-04-03 08:08:28 -05:00
Jeffrey Salmond
b7eee1bd7b add gflags as dependency of glog (#3665) 2017-04-03 07:56:58 -05:00
George Hartzell
cddb7524b7 Hey, I was using that! :) (#3660)
Turns out that we really want 5.22.2 for backward compatibility.
2017-04-02 20:32:20 -05:00
Adam J. Stewart
c0cfaacbc8 Set default providers for all virtual dependencies (#3634)
* Set default providers for everything

* Add default OpenFOAM provider
2017-04-02 14:48:27 -05:00
Massimiliano Culpo
ffef681377 new directive: conflicts() (#3125)
* Add conflicts(<spec>) directive
* openblas: added conflicts for intel@16 refs #3119
* added brief docs and unit tests
2017-04-02 11:40:09 -07:00
Todd Gamblin
0e785e0500 Add epigraph in docs. 2017-04-01 15:32:06 -07:00
Todd Gamblin
221f179716 Add better tests for web.py; fix some bugs found with spidering.
- _spider in web.py was actually failing to spider deeper than a certain
  point.
- Fixed multiprocessing pools to not use daemons and to allow recursive
  spawning.
- Added detailed tests for spidering and for finding archive versions.
- left some xfail URL finding exercises for the reader.
- Fix noqa annotations for some @when decorators
2017-04-01 15:10:45 -07:00
Adam J. Stewart
28d6d375b4 Fix Python 3 support in spack versions
- Add missing import, fixes spack versions in Python 2
- Fix spack versions in Python 3
2017-04-01 15:10:45 -07:00
Kenneth Hoste
b67e2db159 fix version specs for EasyBuild packages (#3659)
* fix Python version requirement for easybuild-framework

* make 'when' for depends_on('py-easybuild-framework') open-ended
2017-04-01 13:14:15 -05:00
Kenneth Hoste
90306cf415 EasyBuild and dependencies (#3658)
* add packages for EasyBuild and dependencies

* fix remarks w.r.t. docstrings & download urls, make dependency specs stricter w.r.t. required versions

* fix easybuild-* dep specs, should be open-ended

* correct dep specs for easybuild and easybuild-easyconfigs using @3.1.2[:]
2017-04-01 10:27:58 -07:00
Kenneth Hoste
fb98f9ee19 fix annoying 'fatal: Not a git repository' error message (#3657)
fix annoying 'fatal: Not a git repository' error message produced by 'spack list' when Spack is not run from a Git repository (#3657)
2017-04-01 03:29:06 -07:00
健美猫
f982cd78ae Add a pcakge: libzip. (#3656) 2017-04-01 00:09:09 -05:00
Gregory Lee
12c1216c34 fix flake8 ignore syntax (use E instead of ignore) (#3651) 2017-03-31 17:51:09 -07:00
Massimiliano Culpo
0c44dd28bb test/package_sanity.py: ported to pytest (#3474) 2017-03-31 13:42:04 -07:00
Massimiliano Culpo
ee7753a597 test/versions.py: ported to pytest (#3635) 2017-03-31 13:41:48 -07:00
Todd Gamblin
a8bcc23fe7 Spack works with Python 3
- Update version guard in spack script to allow python 3
- Update min required version in the docs
2017-03-31 13:40:41 -07:00
Todd Gamblin
3f21f2b088 Clean up tests and add Python3 to Travis.
- Clean up spec_syntax tests: don't dependend on DB order.
- spec_syntax hash parsing tests were strongly dependent on the order the
  DB was traversed.
- Tests now specifically grab the specs they want from the mock DB.
- Tests are more readable as a result.

- Add Python3 versions to Travis tests.
2017-03-31 13:40:41 -07:00
Todd Gamblin
b9ee86cac9 Make packages Python3 compatible. 2017-03-31 13:40:41 -07:00
Todd Gamblin
7f3f493024 Fix concretization bugs with virtuals and deptypes.
1. Fix #2807: Can't depend on virtual and non-virtual package

- This is tested by test_my_dep_depends_on_provider_of_my_virtual_dep in
  the concretize.py test.

- This was actually working in the test suite, but it depended on the
  order the dependencies were resolved in. Resolving non-virtual then
  virtual worked, but virtual, then non-virtual did not.

- Problem was that an unnecessary copy was made of a spec that already
  had some dependencies set up, and the copy lost half of some of the
  dependency relationships.  This caused the "can'd depend on X twice
  error".

- Fix by eliminating unnecessary copy and ensuring that dep parameter of
  _merge_dependency is always safe to own -- i.e. it's a defensive copy
  from somewhere else.

2. Fix bug and simplify concretization of deptypes.

- deptypes weren't being accumulated; they were being set on each
  DependencySpec. This could cause concretization to get into an infinite
  loop.

- Fixed by accumulating deptypes in DependencySpec.update_deptypes()

- Also simplified deptype normalization logic: deptypes are now merged in
  constrain() like everything else -- there is no need to merge them
  specially or to look at dpeendents in _merge_dependency().

- Add some docstrings to deptype tests.
2017-03-31 13:40:41 -07:00
Todd Gamblin
fe6f39b662 Use key sorting instead of cmp()
- Get rid of pkgsort() usage for preferred variants.
- Concretization is now entirely based on key-based sorting.
- Remove PreferredPackages class and various spec cmp() methods.
- Replace with PackagePrefs class that implements a key function for
  sorting according to packages.yaml.
- Clear package pref caches on config test.
- Explicit compare methods instead of total_ordering in Version.
- Our total_ordering backport wasn't making Python 3 happy for some
  reason.
- Python 3's functools.total_ordering and spelling the operators out
  fixes the problem.
- Fix unicode issues with spec hashes, json, & YAML
- Try to use str everywhere and avoid unicode objects in python 2.
2017-03-31 13:40:41 -07:00
Todd Gamblin
0cd6555388 Resolve Python2/Python3 unicode issues by using str()
- Remove ascii encoding assumption from spack_yaml
- proc.communicate() returns bytes; convert to str before adding.
- Fix various byte string/unicode issues for Python 2/3 support
- Need to decode subprocess output as utf-8 in from_sourcing_files.
- Fix comments in strify()
2017-03-31 13:40:41 -07:00
Todd Gamblin
1d1a14dbe9 Convert Python 2 idioms to Python 2/3-compatible ones.
- convert print, StringIO, except as, octals, izip
- convert print statement to print function
- convert StringIO to six.StringIO
  - remove usage of csv reader in Spec, in favor of simple regex
  - csv reader only does byte strings
- convert 0755 octal literals to 0o755
- convert `except Foo, e` to `except Foo as e`
- fix a few places `str` is used.
  - may need to switch everything to str later.
- convert iteritems usages to use six.iteritems
- fix urllib and HTMLParser
- port metaclasses to use six.with_metaclass
- More octal literal conversions for Python 2/3
- Fix a new octal literal.
- Convert `basestring` to `six.string_types`
- Convert xrange -> range
- Fix various issues with encoding, iteritems, and Python3 semantics.
- Convert contextlib.nested to explicitly nexted context managers.
- Convert use of filter() to list comprehensions.
- Replace reduce() with list comprehensions.
-  Clean up composite: replace inspect.ismethod() with callable()
- Python 3 doesn't have "method" objects; inspect.ismethod returns False.
- Need to use callable in Composite to make it work.
- Update colify to use future division.
- Fix zip() usages that need to be lists.
- Python3: Use line-buffered logging instead of unbuffered.
- Python3 raises an error with unbuffered I/O
  - See https://bugs.python.org/issue17404
2017-03-31 13:40:41 -07:00
Todd Gamblin
0331b08c64 Update externals to work with Python 3
- Update YAML version to support Python 3
- Python 3 support for ordereddict backport
- Exclude Python3 YAML from version tests.
- Vendor six into Spack.
- Make Python version-check tests work with Python 3
- Add ability to add version check exceptions with '# nopyqver' line
  comments.
2017-03-31 13:40:41 -07:00
Adam J. Stewart
bc404532ea PythonPackage: Let There Be Tests! (#2869)
* Run python setup.py test if --run-tests
* Attempt to import the Python module after installation
* Add testing support to numpy and scipy
* Remove duplicated comments
* Update to new run-tests callback methodology
* Remove unrelated changes for another PR
2017-03-31 13:39:07 -07:00
Gregory Lee
3ade829566 new and updated PRUNERS tools (#3648) 2017-03-31 13:42:03 -05:00
Michael Kuhn
0d421137f4 Fix mxml (#3639)
mxml is now hosted on GitHub.
2017-03-31 13:41:08 -05:00
Adam J. Stewart
343f3b2fde Add QBank package (#3641)
* Add QBank package

* Install the documentation and add it to the MANPATH

* Add Perl DBI package

* Make sure setup method variables are consistent
2017-03-31 11:51:13 -05:00
Adam J. Stewart
d13854b5d4 Perl: add more versions, add phases (#3640) 2017-03-31 09:18:28 -05:00
Mark Olesen
832f932310 CONFIG: add older bison version 2.7 (#3645)
- for programs that don't yet work with bison 3.x
2017-03-31 07:54:04 -05:00
Hans Pabst
d59ceba9c5 Included LIBXSMM 1.8 into the list of available versions. (#3646) 2017-03-31 07:53:37 -05:00
Milton Woods
890213d5c2 Subversion with perl bindings (#3644)
* perl-term-readkey: add perl Term::ReadKey package
* subversion: add variant with perl bindings
* subversion: use `which('perl')` to find perl executable in package.py
2017-03-31 01:48:18 -07:00
Milton Woods
a81b2b4dc6 perl-xml-parser: add perl XML::Parser package (#3643) 2017-03-30 21:54:56 -07:00
Milton Woods
9e43ff821c Extendable Perl (#3614)
* perl: make extendable and add Module::Build package
* perl: allow 'spack create' to identify perl packages from their contents
* perl-module-build: fix indenting of package docstring
* perl: split install() method for extensions into phases
* perl: auto-detect build method (Makefile.PL vs Build.PL) and define a 'check' method
* PerlPackage: use import statements similar to those in AutotoolsPackage
* PerlModule: fix detection of Build.PL
* PerlPackageTemplate: remove extraneous lines to avoid flake8 warnings
* PerlPackageTemplate: split into separate templates for Makefile.PL and Build.PL
* PerlPackage: add cross-references to docstrings
* AutotoolsPackage: fix ambiguous cross-references to avoid errors in doc tests
* PerlbuildPackageTemplate: depend on perl-module-build if Build.PL exists
2017-03-30 16:38:58 -07:00
Mark Olesen
9e1abb13dc support OpenFOAM package(s) (#3528)
* ENH: add package for building OpenFOAM (1612) from www.openfoam.com
- provide 'openfoam' as virtual package.
- package as openfoam-com to reflect the distribution point.

This initial spack packaging for OpenFOAM supports a number of possible
variants and should handle 64-bit labels properly now that the scotch
package has been updated accordingly.

* ENH: update package for foam-extend (extend-project.de)

- provide 'openfoam' as virtual package.

- much of the build is now aligned with how the openfoam-com package
  looks, with the aim of future refactoring.

- avoid installing intermediate targets.

- contains its own environment sourcing script for the build, for more
  flexibility and robustness (doesn't touch the python build environ)

* ENH: added package for building from openfoam.org

- provide 'openfoam' as a virtual package.

- this is largely a direct copy of the openfoam-com package.
  It has been supplied as a courtesy for users and to ensure maximum
  consistency in quality and naming between the foam-extend,
  openfoam-com and openfoam-org packages.

* CONFIG: add openfoam into bash completion providers list

* ENH: have openfoam-com use spack as USERMPI

- also simplify the generation of mplib/compiler rules

* ENH: have openfoam-org use spack as SYSTEMMPI

- this setup requires more environment settings than USERMPI
  (openfoam-com), but is currently the only means of integration
  for openfoam-org

- simplify generation of mplib/compiler rules

* ENH: simplify generation of mplib/compiler rules (foam-extend)

- rename mpi rules from SPACK,SPACKMPI to USER,USERMPI for consistency
  with openfoam-com and to generalize for any build system.

* STYLE: record spack tree as a log file (openfoam)

- can be useful for future diagnostics and general record keeping
2017-03-30 16:35:57 -07:00
healther
2bd216a6a1 add package py-spykeutils (#3633)
* add package py-spykeutils

* fixed dependency version restriction
2017-03-30 10:33:09 -05:00
Massimiliano Culpo
f9c0348891 cp2k: update to 4.1 + update of dependencies (#3561)
* libint: updated version, ported to AutotoolsPackage

* elpa: updated version, ported to AutotoolsPackage

* wannier90: updated version, ported to MakefilePackage

* pexsi: updated version, ported to MakefilePackage

* cp2k: updated version, removed wannier 90 from dependencies if cp2k@4.1

* plumed: updated version

Modifications:

* modules has been turned into a single variant, as different versions
  have different modules available
* added dictionaries for patches

* netlib-scalapack: ported to CMakePackage
2017-03-30 09:46:05 -05:00
Adam J. Stewart
73f78cb8c4 Move py-dev to the proper repo directory (#3630) 2017-03-30 09:18:40 +02:00
Adam J. Stewart
da6faa086a Fix url fetching for GnuTLS (#3631) 2017-03-30 09:16:56 +02:00
George Hartzell
68415f0fe5 Concretization problem breaks httpie dependency (#3629)
* Concretization problem breaks httpie dependency

Concretization problem breaks this.  Unconditional for now...
https://github.com/LLNL/spack/issues/3628

* Flake8 cleanups
2017-03-29 18:59:54 -05:00
George Hartzell
4cd4a2da46 Add package for snakemake (#3605)
* Add package for snakemake

* Flake8 cleanups

* Add package for py-wrapt

* Address PR feedback

Switch to pypi URL.

Tighten Python3 dependency spec.

Add missing dependencies (wrapt, requests).
    - add package for wrapt.
2017-03-29 14:25:09 -05:00
George Hartzell
6138350c9e Add package for httpie (#3609)
* Add package for httpie

* Remove wayward FIXME and flake8 cleanup

* Update dependencies, address PR feedback

Fine tune dependencies (version constraints).
Add missing dependency.
Make SOCKS support a variant.

* Flake8 cleanup
2017-03-29 14:24:52 -05:00