Commit graph

5973 commits

Author SHA1 Message Date
Todd Gamblin
b9ee86cac9 Make packages Python3 compatible. 2017-03-31 13:40:41 -07:00
Todd Gamblin
7f3f493024 Fix concretization bugs with virtuals and deptypes.
1. Fix #2807: Can't depend on virtual and non-virtual package

- This is tested by test_my_dep_depends_on_provider_of_my_virtual_dep in
  the concretize.py test.

- This was actually working in the test suite, but it depended on the
  order the dependencies were resolved in. Resolving non-virtual then
  virtual worked, but virtual, then non-virtual did not.

- Problem was that an unnecessary copy was made of a spec that already
  had some dependencies set up, and the copy lost half of some of the
  dependency relationships.  This caused the "can'd depend on X twice
  error".

- Fix by eliminating unnecessary copy and ensuring that dep parameter of
  _merge_dependency is always safe to own -- i.e. it's a defensive copy
  from somewhere else.

2. Fix bug and simplify concretization of deptypes.

- deptypes weren't being accumulated; they were being set on each
  DependencySpec. This could cause concretization to get into an infinite
  loop.

- Fixed by accumulating deptypes in DependencySpec.update_deptypes()

- Also simplified deptype normalization logic: deptypes are now merged in
  constrain() like everything else -- there is no need to merge them
  specially or to look at dpeendents in _merge_dependency().

- Add some docstrings to deptype tests.
2017-03-31 13:40:41 -07:00
Todd Gamblin
fe6f39b662 Use key sorting instead of cmp()
- Get rid of pkgsort() usage for preferred variants.
- Concretization is now entirely based on key-based sorting.
- Remove PreferredPackages class and various spec cmp() methods.
- Replace with PackagePrefs class that implements a key function for
  sorting according to packages.yaml.
- Clear package pref caches on config test.
- Explicit compare methods instead of total_ordering in Version.
- Our total_ordering backport wasn't making Python 3 happy for some
  reason.
- Python 3's functools.total_ordering and spelling the operators out
  fixes the problem.
- Fix unicode issues with spec hashes, json, & YAML
- Try to use str everywhere and avoid unicode objects in python 2.
2017-03-31 13:40:41 -07:00
Todd Gamblin
0cd6555388 Resolve Python2/Python3 unicode issues by using str()
- Remove ascii encoding assumption from spack_yaml
- proc.communicate() returns bytes; convert to str before adding.
- Fix various byte string/unicode issues for Python 2/3 support
- Need to decode subprocess output as utf-8 in from_sourcing_files.
- Fix comments in strify()
2017-03-31 13:40:41 -07:00
Todd Gamblin
1d1a14dbe9 Convert Python 2 idioms to Python 2/3-compatible ones.
- convert print, StringIO, except as, octals, izip
- convert print statement to print function
- convert StringIO to six.StringIO
  - remove usage of csv reader in Spec, in favor of simple regex
  - csv reader only does byte strings
- convert 0755 octal literals to 0o755
- convert `except Foo, e` to `except Foo as e`
- fix a few places `str` is used.
  - may need to switch everything to str later.
- convert iteritems usages to use six.iteritems
- fix urllib and HTMLParser
- port metaclasses to use six.with_metaclass
- More octal literal conversions for Python 2/3
- Fix a new octal literal.
- Convert `basestring` to `six.string_types`
- Convert xrange -> range
- Fix various issues with encoding, iteritems, and Python3 semantics.
- Convert contextlib.nested to explicitly nexted context managers.
- Convert use of filter() to list comprehensions.
- Replace reduce() with list comprehensions.
-  Clean up composite: replace inspect.ismethod() with callable()
- Python 3 doesn't have "method" objects; inspect.ismethod returns False.
- Need to use callable in Composite to make it work.
- Update colify to use future division.
- Fix zip() usages that need to be lists.
- Python3: Use line-buffered logging instead of unbuffered.
- Python3 raises an error with unbuffered I/O
  - See https://bugs.python.org/issue17404
2017-03-31 13:40:41 -07:00
Todd Gamblin
0331b08c64 Update externals to work with Python 3
- Update YAML version to support Python 3
- Python 3 support for ordereddict backport
- Exclude Python3 YAML from version tests.
- Vendor six into Spack.
- Make Python version-check tests work with Python 3
- Add ability to add version check exceptions with '# nopyqver' line
  comments.
2017-03-31 13:40:41 -07:00
Adam J. Stewart
bc404532ea PythonPackage: Let There Be Tests! (#2869)
* Run python setup.py test if --run-tests
* Attempt to import the Python module after installation
* Add testing support to numpy and scipy
* Remove duplicated comments
* Update to new run-tests callback methodology
* Remove unrelated changes for another PR
2017-03-31 13:39:07 -07:00
Gregory Lee
3ade829566 new and updated PRUNERS tools (#3648) 2017-03-31 13:42:03 -05:00
Michael Kuhn
0d421137f4 Fix mxml (#3639)
mxml is now hosted on GitHub.
2017-03-31 13:41:08 -05:00
Adam J. Stewart
343f3b2fde Add QBank package (#3641)
* Add QBank package

* Install the documentation and add it to the MANPATH

* Add Perl DBI package

* Make sure setup method variables are consistent
2017-03-31 11:51:13 -05:00
Adam J. Stewart
d13854b5d4 Perl: add more versions, add phases (#3640) 2017-03-31 09:18:28 -05:00
Mark Olesen
832f932310 CONFIG: add older bison version 2.7 (#3645)
- for programs that don't yet work with bison 3.x
2017-03-31 07:54:04 -05:00
Hans Pabst
d59ceba9c5 Included LIBXSMM 1.8 into the list of available versions. (#3646) 2017-03-31 07:53:37 -05:00
Milton Woods
890213d5c2 Subversion with perl bindings (#3644)
* perl-term-readkey: add perl Term::ReadKey package
* subversion: add variant with perl bindings
* subversion: use `which('perl')` to find perl executable in package.py
2017-03-31 01:48:18 -07:00
Milton Woods
a81b2b4dc6 perl-xml-parser: add perl XML::Parser package (#3643) 2017-03-30 21:54:56 -07:00
Milton Woods
9e43ff821c Extendable Perl (#3614)
* perl: make extendable and add Module::Build package
* perl: allow 'spack create' to identify perl packages from their contents
* perl-module-build: fix indenting of package docstring
* perl: split install() method for extensions into phases
* perl: auto-detect build method (Makefile.PL vs Build.PL) and define a 'check' method
* PerlPackage: use import statements similar to those in AutotoolsPackage
* PerlModule: fix detection of Build.PL
* PerlPackageTemplate: remove extraneous lines to avoid flake8 warnings
* PerlPackageTemplate: split into separate templates for Makefile.PL and Build.PL
* PerlPackage: add cross-references to docstrings
* AutotoolsPackage: fix ambiguous cross-references to avoid errors in doc tests
* PerlbuildPackageTemplate: depend on perl-module-build if Build.PL exists
2017-03-30 16:38:58 -07:00
Mark Olesen
9e1abb13dc support OpenFOAM package(s) (#3528)
* ENH: add package for building OpenFOAM (1612) from www.openfoam.com
- provide 'openfoam' as virtual package.
- package as openfoam-com to reflect the distribution point.

This initial spack packaging for OpenFOAM supports a number of possible
variants and should handle 64-bit labels properly now that the scotch
package has been updated accordingly.

* ENH: update package for foam-extend (extend-project.de)

- provide 'openfoam' as virtual package.

- much of the build is now aligned with how the openfoam-com package
  looks, with the aim of future refactoring.

- avoid installing intermediate targets.

- contains its own environment sourcing script for the build, for more
  flexibility and robustness (doesn't touch the python build environ)

* ENH: added package for building from openfoam.org

- provide 'openfoam' as a virtual package.

- this is largely a direct copy of the openfoam-com package.
  It has been supplied as a courtesy for users and to ensure maximum
  consistency in quality and naming between the foam-extend,
  openfoam-com and openfoam-org packages.

* CONFIG: add openfoam into bash completion providers list

* ENH: have openfoam-com use spack as USERMPI

- also simplify the generation of mplib/compiler rules

* ENH: have openfoam-org use spack as SYSTEMMPI

- this setup requires more environment settings than USERMPI
  (openfoam-com), but is currently the only means of integration
  for openfoam-org

- simplify generation of mplib/compiler rules

* ENH: simplify generation of mplib/compiler rules (foam-extend)

- rename mpi rules from SPACK,SPACKMPI to USER,USERMPI for consistency
  with openfoam-com and to generalize for any build system.

* STYLE: record spack tree as a log file (openfoam)

- can be useful for future diagnostics and general record keeping
2017-03-30 16:35:57 -07:00
healther
2bd216a6a1 add package py-spykeutils (#3633)
* add package py-spykeutils

* fixed dependency version restriction
2017-03-30 10:33:09 -05:00
Massimiliano Culpo
f9c0348891 cp2k: update to 4.1 + update of dependencies (#3561)
* libint: updated version, ported to AutotoolsPackage

* elpa: updated version, ported to AutotoolsPackage

* wannier90: updated version, ported to MakefilePackage

* pexsi: updated version, ported to MakefilePackage

* cp2k: updated version, removed wannier 90 from dependencies if cp2k@4.1

* plumed: updated version

Modifications:

* modules has been turned into a single variant, as different versions
  have different modules available
* added dictionaries for patches

* netlib-scalapack: ported to CMakePackage
2017-03-30 09:46:05 -05:00
Adam J. Stewart
73f78cb8c4 Move py-dev to the proper repo directory (#3630) 2017-03-30 09:18:40 +02:00
Adam J. Stewart
da6faa086a Fix url fetching for GnuTLS (#3631) 2017-03-30 09:16:56 +02:00
George Hartzell
68415f0fe5 Concretization problem breaks httpie dependency (#3629)
* Concretization problem breaks httpie dependency

Concretization problem breaks this.  Unconditional for now...
https://github.com/LLNL/spack/issues/3628

* Flake8 cleanups
2017-03-29 18:59:54 -05:00
George Hartzell
4cd4a2da46 Add package for snakemake (#3605)
* Add package for snakemake

* Flake8 cleanups

* Add package for py-wrapt

* Address PR feedback

Switch to pypi URL.

Tighten Python3 dependency spec.

Add missing dependencies (wrapt, requests).
    - add package for wrapt.
2017-03-29 14:25:09 -05:00
George Hartzell
6138350c9e Add package for httpie (#3609)
* Add package for httpie

* Remove wayward FIXME and flake8 cleanup

* Update dependencies, address PR feedback

Fine tune dependencies (version constraints).
Add missing dependency.
Make SOCKS support a variant.

* Flake8 cleanup
2017-03-29 14:24:52 -05:00
Denis Davydov
782882c2a9 petsc: a workaround for macOS Clang 8.1.0 and XCode 8.3 (#3615) 2017-03-29 14:24:18 -05:00
healther
64a80ba822 add package py-bokeh (#3618)
* add package py-bokeh

* remove setuptools dependency

* add missing dependencies
2017-03-29 14:23:53 -05:00
healther
28db55b1a7 Allow py-py2cairo to be built with cairo~X (#3620) 2017-03-29 14:23:24 -05:00
healther
b73f1e557c add package py-pympler (#3621)
* add package py-pympler

* add python@2.5: dependency
2017-03-29 14:21:41 -05:00
healther
8e99c97d21 Add package py-pynn (#3622)
* add package py-pynn

* fix docstring

* fix dependency mess

* add python dependency
2017-03-29 14:21:13 -05:00
healther
8d87c01f62 Add py-pymongo dependencies (#3624)
* add package py-mongo

* add dependencies
2017-03-29 14:20:58 -05:00
George Hartzell
259cc02260 htslib depends_on version must match samtools (#3625) 2017-03-29 14:20:37 -05:00
Adam J. Stewart
46251eb559 Add abipy package and dependencies (#3352)
* Add abipy package and dependencies

* Add first official release of py-abipy

* Ignore files from dependencies during activation
2017-03-29 11:13:23 -05:00
healther
441170f316 add package py-mongo (#3619) 2017-03-29 02:54:09 -07:00
healther
5ba4fce428 add package py-brian2 (#3617) 2017-03-29 02:52:59 -07:00
healther
754be1b275 Add version 3.5.9 to package gnutls (#3616) 2017-03-29 01:55:02 -07:00
George Hartzell
f5bc42f229 Add Blast+ package. (#3562)
* Add Blast+ package.

Adds support for NCBI's blast+@2.6.0.  I'll be adding a few historical versions in the near future.

It's a fairly direct transliteration of the [Homebrew Science recipe][hbs].

I skipped the mysql support.

There is a problem with HDF5 support.  Blast's configure script test program defines a macro 'HOST' to a string value and one of the HDF5 bits has an variable named 'HOST'.  Fun things happen.   I've disabled support for now.

I've run very very basic tests.  I'll get this out to my users and ensure that it works and fine tune the options and etc....

[hbs]: https://github.com/Homebrew/homebrew-science/blob/master/blast.rb

* Flake8 fixes

* Update w.t.r. mamelara's PR and cleanup

Add paths to things in the --with-* config args

Add add'l config flags from #3394.

Update patch to no-op and add comment.

Add add'l dependencies after studying `configure --help`.

* Flake8 cleanup

* Add support for blast+@2.2.30

Need to work around a configure bug.  See https://github.com/Homebrew/homebrew-science/issues/2337#issuecomment-170011511.

* Re-wrap comment, line length not a problem.

I'd broken a comment across to line in a fit of flake8
induced fear.  Adam pointed out that it's ok.

* Remove explicit prefix config

Remove explicit prefix config, the autotools package does it for us.
2017-03-28 21:38:51 -05:00
George Hartzell
53b0d93a7a Add info for version 2.16 (#3601) 2017-03-28 21:37:57 -05:00
George Hartzell
14194a137a Add support for v1.9.9 (#3602) 2017-03-28 21:37:34 -05:00
George Hartzell
18a4be3e56 Add packages for jq and ongiruma (#3607)
jq is a commandline JSON processor.  Oniguruma is a regexp
library on which jq depends.
2017-03-28 21:37:10 -05:00
George Hartzell
83e661604d Add package for direnv (#3608) 2017-03-28 21:36:39 -05:00
George Hartzell
ec3eabfc7a Add package for BATS (#3611)
Add package for the BASH Automated Testing System.
2017-03-28 21:36:12 -05:00
George Hartzell
9385659aa1 bowtie2: add support for version 2.3.1 (#3599)
* Add support for 2.3.1

Updated digest and added a dependency on tbb.

* Cleanup patch name and add description
2017-03-28 21:35:46 -05:00
George Hartzell
d6cbae81c7 Update/samtools (#3595)
* Update samtools family to 1.4

* simple cleanup

* Make bzip depends_on for future versions too
2017-03-28 21:34:41 -05:00
George Hartzell
70ec06949c New package: ncftp client (#3603)
* Add package for ncftp

* Flake8 cleanups
2017-03-28 21:33:59 -05:00
George Hartzell
d94757238a New package: staden-io-lib (#3604)
* Add package for staden-io-lib

* Flake8 cleanups
2017-03-28 21:33:34 -05:00
George Hartzell
bdf0baead7 New package: mummer (#3606)
* Add package for mummer

Blocked on https://github.com/LLNL/spack/pull/3555.

* Make "check" step optional and flake8 cleanups
2017-03-28 21:27:20 -05:00
George Hartzell
81a0881a57 new package: pigz (#3612)
* Add package for pigz

* Whitespace

* Remove leftover instructions (oops...)

* Flake8 cleanup

* Convert to MakefilePackage
2017-03-28 21:15:30 -05:00
George Hartzell
e5eb8e34c5 new package: blat (#3610)
* Add package for blat

* Whitespace cleanup

* Flake8 cleanup
2017-03-28 21:14:26 -05:00
George Hartzell
1e5da65d65 Add packages for miniconda[23] (#3613) 2017-03-28 21:13:46 -05:00
George Hartzell
36f80bdfa4 Add support for version 20170322 (#3598) 2017-03-28 18:56:21 -05:00