Commit graph

7342 commits

Author SHA1 Message Date
Todd Gamblin
96d2488e0c Documentation for dependency patching. 2017-09-30 02:06:59 -07:00
Todd Gamblin
4f8c7d57eb Patches are hashed with specs, and can be associated with dependencies.
- A package can depend on a special patched version of its dependencies.

  - The `Spec` YAML (and therefore the hash) now includes the sha256 of
    the patch in the `Spec` YAML, which changes its hash.

  - The special patched version will be built separately from a "vanilla"
    version of the same package.

  - This allows packages to maintain patches on their dependencies
    without affecting either the dependency package or its dependents.
    This could previously be accomplished with special variants, but
    having to add variants means the hash of the dependency changes
    frequently when it really doesn't need to.  This commit allows the
    hash to change *just* for dependencies that need patches.

  - Patching dependencies shouldn't be the common case, but some packages
    (qmcpack, hpctoolkit, openspeedshop) do this kind of thing and it
    makes the code structure mirror maintenance responsibilities.

- Note that this commit means that adding or changing a patch on a
  package will change its hash.  This is probably what *should* happen,
  but we haven't done it so far.

  - Only applies to `patch()` directives; `package.py` files (and their
    `patch()` functions) are not hashed, but we'd like to do that in the
    future.

- The interface looks like this: `depends_on()` can optionally take a
  patch directive or a list of them:

     depends_on(<spec>,
                patches=patch(..., when=<cond>),
                when=<cond>)
     # or
     depends_on(<spec>,
                patches=[patch(..., when=<cond>),
                         patch(..., when=<cond>)],
                when=<cond>)

- Previously, the `patch()` directive only took an `md5` parameter.  Now
  it only takes a `sha256` parameter.  We restrict this because we want
  to be consistent about which hash is used in the `Spec`.

- A side effect of hashing patches is that *compressed* patches fetched
  from URLs now need *two* checksums: one for the downloaded archive and
  one for the content of the patch itself.  Patches fetched uncompressed
  only need a checksum for the patch.  Rationale:

  - we include the content of the *patch* in the spec hash, as that is
    the checksum we can do consistently for patches included in Spack's
    source and patches fetched remotely, both compressed and
    uncompressed.

  - we *still* need the patch of the downloaded archive, because we want
    to verify the download *before* handing it off to tar, unzip, or
    another decompressor.  Not doing so is a security risk and leaves
    users exposed to any arbitrary code execution vulnerabilities in
    compression tools.
2017-09-30 02:06:59 -07:00
Todd Gamblin
14c141a410 add spack flake8 exception for long checksums 2017-09-30 02:06:59 -07:00
Todd Gamblin
bf610a379f Clean up exceptions and function names in directives.
- Functions returned by directives were all called `_execute`, which made
  reading stack traces hard because you couldn't tell what directive a
  frame came from.
  - renamed them all to `_execute_<directive>`

- Exceptions in directives were only really used in one or two places --
  get rid of the boilerplate init functions and let the callsite specify
  the message.
2017-09-30 02:06:59 -07:00
Todd Gamblin
94d85d842c Consolidate some web-spidering commands in spack.util.web
- move `spack.cmd.checksum.get_checksums` to `spack.util.web.spider_checksums`

- move `spack.error.NoNetworkError` to `spack.util.web.NoNetworkError` since
  it is only used there.
2017-09-30 02:06:59 -07:00
Todd Gamblin
2198a0e229 Disable duplicate cross-reference warnings in Sphinx. 2017-09-30 02:06:59 -07:00
Todd Gamblin
0e8bb9ec5e Refactor Package dependency metadata
- Previously, dependencies and dependency_types were stored as separate
  dicts on Package.
  - This means a package can only depend on another in one specific way,
    which is usually but not always true.
  - Prior code unioned dependency types statically across dependencies on
    the same package.

- New code stores dependency relationships as their own object, with a
  spec constraint and a set of dependency types per relationship.
  - Dependency types are now more precise
  - There is now room to add more information to dependency relationships.

- New Dependency class lives in dependency.py, along with deptype
  definitions that used to live in spack.spec.

Move deptype definitions to spack.dependency
2017-09-30 02:06:59 -07:00
Todd Gamblin
a3cb6b61ea Patch.apply() shouldn't affect working directory of caller. 2017-09-30 02:06:59 -07:00
Todd Gamblin
29ce69f3ed Only print "no patches needed" if there were no patches. 2017-09-30 02:06:59 -07:00
scheibelp
9e7faff6c9 Add test deptype (#5132)
* Add '--test=all' and '--test=root' options to test either the root or the root and all dependencies.
* add a test dependency type that is only used when --test is enabled.
* test dependencies are not added to the spec, but they are provided in the test environment.
2017-09-29 22:08:15 -07:00
Yifan Zhu
3c0e799a51 r-limma: added version 3.32.7 (#5558) 2017-09-29 16:29:41 -06:00
Jon Rood
ebbdde9625 Masa requires python if +python. (#5547) 2017-09-29 16:14:12 -06:00
lingnanyuan
756fb83f04 r-a4classif: new package. (#5557) 2017-09-29 16:06:27 -06:00
lingnanyuan
17b8f6b818 r-yaml: Correct url (#5550)
* pr-quantities: add version & py-numpy conflict (#5307)

* r-yaml: Correct url.
2017-09-29 16:05:55 -06:00
Yifan Zhu
7c87cb1ad1 r-downloader: created new package (#5556) 2017-09-29 16:05:30 -06:00
lingnanyuan
f9e33e9b59 r-mlinterfaces: new package. (#5555)
* pr-quantities: add version & py-numpy conflict (#5307)

* r-mlinterfaces: Create new package.
2017-09-29 15:34:44 -06:00
Yifan Zhu
06a6c6a399 r-biomart: created new package (#5554) 2017-09-29 15:33:40 -06:00
lingnanyuan
18029f5f47 r-threejs: Correct url. (#5553)
* pr-quantities: add version & py-numpy conflict (#5307)

* r-threejs: Correct url.
2017-09-29 15:33:05 -06:00
Jon Rood
d19eb152c8 Adding new Trilinos version and updating Nalu to utilize it. (#5552) 2017-09-29 15:31:15 -06:00
lingnanyuan
e1f33bd903 r-hwriter: new package. (#5549)
* pr-quantities: add version & py-numpy conflict (#5307)

* r-hwriter: Create new package.
2017-09-29 15:29:26 -06:00
lingnanyuan
211f731065 r-gbm: new package. (#5548)
* pr-quantities: add version & py-numpy conflict (#5307)

* r-gbm: Create new package.
2017-09-29 15:29:11 -06:00
lingnanyuan
4cc1ff5ce8 r-htmlwidgets: Update version to 0.8. (#5551) 2017-09-29 14:28:48 -07:00
lingnanyuan
89b30bea7f r-a4preproc: Create new package. (#5499) 2017-09-29 15:27:20 -06:00
Massimiliano Culpo
554937780b modules: specialized configure_options for external packages (#5543)
closes #5473

Prior to this PR we were not exiting early for external packages, which
caused the `configure_options` property of the contexts to fail with
e.g. a key error because the DAG gets truncated for them. More
importantly Spack configure options don't make any sense for externals.

Now we exit early, and leave a message in the module file clarifying
that this package has been installed outside of Spack.
2017-09-29 09:23:13 -07:00
Yifan Zhu
a63fdc8f35 r-dbi: updated to version 0.7 (#5537)
* pr-quantities: add version & py-numpy conflict (#5307)

* r-dbi: Create new package.

* r-dbi: modified according to flake8

* Update package.py

* Update package.py
2017-09-28 22:14:56 -06:00
lingnanyuan
aab68aaecf r-gdata: Update version to 2.18.0 (#5495) 2017-09-28 18:13:06 -07:00
Mark C. Miller
e851d15974 fixing tag name for development branch (#5496) 2017-09-28 18:12:02 -07:00
Yifan Zhu
aee44efada r-biostrings: created new package (#5498) 2017-09-28 18:11:28 -07:00
Yifan Zhu
b455fa2a7b r-gcrma: created new package (#5500) 2017-09-28 18:11:02 -07:00
Yifan Zhu
e42fd90df2 r-yaqcaffy: created new package (#5538) 2017-09-28 18:02:58 -07:00
Jon Rood
5e3024368e Add metaphysicl package (#5539)
* Adding package Metaphysicl.

* Removing list_url from metaphysical package.
2017-09-28 18:02:07 -07:00
Jon Rood
90e4b56362 Adding masa package. (#5541) 2017-09-28 18:01:38 -07:00
Yifan Zhu
6d63be9eb6 r-rlang: created new package (#5535) 2017-09-28 16:07:13 -06:00
Christoph Junghans
337256ee32 tycho2: initial commit (#5196) 2017-09-28 14:48:54 -07:00
Mark Olesen
aff6148116 Restrict openfoam to using openmpi+thread_multiple (partially resolves #5428) (#5479)
- latest version requires openmpi+thread_multiple for collated output.
2017-09-28 14:42:14 -06:00
Michael Kuhn
97bc86d617 snappy: Build shared library by default (#5534) 2017-09-28 14:18:50 -06:00
lingnanyuan
aa40509fa0 r-ggvis: Update version to 0.4.3 (#5532)
* pr-quantities: add version & py-numpy conflict (#5307)

* r-ggvis: Update version to 0.4.3

* Update package.py

* Update package.py
2017-09-28 13:13:47 -06:00
lingnanyuan
0962088776 r-dplyr: Update version to 0.7.3 (#5531)
* pr-quantities: add version & py-numpy conflict (#5307)

* r-dplyr: Update version to 0.7.3

* Update package.py

* Update package.py
2017-09-28 13:09:30 -06:00
Michael Kuhn
ac98e0978a ncl: Add tcsh dependency (#5195)
All ncl scripts hardcode /bin/csh, so fix them up.
2017-09-28 12:51:04 -06:00
Michael Kuhn
17be89c210 tcsh: New package (#5194)
The patches are taken from the Fedora package.
2017-09-28 12:50:52 -06:00
lingnanyuan
0bc3578dac r-annaffy: new package. (#5414)
* r-annaffy: Create New Package.

* r-annaffy: Create new package.

* r-go-db: Create new package.

* r-kegg-db: Create new package.

* r-annaffy: Correct indentation.

* r-go-db: Correct indentation.

* r-kegg-db: Correct format.

* r-kegg-db: Correct format.

* r-go-db: Correct format.

* r-annaffy: Correct format.
2017-09-28 12:49:31 -06:00
Audrey Thoma
3f44eb0179 fastq-screen: new package (#5431)
* fastq-screen: new package

* fixing descriptions and previously made packages' old copyright date

* added wrong perl package after change
2017-09-28 12:48:31 -06:00
Brian Homerding
0ce93d7333 Added hacckernels package (#5533) 2017-09-28 12:46:26 -06:00
lingnanyuan
d61e92b439 r-r6: Update version to 2.2.2 (#5529)
* pr-quantities: add version & py-numpy conflict (#5307)

* r-r6: Update version to 2.2.2
2017-09-28 12:46:11 -06:00
lingnanyuan
c0e1b1f68e r-shiny: update version to 1.0.5 (#5530)
* pr-quantities: add version & py-numpy conflict (#5307)

* r-shiny: Update version to 1.0.5
2017-09-28 11:31:27 -06:00
lingnanyuan
a613244602 r-htmltools: Update version to 0.3.6 (#5528)
* pr-quantities: add version & py-numpy conflict (#5307)

* r-htmltools: Update version to 0.3.6
2017-09-28 11:31:16 -06:00
lingnanyuan
0c385e94e6 r-httpuv: Update version to 1.3.5 (#5527)
* pr-quantities: add version & py-numpy conflict (#5307)

* r-httpuv: Update version to 1.3.5
2017-09-28 11:30:57 -06:00
lingnanyuan
38ea408126 r-rcpp: Update version to 0.12.13 (#5526)
* pr-quantities: add version & py-numpy conflict (#5307)

* r-rcpp: Update version to 0.12.13

* r-rcpp: Correct url.
2017-09-28 11:30:25 -06:00
Kelly (KT) Thompson
4e4247dd18 Provide a new Perl package 'cloc'. (#5525)
+ Count, or compute differences of, physical lines of source code in the
  given files (may be archives such as compressed tarballs or zip files)
  and/or recursively below the given directories.
2017-09-28 11:30:08 -06:00
lingnanyuan
e101665e5e r-jsonlite: Update version to 1.5 (#5524)
* pr-quantities: add version & py-numpy conflict (#5307)

* r-jsonlite: Update version to 1.5
2017-09-28 11:28:49 -06:00