Commit graph

6868 commits

Author SHA1 Message Date
Christoph Junghans
fa1d0a8a4d Add --source option to spack install (#4102)
- -- source will copy source into prefix along with the package.
- added a test for --source, as well
2017-08-23 14:08:52 -07:00
Audrey Thoma
359b21c888 pgdspider: new package (#5186)
* pgdspider: new package

* couple of fixes
2017-08-23 08:34:12 -05:00
Axel Huebl
0daa639788 Boost: develop (#5184)
Adds a development version of boost, based on their github master
repo.
Git clones the submodules forever, but installs! :)

Useful for testing since boost tends to break complex software
dependencies quite often. This helps testing it before releases.
2017-08-23 07:31:34 -05:00
healther
15186d4ae8 add test for lua and node patching in sbang (#5169)
* add test for lua and node handling in sbang patching (cf #5086)
2017-08-23 01:05:36 -07:00
Todd Gamblin
40e9171390 Add testing for new build output.
- Update handling of ChildError so that its output is capturable from a
  SpackCommand

- Update cmd/install test to make sure Python and build log output is
  being displayed properly.
2017-08-22 16:14:38 -07:00
Todd Gamblin
f51b541ef8 Make install command reusable within single Spack run
- install and probably other commands were designed to run once, but now
  we can to test them from within Spack with SpackCommand

- cmd/install.py assumed that it could modify do_install in PackageBase
  and leave it that way; this makes the decorator temporary

- package.py didn't properly initialize its stage if the same package had
  been built successfully before (and the stage removed).
  - manage stage lifecycle better and remember when Package needs to
    re-create the stage
2017-08-22 16:14:38 -07:00
Todd Gamblin
fa1faa61c4 SpackCommand uses log_output to capture command output. 2017-08-22 16:14:38 -07:00
Todd Gamblin
4f444c5f58 log_ouptut can take either a filename or a file object 2017-08-22 16:14:38 -07:00
Todd Gamblin
139d5bfa6b Display build output on ProcessError, instead of Python context.
- If a failure comes from an external command and NOT the Python code,
  display errors highlighted with some context.

- Add some rudimentary support for parsing errors out of the build log
  (not very sophisticated yet).

- Build errors in Python code will still display with Python context.
2017-08-22 16:14:38 -07:00
Todd Gamblin
d54110d208 Limit package context to 3 lines and colorize in error output. 2017-08-22 16:14:38 -07:00
Massimiliano Culpo
4600d106e2 Config scopes are now returning OrderedDicts instead of dicts. (#5183)
It seems 8f21332fec introduced a bug
in that normal dictionaries are returned from ConfigScope objects
instead of OrderedDicts. This is fixed here.
2017-08-22 14:20:19 -07:00
Audrey Thoma
79df4db8e5 tcoffee: new package (#5185) 2017-08-22 13:48:23 -05:00
Axel Huebl
61c5b5faa7 Boost: 1.65.0 (#5182)
Adds the latest boost release.
Downloads seem to be pointing to bintray from the official homepage.
2017-08-22 13:46:54 -05:00
Sergey Kosukhin
f52bdeda2a Package 'libaec': no need to specify per version urls. (#5176) 2017-08-22 09:27:03 +02:00
Sergey Kosukhin
e55efdf8ed Updates for package 'cdo'. (#5179)
* Package 'cdo': removed obsolete version 1.6.9.

* Package 'cdo': updated urls.

* Package 'cdo': added version 1.9.0.

* Package 'cdo': no need to specify per version urls.

* Package 'cdo': switched from Package to AutotoolsPackage.

* Package 'cdo': added new variant 'openmp'.

* Package 'cdo': variants 'curl' and 'magics' by default.
2017-08-22 08:42:55 +02:00
Matthew Scott Krafczyk
48bf1e276b Add environment variables to path substitution
Update documentation on config file variable substitutions and
add expansion of environment variables in config files.
2017-08-21 18:35:00 -07:00
Audrey Thoma
e0ebf44239 py-pyrad: new package (#5181) 2017-08-21 17:58:05 -05:00
Massimiliano Culpo
581f70ff6f Added custom messages for conflicts directive. fixes #4965 (#5083)
Users can now add an optional custom message to the conflicts directive.
Layout on screen has been changed to improve readability and the long
spec is shown in tree format. Two conflicts in `espresso` have been
modified to showcase the feature.
2017-08-21 09:20:07 -07:00
Axel Huebl
bd8ac0a738 Protobuf: 3.4 (#5177) 2017-08-21 07:29:47 -05:00
Todd Gamblin
10bb681b57 Unbuffer so that output from packages appears when redirecting
- Python I/O would not properly interleave (or appear) with output from
  subcommands.

- Add a flusing wrapper around sys.stdout and sys.stderr when
  redirecting, so that Python output is synchronous with that of
  subcommands.
2017-08-20 16:51:10 -07:00
Todd Gamblin
79045afada Add tests for output redirection. 2017-08-20 16:51:10 -07:00
Todd Gamblin
11196e7b69 Preserve verbosity across installs when 'v' is pressed.
- 'v' toggle was previously only good for the current install.
- subsequent installs needed user to press 'v' again.
- 'v' state is now preserved across dependency installs.
2017-08-20 16:51:10 -07:00
Todd Gamblin
48440766df Fix exit call in SpackError.die()
- Previously we would use `os._exit()` in to avoid Spack error handling
  in the parent process when build processes failed.  This isn't
  necessary anymore since build processes propagate their exceptions to
  the parent process.

- Use `sys.exit` instead of `os._exit`. This has the advantage of
  automatically flushing output streams on quit, so output from child
  processes is not lost when Spack exits.
2017-08-20 16:51:10 -07:00
Todd Gamblin
05cc6c966f Rework output redirection in Spack.
- Simplify interface to log_output. New interface requires only one
  context handler instead of two.  Before:

      with log_output('logfile.txt') as log_redirection:
           with log_redirection:
               # do things ... output will be logged

  After:

      with log_output('logfile.txt'):
          # do things ... output will be logged

  If you also want the output to be echoed to ``stdout``, use the
  `echo` parameter::

      with log_output('logfile.txt', echo=True):
          # do things ... output will be logged and printed out

  And, if you just want to echo *some* stuff from the parent, use
  ``force_echo``:

      with log_output('logfile.txt', echo=False) as logger:
          # do things ... output will be logged

          with logger.force_echo():
              # things here will be echoed *and* logged

  A key difference between this and the previous implementation is that
  *everything* in the context handler is logged.  Previously, things like
  `Executing phase 'configure'` would not be logged, only output to the
  screen, so understanding phases in the build log was difficult.

- The implementation of `log_output()` is different in two major ways:

  1. This implementation avoids race cases by using only one pipe (before
     we had a multiprocessing pipe and a unix pipe).  The logger daemon
     stops naturally when the input stream is closed, which avoids a race
     in the previous implementation where we'd miss some lines of output
     because the parent would shut the daemon down before it was done
     with all output.

  2. Instead of turning output redirection on and off, which prevented
     some things from being logged, this version uses control characters
     in the output stream to enable/disable forced echoing.  We're using
     the time-honored xon and xoff codes, which tell the daemon to echo
     anything between them AND write it to the log.  This is how
     `logger.force_echo()` works.

- Fix places where output could get stuck in buffers by flushing more
  aggressively.  This makes the output printed to the terminal the same
  as that which would be printed through a pipe to `cat` or to a file.
  Previously these could be weirdly different, and some output would be
  missing when redirecting Spack to a file or pipe.

- Simplify input and color handling in both `build_environment.fork()`
  and `llnl.util.tty.log.log_output()`.  Neither requires an input_stream
  parameter anymore; we assume stdin will be forwarded if possible.

- remove `llnl.util.lang.duplicate_stream()` and remove associated
  monkey-patching in tests, as these aren't needed if you just check
  whether stdin is a tty and has a fileno attribute.
2017-08-20 16:51:10 -07:00
Todd Gamblin
e0dd55e090 Make SpackCommand a bit more testable
- add fail_on_error argument
- record exception and return code when the command fails
2017-08-20 16:51:10 -07:00
Todd Gamblin
e77c1a20c5 Fix issue with color formatting regular expression. (#5171)
- Fix issue with color formatting regular expression.
  - _separators regex in spec.py could be constructed such that '^' came
    first in the character matcher, e.g. '[^@#/]'.  This inverts the match
    and causes transient KeyErrors.

  - Fixed to escape all characters in the constructed regex.

  - This bug comes up in Python3 due to its more randomized hash iteration
    order, but it could probably also happen in a Python 2 implementation.

- also clean up variable docstrings in spec.py
2017-08-20 15:34:35 -07:00
Massimiliano Culpo
99fb394ac1 Group Travis CI jobs in stages (#5104)
- This should speed-up Travis CI tests and refers to #5049

- Travis uses build-stages to group tests together
    - The idea is to let fast tests fail first, then move to longer ones.
    - Added external perl to avoid download failure from CPAN and reduce build time
    - Disabling perl-dbi: continues to fail with (504 Gateway Time-out) on Travis

- We now cover all the build systems in tests:
    - Add back `openblas` to Travis as a separate package.
    - Switched `openblas` for `astyle` to build a simpler MakefilePackage.
    - Added 'tut' (WafPackage)
    - Added 'py-setuptools' (PythonPackage)
    - Added 'perl-dbi' (PerlPackage)
    - Added 'build_systems' directory to the ones for which we get a summary
    - Added 'openjpeg' (CMakePackage)
    - Added 'r-rcpp' (RPackage)
    - Added comments to build tests to show the covered build system
2017-08-19 14:52:27 -07:00
Todd Gamblin
bb10bc39ab Make test_inspect_path work on Mac OS X without XCode (#5168)
- Mac OS X Sierra has no /usr/include by default
- Instead of assuming there's an include directory in /usr, mock up a directory that looks like we expect.
2017-08-19 14:48:24 -07:00
健美猫
be0e051870 Add version 1.7.1 for cub. (#5164) 2017-08-19 09:22:19 -05:00
Adam Moody
d80c70538f lbann: add depends_on hwloc (#5165) 2017-08-19 01:00:52 -05:00
Audrey Thoma
fa39afb598 cufflinks: new package (#5158) 2017-08-18 16:02:15 -05:00
Audrey Thoma
3d1c5f679f muscle: fix (#5160) 2017-08-18 16:01:30 -05:00
Kelly (KT) Thompson
c41c5bc9b8 For Tk, also provide a symlink for wishX.X. (#5161)
+ gitk (and possibly other tools) expect to find the Tk executable `wish`, so
  add a symlink with this name that points to the the versioned filename.
+ Example: `wish --> wish8.6`
2017-08-18 16:01:02 -05:00
healther
d6d2dff324 sbang support: add node-js and fix lua
This adds sbang hook support for node-js and fixes the sbang filter
for lua (the character class exclusion was swallowing newlines and
reporting a false positive if lua was mentioned anywhere in the 
file).
2017-08-18 11:57:52 -07:00
healther
6d155833c5 build boost with graph library even when +mpi
Previously, when +graph and +mpi were enabled, the graph library
was replaced with the graph_parallel library. This alters the
logic for +graph+mpi to build both libraries.
2017-08-18 11:19:55 -07:00
Audrey Thoma
9ad8e79332 corset: new package (#5157) 2017-08-18 11:02:58 -05:00
Audrey Thoma
6abe8e7d91 parsimonator: new package (#5143)
* parsimonator: new package

* adding variant conflict
2017-08-18 09:36:35 -05:00
Audrey Thoma
5fb23b21e0 vegas2: new package (#5150)
* vegas2: new package

* small version things
2017-08-18 09:36:18 -05:00
George Hartzell
9af953a4cb Add info for nextflow@v0.25.6 (#5148) 2017-08-18 09:36:02 -05:00
Stephen McDowell
61238baf56 add man-db package and friends (#4844)
* add man-db package and friends

* fix groff, libpipeline, and man-db dependencies

* remove parallel groff altogether, build-only deps
2017-08-18 14:17:25 +02:00
zenln
6d2092e1eb Add a new package for gaussian (#5149)
* Gaussian: Gaussian install

* Adding Patch to spack installed files

* Adding extra environment variables for gaussian at run time

* Adding provides linda

* Adding linda as a provided package

* Shortning vars and fixing flake8 indentation and spacings
2017-08-17 18:33:44 -05:00
Audrey Thoma
cf796ed595 pagit: new package (#5140)
* pagit: new package

* removed unneeded env var
2017-08-17 15:32:30 -05:00
Audrey Thoma
94af7ccb77 new package libfabric (#5146)
* pbmpi: new package

* forgot about libfabric
2017-08-17 15:31:31 -05:00
Audrey Thoma
cb4ede001c pbmpi: new package (#5144) 2017-08-17 11:59:02 -05:00
Axel Huebl
6472c39c2e Docs: Travis-CI Workflow (#5133)
* Docs: Travis-CI Workflow

Add a workflow how to use spack on Travis-CI.

Future Work:

depending if and how we can simplify 5101:
add a multi-compiler, multi-C++-standard, multi-software
build matrix example

* Fix Typos
2017-08-17 11:25:40 -05:00
Audrey Thoma
46d8cb8913 paml: new package (#5142) 2017-08-17 11:21:56 -05:00
Massimiliano Culpo
0f5582cefc Colorize spack info. Adds prominence to preferred version. (#4994)
* Colorize spack info. Adds prominence to preferred version. fixes #2708

This uses 'llnl.util.tty.color' to colorize the output of 'spack info'.
It also displays versions in the order the concretizer would choose
them and shows the preferred in a line on its own and in bold.

* Modified output according to Adam and Denis reviews.

Section titles are not bold + black, but bold + blue. Added a new
section named "Preferred version", which prints the preferred version
in bold characters.

* Further modifications according to Adam and Denis reviews.

After "Homepage:" we now have a single space. Removed newline after each
variant. Preferred version is not in bold fonts anymore. Added a simple
test that just runs the command.
2017-08-17 09:15:57 -07:00
Audrey Thoma
aa21590099 microbiomeutil: new package (#5134)
* microbiomeutil: new package

* got rid of unnecessary self

* oops
2017-08-17 09:19:00 -05:00
Adam J. Stewart
187f4420e2 Add new package for rhash, a cmake dependency (#5120)
* Add new package for rhash, a cmake dependency

* Add missing libuv dependency to cmake
2017-08-17 08:17:45 -05:00
Audrey Thoma
3bb7767839 py-faststructure: new package (#5131) 2017-08-17 07:49:57 -05:00