Commit graph

106 commits

Author SHA1 Message Date
Michael Kuhn
f45916fad2 Set LANG= for _spack_fn_exists (#5475)
type's output can be localized, causing the grep to fail.
2017-09-26 12:28:50 -07:00
Matthew Scott Krafczyk
0bd838bd11 Remove echo statements from setup-env.sh
setup-env.sh adds the 'module' command to the user's environment
if it is not defined and if there is a Spack installation of
environment-modules available. This commit updates that logic to
perform these checks and updates quietly.
2017-09-14 18:49:25 -07:00
Christoph Junghans
80ac613391 Shell detection: filter preceding "-"
On OSX, the shell detection code may get a preceding dash, like
"-bash". This adds a filter to remove it.
2017-09-12 12:17:20 -07:00
Matthew Scott Krafczyk
f57559e4e2 Fix two bugs from the bootstrap update (#5312)
These were discovered with bash 4.1.2.

Add quotations around a variable to prevent the destruction of a
newline. Without this fix a conditional doesn't work properly.

Remove square brackets around a conditional meant to be evaluated based
on the return code of a command. This wasn't working properly with an
old bash.

Fix a typo.
2017-09-10 16:54:23 -10:00
Matthew Scott Krafczyk
51828dd982 Bootstrap environment-modules
Renames the existing bootstrap command to 'clone'. Repurposes
'spack bootstrap' to install packages that are useful to the
operation of Spack (for now this is just environment-modules).
For bash and ksh users running setup-env.sh, if a Spack-installed
instance of environment-modules is detected and environment modules
and dotkit are not externally available, Spack will define the 
'module' command in the user's shell to use the environment-modules
built by Spack.
2017-09-08 10:15:06 -07:00
Michael Kuhn
84ae7872d3 Update copyright notices for 2017 (#5295) 2017-09-06 17:44:16 -10:00
Matthew Scott Krafczyk
c94933343a Add --show-full-compiler option to 'spack find'
When 'spack find' is invoked with the '--show-full-compiler' option,
the compiler flags and version are shown for each spec that is found.
2017-08-28 10:35:46 -07:00
Christoph Junghans
fa1d0a8a4d Add --source option to spack install (#4102)
- -- source will copy source into prefix along with the package.
- added a test for --source, as well
2017-08-23 14:08:52 -07:00
Massimiliano Culpo
99fb394ac1 Group Travis CI jobs in stages (#5104)
- This should speed-up Travis CI tests and refers to #5049

- Travis uses build-stages to group tests together
    - The idea is to let fast tests fail first, then move to longer ones.
    - Added external perl to avoid download failure from CPAN and reduce build time
    - Disabling perl-dbi: continues to fail with (504 Gateway Time-out) on Travis

- We now cover all the build systems in tests:
    - Add back `openblas` to Travis as a separate package.
    - Switched `openblas` for `astyle` to build a simpler MakefilePackage.
    - Added 'tut' (WafPackage)
    - Added 'py-setuptools' (PythonPackage)
    - Added 'perl-dbi' (PerlPackage)
    - Added 'build_systems' directory to the ones for which we get a summary
    - Added 'openjpeg' (CMakePackage)
    - Added 'r-rcpp' (RPackage)
    - Added comments to build tests to show the covered build system
2017-08-19 14:52:27 -07:00
Adam J. Stewart
af02774b3e Add tab completion & update docs for buildcache
This adds tab completion and fixes some formatting issues in the
documentation for the "spack buildcache" command.
2017-08-16 13:58:09 -07:00
Massimiliano Culpo
faeb1b77b2 Merged 'purge' command with 'clean' and deleted 'purge' (#4970)
* Merged 'purge' command with 'clean'. Deleted 'purge'. fixes #2942

'spack purge' has been merged with 'spack clean'. Documentation has been
updated accordingly. The 'clean' and 'purge' behavior are not mutually
exclusive, and they log brief information to tty while they go.

* Fixed a wrong reference to spack clean in the docs

* Added tests for 'spack clean'. Updated bash completion.
2017-08-09 10:02:38 -07:00
paulhopkins
1c7e5724d9 Add --color=[always|never|auto] argument; fix color when piping (#3013)
* Disable spec colorization when redirecting stdout and add command line flag to re-enable
* Add command line `--color` flag to control output colorization
* Add options to `llnl.util.tty.color` to allow color to be auto/always/never
* Add `Spec.cformat()` function to be used when `format()` should have auto-coloring
2017-07-31 12:57:47 -07:00
Jon Rood
158f99f9ad Return an error exit code if spack cd does not succeed. (#4623)
* Return an error exit code if spack cd does not succeed.

* Reducing amount of return statements in spack cd exit code.
2017-06-28 06:41:04 -05:00
Todd Gamblin
cac4362f64 Make LICENSE recognizable by GitHub. (#4598) 2017-06-24 22:22:55 -07:00
Adam J. Stewart
0bbafb1673 Fix tab completion of Spack subcommands (#4442) 2017-06-07 11:52:07 -05:00
Adam J. Stewart
cafc3cc3ca Sphinx no longer supports Python 2.6 (#4266)
* Sphinx no longer supports Python 2.6

* Update vendored sphinxcontrib.programoutput from 0.9.0 to 0.10.0

* Documentation cannot be built in parallel

* Let Travis install programoutput for us

* Remove vendored sphinxcontrib-programoutput

Recent updates to the sphinx package prevent the vendored version
from being found in sys.path. We don't vendor sphinx, so it doesn't
make sense to vendor sphinxcontrib-programoutput either.
2017-05-17 11:36:02 -05:00
Matthew Thompson
604f65f395 Edits to get setup-env.csh working better (#4044)
* Edits to get setup-env.csh working better.

Autosets the sys_type a la setup-env.sh

* More stealing from bash setup script for module roots

* Add error message if SPACK_ROOT isn't set

* Remove _sp_lmod_root per Adam J Stewart
2017-05-10 10:18:45 -05:00
Todd Gamblin
ff3b5d88e4 rework spack help (#3033)
- Full help is now only generated lazily, when needed.
  - Executing specific commands doesn't require loading all of them.
  - All commands are only loaded if we need them for help.

- There is now short and long help:
  - short help (spack help) shows only basic spack options
  - long help (spack help -a) shows all spack options
  - Both divide help on commands into high-level sections

- Commands now specify attributes from which help is auto-generated:
  - description: used in help to describe the command.
  - section: help section
  - level: short or long

- Clean up command descriptions

- Add a `spack docs` command to open full documentation
  in the browser.

- move `spack doc` command to `spack pydoc` for clarity

- Add a `spack --spec` command to show documentation on 
  the spec syntax.
2017-05-08 13:18:29 -07:00
Massimiliano Culpo
060351e121 bash completion: fixed invalid identifier (#4079)
* bash completion: fixed `_spack_create-db-tarball': not a valid identifier

* bash completion: dashes are translated to underscores

This also fixes the name of the subfunction to be called, as apparently
it was not updated after moving the command `create-db-tarball`.
2017-05-03 08:12:33 -05:00
Todd Gamblin
094d47bff1 Allow user to specify profile sort column on the command line. (#4056)
- Add -P <STAT> argument so that caller can specify a sort column for
  cProfile. Can specify multiple columns with commas. e.g.:
      spack -P cumtime,module

- Add --lines option to Spack spec to control number of profile lines
  displayed

- Sort by time by default (because it works in all Python versions)

- Show sort column options in command help.

- Do a short profile run in the unit tests.
2017-05-01 14:32:33 -07:00
Todd Gamblin
bb5a433a46 Separate integration tests; simplify test scripts (#4006)
* Separate build integration tests; simplify test scripts

- Move build tests out of the regular Travis unit tests, add more smoke
  test packages to build.

- Run all test scripts with bash -e, which fails on error.

- Factor coverage out into a Travis environment variable, so it's more
  obvious from .travis.yml which tests contribute to coverage and which
  don't.

- Factor dependency checking and much of the front-matter in tests
  scripts into a setup.sh script, which is sourced by all the test
  scripts.  Extra cruft in each tests script now reduced to 2 lines at
  the beginning.
2017-04-27 11:47:56 -07:00
Todd Gamblin
4572685710 Coverage for multiple Python versions. (#3951)
Update tests to use codecov for multiple python versions.
2017-04-21 17:41:30 -07:00
Todd Gamblin
beeca6bb54 Revert "Override partial installs by default" (#3918)
* Revert "Override partial installs by default (#3530)"

This reverts commit a65c37f15d.
2017-04-20 03:53:41 -07:00
Todd Gamblin
fa3e91bf2a Revert "Add lmod files to MODULEPATH" (#3917)
* Revert "Add lmod files to MODULEPATH (#3912)"

This reverts commit 186d1f4511.
2017-04-20 03:05:44 -07:00
scheibelp
a65c37f15d Override partial installs by default (#3530)
* Package install remove prior unfinished installs

Depending on how spack is terminated in the middle of building a
package it may leave a partially installed package in the install
prefix. Originally Spack treated the package as being installed if
the prefix was present, in which case the user would have to
manually remove the installation prefix before restarting an
install. This commit adds a more thorough check to ensure that a
package is actually installed. If the installation prefix is present
but Spack determines that the install did not complete, it removes
the installation prefix and starts a new install; if the user has
enabled --keep-prefix, then Spack reverts to its old behavior.

* Added test for partial install handling

* Added test for restoring DB

* Style fixes

* Restoring 2.6 compatibility

* Relocated repair logic to separate function

* If --keep-prefix is set, package installs will continue an install from an existing prefix if one is present

* check metadata consistency when continuing partial install

* Added --force option to make spack reinstall a package (and all dependencies) from scratch

* Updated bash completion; removed '-f' shorthand for '--force' for install command

* dont use multiple write modes for completion file
2017-04-19 21:59:18 -07:00
Adam J. Stewart
186d1f4511 Add lmod files to MODULEPATH (#3912) 2017-04-19 21:30:08 -07:00
Adam J. Stewart
50df071ad9 Overhaul Spack's URL parsing (#2972)
* Remove fake URLs from Spack

* Ignore long lines for URLs that start with ftp:

* Preliminary changes to version regexes

* New redesign of version regexes

* Allow letters in version-only

* Fix detection of versions that end in Final

* Rearrange a few regexes and add examples

* Add tests for common download repositories

* Add test cases for common tarball naming schemes

* Finalize version regexes

* spack url test -> spack url summary

* Clean up comments

* Rearrange suffix checks

* Use query strings for name detection

* Remove no longer necessary url_for_version functions

* Strip off extraneous information after package name

* Add one more test

* Dot in square brackets does not need to be escaped

* Move renaming outside of parse_name_offset

* Fix versions for a couple more packages

* Fix flake8 and doc tests

* Correctly parse Python, Lua, and Bio++ package names

* Use effective URLs for mfem

* Add checksummed version to mitos

* Remove url_for_version from STAR-CCM+ package

* Revert changes to version numbers with underscores and dashes

* Fix name detection for tbb

* Correctly parse Ruby gems

* Reverted mfem back to shortened URLs.

* Updated instructions for better security

* Remove preferred=True from newest version

* Add tests for new `spack url list` flags

* Add tests for strip_name_suffixes

* Add unit tests for version separators

* Fix bugs related to parseable name but in parseable version

* Remove dead code, update docstring

* Ignore 'binary' at end of version string

* Remove platform from version

* Flip libedit version numbers

* Re-support weird NCO alpha/beta versions

* Rebase and remove one new fake URL

* Add / to beginning of regex to avoid picking up similarly named packages

* Ignore weird tar versions

* Fix bug in url parse --spider when no versions found

* Less strict version matching for spack versions

* Don't rename Python packages

* Be a little more selective, version must begin with a digit

* Re-add fake URLs

* Fix up several other packages

* Ignore more file endings

* Add parsing support for Miniconda

* Update tab completion

* XFAILS are now PASSES for 2 web tests
2017-04-03 15:34:16 -07:00
Adam J. Stewart
c0cfaacbc8 Set default providers for all virtual dependencies (#3634)
* Set default providers for everything

* Add default OpenFOAM provider
2017-04-02 14:48:27 -05:00
Mark Olesen
9e1abb13dc support OpenFOAM package(s) (#3528)
* ENH: add package for building OpenFOAM (1612) from www.openfoam.com
- provide 'openfoam' as virtual package.
- package as openfoam-com to reflect the distribution point.

This initial spack packaging for OpenFOAM supports a number of possible
variants and should handle 64-bit labels properly now that the scotch
package has been updated accordingly.

* ENH: update package for foam-extend (extend-project.de)

- provide 'openfoam' as virtual package.

- much of the build is now aligned with how the openfoam-com package
  looks, with the aim of future refactoring.

- avoid installing intermediate targets.

- contains its own environment sourcing script for the build, for more
  flexibility and robustness (doesn't touch the python build environ)

* ENH: added package for building from openfoam.org

- provide 'openfoam' as a virtual package.

- this is largely a direct copy of the openfoam-com package.
  It has been supplied as a courtesy for users and to ensure maximum
  consistency in quality and naming between the foam-extend,
  openfoam-com and openfoam-org packages.

* CONFIG: add openfoam into bash completion providers list

* ENH: have openfoam-com use spack as USERMPI

- also simplify the generation of mplib/compiler rules

* ENH: have openfoam-org use spack as SYSTEMMPI

- this setup requires more environment settings than USERMPI
  (openfoam-com), but is currently the only means of integration
  for openfoam-org

- simplify generation of mplib/compiler rules

* ENH: simplify generation of mplib/compiler rules (foam-extend)

- rename mpi rules from SPACK,SPACKMPI to USER,USERMPI for consistency
  with openfoam-com and to generalize for any build system.

* STYLE: record spack tree as a log file (openfoam)

- can be useful for future diagnostics and general record keeping
2017-03-30 16:35:57 -07:00
Adam J. Stewart
f0e99456ed Fix tab completion of directories and filenames (#3184) 2017-02-20 15:44:58 -08:00
Adam J. Stewart
c2d210568c Fix tab completion after change to uninstall flag (#3175) 2017-02-18 13:08:59 -06:00
Adam J. Stewart
941dfcbe43 Bash Programmable Completion for Spack (#3026) 2017-02-06 12:34:35 -08:00
Michael Kuhn
258dfc707d Export spack function so it works in subshells (#2908) 2017-01-23 15:29:25 -08:00
Massimiliano Culpo
34d23c617c tests: coverage combine run only if tests succeeded (#2712) 2017-01-01 13:05:33 -08:00
Todd Gamblin
b5537553f2 Run coverage for the spack install command as well as spack test.
- Also don't run coverage on OSX.
2016-12-30 17:18:19 -08:00
Massimiliano Culpo
7ea10e768e unit tests: replace nose with pytest (#2502)
* Porting: substitute nose with ytest

This huge commit substitutes nose with pytest as a testing system. Things done here:

* deleted external/nose as it is no longer used
* moved mock resources in their own directory 'test/mock/'
* ported two tests (cmd/find, build_system) to pytest native syntax as an example
* build_environment, log: used monkeypatch instead of try/catch
* moved global mocking of fetch_cache to an auto-used fixture
* moved global mocking from test/__init__.py to conftest.py
* made `spack test` a wrapper around pytest
* run-unit-tests: avoid running python 2.6 tests under coverage to speed them up
* use `pytest --cov` instead of coverage run to cut down testing time

* mock/packages_test: moved mock yaml configuration to files instead of leaving it in the code as string literals

* concretize.py: ported tests to native pytest, reverted multiprocessing in pytest.ini as it was creating the wrong report for coveralls

* conftest.py, fixtures: added docstrings

* concretize_preferences.py: uses fixtures instead of subclassing MockPackagesTest

* directory_layout.py: uses fixtures instead of subclassing MockPackagesTest

* install.py: uses fixtures instead of subclassing MockPackagesTest

* optional_deps.py: uses fixtures instead of subclassing MockPackagesTest

optional_deps.py: uses fixtures instead of subclassing MockPackagesTest

* packages.py: uses fixtures instead of subclassing MockPackagesTest

* provider_index.py: uses fixtures instead of subclassing MockPackagesTest

* spec_yaml.py: uses fixtures instead of subclassing MockPackagesTest

* multimethod.py: uses fixtures instead of subclassing MockPackagesTest

* install.py: now uses mock_archive_url

* git_fetch.py: uses fixtures instead of subclassing MockPackagesTest

* hg_fetch.py: uses fixtures instead of subclassing MockPackagesTest

* svn_fetch.py, mirror.py: uses fixtures instead of subclassing MockPackagesTest
repo.py: deleted

* test_compiler_cmd.py: uses fixtures instead of subclassing MockPackagesTest

* cmd/module.py, cmd/uninstall.py: uses fixtures instead of subclassing MockDatabase

* database.py: uses fixtures instead of subclassing MockDatabase, removed mock/database

* pytest: uncluttering fixture implementations

* database: changing the scope to 'module'

* config.py: uses fixtures instead of subclassing MockPackagesTest

* spec_dag.py, spec_semantics.py: uses fixtures instead of subclassing MockPackagesTest

* stage.py: uses fixtures instead of subclassing MockPackagesTest. Removed mock directory

* pytest: added docstrings to all the fixtures

* pytest: final cleanup

* build_system_guess.py: fixed naming and docstrings as suggested by @scheibelp

* spec_syntax.py: added expected failure on parsing multiple specs closes #1976

* Add pytest and pytest-cov to Spack externals.

* Make `spack flake8` ignore externals.

* run-unit-tests runs spack test and not pytest.

* Remove all the special stuff for `spack test`

- Remove `conftest.py` magic and all the special case stuff in `bin/spack`

- Spack commands can optionally take unknown arguments, if they want to
  handle them.

- `spack test` is now a command like the others.

- `spack test` now just delegates its arguments to `pytest`, but it does
  it by receiving unknown arguments and NOT taking an explicit
  help argument.

* Fix error in fixtures.

* Improve `spack test` command a bit.

- Now supports an approximation of the old simple interface
- Also supports full pytest options if you want them.

* Use external coverage instead of pytest-cov

* Make coverage use parallel-mode.

* change __init__.py docs to include pytest
2016-12-29 07:48:48 -08:00
Adam J. Stewart
6cc9b31747 Allow shell support to work when set -u is set (#2418) 2016-11-29 17:04:29 -08:00
Gregory Becker
7201053f2a Basic usage step-by-step tutorial 2016-11-12 16:55:37 -07:00
Massimiliano Culpo
296a349d49 setup script for bash : fixes #2209 for sh setup (#2212) 2016-11-04 12:00:01 -07:00
Todd Gamblin
1b7f9e24f4 Add spack flake8 command. (#2186)
- Ported old run-flake8-tests qa script to `spack flake8` command.
- New command does not modify files in the source tree
- Copies files to a temp stage modifies them there, and runs tests.
- Updated docs and `run-flake8-tests` script to call `spack flake8`.
2016-10-31 11:40:20 -07:00
Joseph Ciurej
82fcc5e1de Feature Proposal : Make All Python Extensions User Configuration Independent (#1435)
* Updated all Python extension packages to use 'setup_py' on install.

* Fixed a few minor style issues with the updated Python packages.
2016-10-06 14:43:49 -07:00
Adam J. Stewart
d5e5e4ed1a Add missing dependencies for Flake8 (#1883)
* Warn user if flake8 can't find setuptools

* Add missing dependencies of flake8

* Updates to py-autopep8, make packages activateable

* Check for presence of setuptools for Sphinx too

* Fix bug in order of commands
2016-10-05 10:00:31 -07:00
Adam J. Stewart
2ce1b388c9 Run make clean to prevent warning messages (#1742)
* Run make clean to prevent warning messages

* Don't delete temporary files after completion
2016-10-02 18:51:09 -07:00
Adam J. Stewart
64194a823a Remove duplicate ICU package (#1837)
* Remove duplicate ICU package

* Ignore deleted files during flake8 tests

* Rename Boost ICU variant
2016-09-23 14:16:59 -07:00
Adam J. Stewart
06d8a3ad23 Add missing dependency for run-doc-tests (#1691) 2016-09-08 09:02:38 -07:00
Adam J. Stewart
f6d07b54f1 Always run spack unit tests 2016-08-30 16:01:00 -05:00
Adam J. Stewart
e04b76c2c3 Add spack to the PATH for doc tests 2016-08-30 15:35:58 -05:00
Adam J. Stewart
a0c8aca3a2 Remove py-mercurial, going with mercurial package instead 2016-08-30 15:35:58 -05:00
Adam J. Stewart
f856952728 Always run Documentation tests 2016-08-30 15:32:54 -05:00
Adam J. Stewart
c36f13e44d Added more missing dependencies 2016-08-30 15:21:15 -05:00