Commit graph

1991 commits

Author SHA1 Message Date
Elizabeth Fischer
3de3664671 Fix fetch() method. Was being called with "wrong" arguments, raising exception. (#1916)
DIYFetchBugfix
2016-10-06 02:40:24 -07:00
Elizabeth Fischer
208537f6f2 Fix Issues with non-numeric versions, as well as preferred=True (#1561)
* Fix bug in handling of precedence of preferred=True vs. versions given in packages.yaml (#1556)

* Standardized comparison of versions: numeric versions are always greater than non-numeric versions; and non-numeric versions are sorted alphabetically.

This is
   a) simple
   b) ensures that non-numeric versions (such as 'develop') in package.py are not chosen ahead of numeric versions, when nothing is specified in packages.yaml

Fixes Issue #1557

* Removed debugging output

* Fix variable shadowing bug

* Ensure develop < numeric version.

* Bug fix.

* Passes all unit tests in versions.py

* flake8 fixes

* flake8 fixes

* Changed type test to be more correct.

See http://stackoverflow.com/questions/8203336/difference-between-int-and-numbers-integral-in-python
2016-10-06 02:35:34 -07:00
scheibelp
508d79c475 Handle packages with unparseable extensions (#1758)
This closes #1757 which provides an example of a url scheme where the
version appears after the extension. Instead of extending the parsing
logic to handle this case, this commit allows the user to specify
their extension type. This helps Spack choose the appropriate
decompressor and mirror archive filename.
2016-10-05 22:45:02 -07:00
Geoffrey Oxberry
6f0b32a71f docs/getting_started: s/Macintosn/Macintosh/; (#1928) 2016-10-05 21:15:54 -07:00
Elizabeth Fischer
015e29efe1 Documentation Improvements for SC16 (#1676)
* Transferred pending changes from efischer/develop

* 1. Rewrite of "Getting Started": everything you need to set up Spack, even on old/ornery systems.  This is not a reference manual section; items covered here are covered more systematically elsewhere in the manual.  Some sections were moved here from elsewhere.

2. Beginning to write three methods of application developer support.  Two methods were moved from elsewhere.

* Edits...

* Moved sections in preparation for additional text to be added from old efischer/docs branch.

* Moved 2 more sections.

* Avoid accid

* Applied proofreading edits from @adamjstewart

* Fixed non-standard section characters.

* Moved section on profiling to the developer's guide.

* Still working on Spack workflows...

* Finished draft of packaging_guide.rst

* Renamed sample projects.

* Updates to docstrings

* Added documentation to resolve #638 (content taken from #846)

* Added section on resolving inconsistent run dependencies.  Addresses #645

* Showed how to build Python extensions only compatible with certain versions of Python.

* Added examples of getting the right behavior from depends_on().  See #1035

* Added section on Intel compilers and their GCC masquerading feature.  Addresses #638, #1687.

* Fixed formatting

* Added fixes to filesystem views.  Added a caveats section to ``spack setup``.

* Updated section on Intel compiler configuration because compiler flags currently do not work (see #1687)

* Defined trusted downloads, and updated text based on them. (See #1696)

* Added workflow to deal with buggy upstream software.  See #1683

* Added proper separation between Spack Docs vs. Reference Manual

* Renamed spack_workflows to workflows.  Resolves a conflict with the .gitignore file.

* Removed repeated section.

* Created new "Vendor Specific Compiler Configuration" section and organized existing Intel section into it.  Added new PGI and NAG sections; but they need to be expanded  / rewritten based on the existing text plus research through Spack issues on GitHub.

* Fixed text on `spack load --dependencies` to conform to reality.  See #1662

* Added patching as option for upstream bugfixes.

* Added section on using licensed compilers.

* Added section on non-downloadable tarballs.

* Wrote sections on NAG and PGI.  Arranged compilers in alphabetical order.

* Fix indent.

* Fixed typos.

* Clarified dependency types.

* Applied edits from Adam J. Stewart.  Spellchecked workflows and getting_started.

* Removed spurious header

* Fixed Sphinx errors

* Fixed erroneous symbol in docstring.

* Fix many typos and formatting problems.

* Spacing changes

* Added section on fixing Git problems.  See #1779

* Fixed signature of install() method.

* Addressed system packages in greater detail.  See #1794 #1795

* Fixed typos

* Fixed quotes

* Duplicate section on Spack profiling removed from configuration.rst.  It had earlier been moved to developer_guide.rst, where it fits better.

* Minor edits

- Tweak supported platform language.
- Various small changes to the new getting started guide.

* Fixed bug with quotes.
2016-10-05 13:00:27 -07:00
Elizabeth Fischer
abc9412f23 New command flag: spack install --dependencies-only (#1603)
* 1. Renamed ignore_deps variable to install_deps (use positive logic).  UI remains the same.

2. install_self kwarg added to do_install().  Enables installation of a package's dependencies without installing the package itself.

3. Added `spack install --dependencies-only <package>` command.

* Flak8 fixes

* Indentation problem
2016-10-05 11:23:06 -07:00
Adam J. Stewart
6ee020cea4 Add package for CONVERGE CFD (#1905) 2016-10-05 10:37:50 -07:00
Elizabeth Fischer
8e75575b88 Added missing function for CMake builds. (#1250)
* Added missing function for CMake builds.

* Update build_environment.py

* std_cmake_args

Changed std_cmake_args to use get_std_cmake_args().

* Update build_environment.py

Delete additional lines subsumed by get_std_cmake_args()

* flake8.
2016-10-05 10:37:20 -07:00
Todd Gamblin
bff1656a1a Read-only locks should close fd before opening for write. (#1906)
- Fixes bad file descriptor error in lock acquire, #1904
- Fix bug introduced in previous PR #1857
- Backported fix from soon-to-be merged fine-grained DB locking branch.
2016-10-04 15:36:37 -07:00
Todd Gamblin
544fa3eddb Fix some documentation build warnings. (#1902)
- Fix issue with package_list.py regeneration confusing Sphinx.
- Add -E option to avoid caching and make Sphinx happy.
2016-10-04 12:37:57 -07:00
Todd Gamblin
e9d4780bbc Rework build environment and cc to use smaller RPATHs. (#1894)
- Fixed up dependency management so that:
  - build deps go in PATH and -I
  - link deps go in -L args
  - only *immediate* link deps are RPATH'd

The latter reduces the number of libraries that need to be added to
DT_NEEDED / LC_RPATH.  This removes redundant RPATHs to transitive
dependencies.
2016-10-04 09:40:28 -07:00
Todd Gamblin
36c79e9df6 Fixes #1893: missing import 2016-10-04 01:49:07 -07:00
Michael Kuhn
8d1ec0df3d Fix read locks on read-only file systems (#1857) 2016-09-30 09:45:08 -07:00
Todd Gamblin
295742e2b5 Fixes #1860. Subversion builds sequentially due to race. 2016-09-28 15:18:52 -04:00
Todd Gamblin
cb229f0842 Fixes #1720: spack reindex fails with invalid ref count. (#1867)
* Fixes #1720: spack reindex fails with invalid ref count.

- Database graph wasn't being built properly; dependencies were set up
  incorrectly in the nodes that ended up in the graph on reindex.

- Reworked _add to increment ref count properly and to always build
  bottom-up to make the logic simpler to understand.

* Add checks to ensure DB is a valid merkle tree.
2016-09-28 15:00:26 -04:00
Adam J. Stewart
6df84a794b Add single letter arg flags for spack graph (#1871) 2016-09-28 12:09:31 -04:00
Todd Gamblin
05d52752ff Make graph_ascii support deptypes.
- fix deptype support
- by default, graph command omits build depedencies
- update docs to use deptype args
2016-09-27 23:49:41 -04:00
Todd Gamblin
f082d26ddd Fixes #1098: spack graph crashes for large graphs.
- Fixed logic for collapsing backward edges

- Last collapse now depends on whether prior step in left collapse
  sequence alrady did the collapse.
2016-09-27 23:28:51 -04:00
Todd Gamblin
0d3d74e5c2 Improvements to the Spack graph command.
- Distinguish between static (package) and dynamic (spec) graphs.

  - static graphs ignore conditions and multiple instances (hashes) and
    plot raw dependencies among packages.

  - dynamic graphs include information from particular specs (instances of
    packages) and can have multiple instances with hashes.

- Allow graphing all packages in the install DB.

  - useful for debugging.
2016-09-27 11:00:50 -04:00
Todd Gamblin
43ca805248 Factor out canonical_deptype function, cleanup spec.py 2016-09-27 11:00:50 -04:00
Todd Gamblin
cd960caf8d Clean up Exceptions in spec.py 2016-09-27 11:00:45 -04:00
Todd Gamblin
cb36aadaf6 Fix doc bugs, widen columns in the package list 2016-09-23 18:07:33 -07:00
Ben Boeckel
c7860322f5 package: fix variable name usage (#1829) 2016-09-22 11:47:25 -07:00
Adam J. Stewart
f1f301ad3b Fix spack checksum output indentation (#1826) 2016-09-22 11:33:29 -07:00
Eric
025b779a30 Fix sbang for perl (#1802)
* Perform shebang fix for all files

* Fix sbang for perl scripts

Otherwise perl would look at the #! line and call sbang again, resulting
in an infinite loop.
2016-09-22 00:43:47 -07:00
Elizabeth Fischer
98f9dd266f Remove SPACK_DIRTY env var (#1818)
* Removed SPACK_DIRTY env var support.

* Finished removing SPACK_DIRTY support.

* Minor changes.
2016-09-22 00:43:33 -07:00
Massimiliano Culpo
1db2552644 fix : stops infinite recursion for python 2.6 (#1823) 2016-09-22 00:31:28 -07:00
Massimiliano Culpo
d848559f70 Reworking of lapack_shared_libs and similar properties (#1682)
* Turned <provider>_libs into an iterable

Modifications :
- added class LibraryList + unit tests
- added convenience functions `find_libraries` and `dedupe`
- modifed non Intel blas/lapack providers
- modified packages using blas_shared_libs and similar functions

* atlas : added pthread variant

* intel packages : added lapack_libs and blas_libs

* find_library_path : removed unused function

* PR review : fixed last issues

* LibraryList : added test on __add__ return type

* LibraryList : added __radd__ fixed unit tests

fix : failing unit tests due to missing `self`

* cp2k and dependecies : fixed blas-lapack related statements in package.py
2016-09-21 12:27:59 -07:00
Elizabeth Fischer
899f3a7e37 Fixed bug propagating --dirty flag to sub-installs. (#1625)
* Fixed bug propagating --dirty flag to sub-installs.

* Fix syntax error

* Allow --dirty flag to be set with SPACK_DIRTY env var.

* Added dirty flag to `spack diy` and `spack setup`, as is currently in `spack install`

* flake8
2016-09-21 03:56:56 -07:00
Matt Belhorn
94b24e8893 Adds all available CrayPE CPU targets to platform by default. (#1745) 2016-09-21 01:48:33 -07:00
Alfredo Adolfo Gimenez
02307cf7ce Set JAVA_HOME on module load and on dependent installs. (#1716)
* Set JAVA_HOME on module load and on dependent installs.

* Include environment setup from dependencies in dependee module (#1714)
2016-09-21 00:52:09 -07:00
Denis Davydov
27801c354b fix MacOs class for Sierra (#1811) 2016-09-20 17:25:23 -07:00
Massimiliano Culpo
eb8a0ef75e fix : failing unit tests due to missing self (#1806) 2016-09-20 07:47:13 -07:00
Eric
c9fe2cd469 Fix format string (in class RemoveFailedError) (#1803) 2016-09-20 02:27:35 -07:00
Massimiliano Culpo
ea446c0f0e lmod : added support for the creation of hierarchical lua module files (#1723)
Includes :
- treatment of a generic hierarchy (i.e. lapack + mpi + compiler)
- possibility to specify which compilers are to be considered Core
- correct treatment of the 'family' directive
- unit tests for most new features
2016-09-20 02:26:25 -07:00
Eric
efadc0e299 Allow multi-user installations (#1804)
When re-using previously downloaded tarballs, spack copies from
`var/spack/stage/PACKAGE-VERSION-HASH/downloaded_file` to
`var/spack/cache/PACKAGE/downloaded_file`. This fails if the source is owned by
a different user (`shutil.copy` tries to retain all meta data including file
ownership). Change to a non-meta-data copy function (`shutil.copyfile`).
2016-09-20 02:18:26 -07:00
Michael Kuhn
7043fff807 Fix location -p and -P (#1776) 2016-09-15 10:48:28 +08:00
Erik Schnetter
6117ef44e4 Make libstdc++-detection work on Darwin (#1778) 2016-09-15 10:42:14 +08:00
Adam J. Stewart
a22f5d8d86 Don't overwrite PYTHONPATH in docs Makefile (#1693) 2016-09-08 09:02:56 -07:00
scheibelp
fd02a140c4 Fix fetching non-expanded resources from mirrors (#1310)
This closes #1308, where fetching a non-expanded resource from a mirror
will cause an error.

This also ensures that when a URL resource is fetched from a mirror,
that it will be named as though it were retrieved from the original
URL. This is particularly useful for non-expanded resources since it
ensures that the resource name is consistent for the installation
(this is less important for expanded resources because the build takes
place inside the expanded resource).
2016-09-07 07:02:24 -07:00
Todd Gamblin
f34dd94166 Add debug command for creating tarball of install DB. 2016-09-06 07:55:54 -07:00
alalazo
fc576a40d6 modules : ('build',) type dependencies are not accounted when autoloading
fixes #1681
2016-09-06 15:07:11 +02:00
Axel Huebl
16c5403ab3 spack boostrap: fix missing include (#1657)
fix a missing include in booststrap.
found on debian 8 with python 2.7
2016-09-03 09:43:00 -07:00
Massimiliano Culpo
497bbb3ae3 added package : pexsi (#1550)
* pexsi : added package
* cp2k : added pexsi (works for gcc+openmpi with and without plumed)
2016-09-03 08:13:47 -07:00
Todd Gamblin
07fa1cd5f8 Attributes for directives need to be set at class construction time. (#1718)
- moved ensure_dicts() call from Package.__init__ to
  Package.__metaclass__.
2016-09-02 15:10:30 -07:00
Todd Gamblin
c8b4f978e1 Clean up stable hashing so that specs don't contain !!python/object/apply
- only output basic lists, dicts, etc.
- spec and database now parse and write specs as ordered data.
2016-09-02 03:03:47 -07:00
Todd Gamblin
674434b0fc Merge branch 'UCL-RITS-hash_stability' into bugfix/hash-collision 2016-09-02 03:03:25 -07:00
Todd Gamblin
9268b7aa7c Fix hash copying in _dup.
- Spec._dup() incorrectly copied cached hashes and normal/concrete values
  even when dependency structure was not preserved.

- Now these are only copied when *all* dependencies are copied.
2016-09-02 01:26:19 -07:00
Todd Gamblin
69d45b49e9 Fix hash handling in directory layout
- Currently, build dependencies are not currently hashed; we are waiting
  to hash these until we have smarter concretization that can reuse more
  installed specs.  The layout needs to account for this when checking
  whethert things are installed.
2016-09-02 01:26:01 -07:00
Todd Gamblin
69b68153a1 Fix spack reindex so that it will work if DB is corrupt (duh).
- Transaction logic had gotten complicated -- DB would not reindex when
  corrupt, rather the error would be reported (ugh).

- DB will now print the error and force a rebuild when errors are
  detected reading the old databse.
2016-09-01 11:29:32 -07:00
Todd Gamblin
bee5c05568 Update tests to reflect new in-memory hashing vs. coarser dag_hash.
- Spack currently not hashing build deps (to allow more reuse of packages
  and less frequent re-installing)

- Fast in-memory hash should still hash *all* deptypes, and installed
  specs will only reflect link and run deps.

- We'll revert this when we can concretize more liberally based on what
  is already installed.
2016-09-01 11:29:32 -07:00
Todd Gamblin
235a045d08 Add option to copy only certain deptypes to Spec.copy()
- can now pass these to Spec.copy() and Spec._dup():
  - deps=True
  - deps=False
  - deps=(list of deptypes)

- Makes it easy to filter out only part of a spec.
2016-09-01 11:29:32 -07:00
Todd Gamblin
409e7a2e64 Faster database loading.
- use a 3-pass algorithm to load the installed package DAG.

- avoid redundant hashing/comparing on load.
2016-09-01 11:29:32 -07:00
Todd Gamblin
a8aad95d41 Specs now cache result of "fast" in-memory hash.
- Hash causes major slowdown for reading/setting up large DBs

- New version caches hash for concrete specs, which includes all specs in
  the install DB
2016-09-01 11:29:32 -07:00
Todd Gamblin
c46a15b574 Fix dependency bug in database. Fixes #1265, #1320. 2016-09-01 11:25:01 -07:00
Todd Gamblin
4a87d4ab8e Make gh-pages redirect to readthedocs.io 2016-08-31 19:40:22 -07:00
Erik Schnetter
efc570afab Expand/clarify description of dependency types 2016-08-31 14:42:32 -04:00
Todd Gamblin
0ff048521b Fix modules.rst doc build warning. 2016-08-30 22:15:13 -07:00
Todd Gamblin
3a9388b43a Move sphinx-apidoc into conf.py, too. 2016-08-30 21:02:34 -07:00
Todd Gamblin
32834dd67d fix sys.path in conf.py 2016-08-30 20:50:14 -07:00
Todd Gamblin
05545a2528 test. 2016-08-30 20:49:56 -07:00
Todd Gamblin
41675e5ed6 sorted command index, better colification. 2016-08-30 20:49:56 -07:00
Todd Gamblin
176a84a828 Don't test sphinx-generated conf.py for flake8 issues. 2016-08-30 20:30:43 -07:00
Todd Gamblin
d9c5191563 Fix issue with path to Spack. 2016-08-30 19:57:33 -07:00
Todd Gamblin
fc748eb3d0 Exclude spack.__all__ from documentation.
Everything in the __all__ list in the spack module is from some other
module, so only do their documentation in their original location.  This
also avoids issues like the fact that some directive names shadow spack
core module names.
2016-08-30 16:36:50 -05:00
Adam J. Stewart
1be6267149 Undo changes to trailing triple quotes in docstring 2016-08-30 16:28:55 -05:00
Adam J. Stewart
9b455e9254 Resolve new documentation issues that have cropped up 2016-08-30 15:58:22 -05:00
Adam J. Stewart
ae2a803496 Make subcommands importable, '-' -> '_', fixes #1642 2016-08-30 15:36:33 -05:00
Adam J. Stewart
8d1a753a1b Fix spack --profile, fixes #1639 2016-08-30 15:35:58 -05:00
Adam J. Stewart
6795f4ca12 Use console instead of shell syntax lexer, fixes #1634 2016-08-30 15:35:58 -05:00
Adam J. Stewart
f78134efa5 Fixes #1620, Missing references 2016-08-30 15:32:54 -05:00
Adam J. Stewart
867a92f083 Fixes #1618, Problems with spack package-list 2016-08-30 15:32:54 -05:00
Adam J. Stewart
69e50595bf Fix docstrings for core Spack libraries, fixes #1612 2016-08-30 15:32:54 -05:00
Adam J. Stewart
02239c094e Fix unexpected indentation in .rst files, #1612 2016-08-30 15:29:51 -05:00
Adam J. Stewart
0614bd206f Fix #1608, Include file not found 2016-08-30 15:28:09 -05:00
Adam J. Stewart
2326355497 Fix #1604 and fix #1605, title underline problems 2016-08-30 15:28:08 -05:00
Adam J. Stewart
fda7fcd73d Fix #1594, duplicate explicit target name 2016-08-30 15:21:15 -05:00
Todd Gamblin
dae00fec29 Move all documentation generation into conf.py
- extra steps in Makefile are ignored by readthedocs
2016-08-30 00:47:04 -07:00
Massimiliano Culpo
3ecf4ef8c9 resources : added docs fixes #1551 (#1627) 2016-08-29 22:40:50 -07:00
Todd Gamblin
8ab47537c7 Fix exception constructors. (#1663) 2016-08-29 22:34:47 -07:00
Todd Gamblin
f7ee8d001a Merge pull request #1564 from adamjstewart/features/create
Suggest setuptools when creating Python package
2016-08-29 09:11:45 -07:00
Tyler Esser
83897af710 Documentation: Fix accidental quotation. version() takes expand as a keyword. (#1640) 2016-08-28 20:16:08 -07:00
Todd Gamblin
2053db4d17 Make clang detection work on AWS Ubuntu14. 2016-08-27 17:04:46 -07:00
Todd Gamblin
7ebb17e459 Remove stale operating_system.py test per discussion in #1631
- Test has not been registered in `__init__.py` for a while and needs a
  refresh.
2016-08-25 21:20:34 -07:00
Todd Gamblin
e9bc3a9e32 Merge pull request #663 from citibeth/efischer/160327-DocumentationImprovements
[RtM] Documentation Improvements
2016-08-25 02:04:30 -07:00
Todd Gamblin
062ff13da6 Make spack info print deps in sorted order 2016-08-24 14:10:25 -07:00
Elizabeth Fischer
85be3aefa7 Put url_for_version() after versions in example. 2016-08-24 15:26:19 -04:00
Tom Scogland
2b6833cb80 appeasing flake8, also cleaning up header
The header append lines were too long, so I just converted it into a
multi-line string template so it's all one piece now instead of a bunch
of appends.
2016-08-24 11:56:33 -07:00
Tom Scogland
73620fe868 avoid specifying --delim unless required
This increases portability of generated module files to versions of
environment_modules hearkening back to just after the turn of the
millennium.
2016-08-24 11:00:16 -07:00
citibeth
8a481e7e13 Added CMake-based Development case study to docuemntation. 2016-08-23 16:39:01 -04:00
Elizabeth F
86d39255ec Explained url_for_version() 2016-08-23 16:39:01 -04:00
citibeth
2efd7a5e0b Added note on use of 'less -R' for colorized output. 2016-08-23 16:36:02 -04:00
Adam J. Stewart
df623dad15 Suggest setuptools when creating Python package 2016-08-19 14:31:42 -05:00
Nicolas Richart
37faa41b6f stripping the links to remove spaces and '\n' 2016-08-15 21:13:25 +02:00
Todd Gamblin
1e673bfa42 Don't assume spack is in the path when buildign docs. 2016-08-15 10:49:09 -07:00
Adam J. Stewart
c815aed207 Update documentation for spack md5 2016-08-12 10:51:56 -05:00
becker33
ec7f58bf96 Merge pull request #1493 from xjrc/bugfix/nonbool-variants
Bug Fix : Allow Packages to Have Variants w/ Non-Bool, Non-String Defaults
2016-08-11 11:33:54 -07:00
Matt Belhorn
568305ddff Corrects Cray compiler name and adds missing version flag. 2016-08-11 13:56:21 -04:00
Mayeul d'Avezac
7769367747 dag_hash stabilised by depending on sorted dict
Spec.to_node_dict uses OrderedDict

This is to try and ensure that the dag_hash is stable across python
version and processes.
2016-08-11 16:03:50 +01:00
Todd Gamblin
867121ca68 Move jsonschema files out of config.py to their own package. 2016-08-10 16:33:38 -07:00