Commit graph

4492 commits

Author SHA1 Message Date
Todd Gamblin
3b3314802e database: don't create a date object in query() unless it's necessary
`query()` calls `datetime.datetime.fromtimestamp` regardless of whether a
date query is being done. Guard this with an if statement to avoid the
unnecessary work.
2021-02-11 15:30:57 +01:00
Todd Gamblin
c81ca37dfc version: precompile regexes in Version constructor 2021-02-11 15:30:57 +01:00
Todd Gamblin
e3e913ef8f spack_json: _strify should be a no-op in Python 3
_strify traverses *all* of the read-in data and converts strings, but
only for Python 2.  Ignore it in Python 3.
2021-02-11 15:30:57 +01:00
Todd Gamblin
52508e6ee7 database: do not redundantly mark specs concrete
Speed up reads by not traversing each spec recursively when marking them
concrete.
2021-02-11 15:30:57 +01:00
Todd Gamblin
6fe931ccb1 spec performance: don't parse in from_node_dict
Constructing a spec from a name instead of setting name directly forces
from_node_dict to call Spec.parse(), which is slow. Avoid this by using a
zero-arg constructor and setting name directly.
2021-02-11 15:30:57 +01:00
Massimiliano Culpo
4558dc06e2 Added a context manager to swap architectures
This solves a few FIXMEs in conftest.py, where
we were manipulating globals and seeing side
effects prior to registering fixtures.

This commit solves the FIXMEs, but introduces
a performance regression on tests that may need
to be investigated
2021-02-10 16:50:09 -08:00
Massimiliano Culpo
553d37a6d6 Move context manager to swap the current configuration into spack.config
The context manager can be used to swap the current
configuration temporarily, for any use case that may need it.
2021-02-10 16:50:09 -08:00
Massimiliano Culpo
cb2c233a97 Move context manager to swap the current store into spack.store
The context manager can be used to swap the current
store temporarily, for any use case that may need it.
2021-02-10 16:50:09 -08:00
Massimiliano Culpo
1a8963b0f4 repo: generalize "swap" context manager to also accept paths
The method is now called "use_repositories" and
makes it clear in the docstring that it accepts
as arguments either Repo objects or paths.

Since there was some duplication between this
contextmanager and "use_repo" in the testing framework,
remove the latter and use spack.repo.use_repositories
across the entire code base.

Make a few adjustment to MockPackageMultiRepo, since it was
stating in the docstring that it was supposed to mock
spack.repo.Repo and was instead mocking spack.repo.RepoPath.
2021-02-10 16:50:09 -08:00
Scott McMillan
0007ed72c3
Fix: Add a space between -isystem and the directory (#21599)
Some compilers, such as the NV compilers, do not recognize -isystem
dir when specified without a space.

Works: -isystem ../include
Does not work: -isystem../include

This PR updates the compiler wrapper to include the space with -isystem.
2021-02-10 15:56:42 -08:00
Greg Becker
2b6f896ca7
Bugfix: environments/views on separate mounts (#20720)
Environment views fail when the tmpdir used for view generation is
on a separate mount from the install_tree because the files cannot
by symlinked between the two. The fix is to use an alternative
tmpdir located alongside the view.
2021-02-10 13:39:11 -08:00
Adam J. Stewart
7ccb9992a6
Procedure to deprecate old versions of software (#20767)
* Procedure to deprecate old versions of software

* Add documentation

* Fix bug in logic

* Update tab completion

* Deprecate legacy packages

* Deprecate old mxnet as well

* More explicit docs
2021-02-09 13:51:18 -05:00
BenWeber42
e5ab686d77
Changed stages permissions to 750 (#21425) 2021-02-09 10:12:43 -08:00
BenWeber42
e4d74825f3
Fixed uninstall rm parent folder race condition (#21424) 2021-02-09 10:05:06 -08:00
Mark Meredith
784b421eda
Fix typo (#21567) 2021-02-09 15:34:42 +00:00
Adam J. Stewart
48954a5a3e
Restore verbosity of patching (#21233) 2021-02-08 10:29:19 -08:00
albestro
02b64c8903
let intel consider cray-mpich as mpich (#21512) 2021-02-05 17:55:39 +00:00
Adam J. Stewart
ba70f90ee0
Fix usage of PythonPackage.test outside of PythonPackage (#20555) 2021-02-04 11:00:37 -06:00
Adam J. Stewart
df5992293a
Python: add maintainer(s) (#21125)
* Python: add maintainer(s)

* Fix unit tests
2021-02-04 10:00:21 -06:00
Massimiliano Culpo
694d633a2c
spack external find: allow to search by tags (#21407)
This commit adds an option to the `external find`
command that allows it to search by tags. In this
way group of executables with common purposes can
be grouped under a single name and a simple command
can be used to detect all of them.

As an example introduce the 'build-tools' tag to
search for common development tools on a system
2021-02-04 13:17:32 +01:00
Massimiliano Culpo
ba42c36f00
concretizer: simplify "fact" method (#21148)
The "fact" method before was dealing with multiple facts
registered per call, which was used when we were emitting
grounded rules from knowledge of the problem instance.

Now that the encoding is changed we can simplify the method
to deal only with a single fact per call.
2021-02-03 10:12:03 -08:00
Greg Becker
7dcf3f7aed
add intel oneapi to compiler/pkg translations (#21448) 2021-02-02 19:11:51 +00:00
Adam J. Stewart
b597cbe1c0
PythonPackage: fewer phases (#20738) 2021-02-01 12:48:45 -06:00
Adam J. Stewart
1d636bed6a
R: add maintainer(s) (#21119)
* R: add maintainer(s)

* Add maintainer to base class
2021-02-01 11:33:33 -06:00
Adam J. Stewart
40a40e0265
Python 3.10 support: collections.abc (#20441) 2021-02-01 11:30:25 -06:00
Yang Zongze
f8124369f9
Print groups properly for spack find -d (#20028) 2021-01-29 11:05:36 -08:00
Adam J. Stewart
eeddaaea52
Make git fetching even quieter (for git v1.7.2+)(#21013) 2021-01-28 20:41:05 +01:00
Sergey Kosukhin
4d7a9df810
Add a context wrapper for mtime preservation (#21258)
Sometimes we need to patch a file that is a dependency for some other
automatically generated file that comes in a release tarball. As a
result, make tries to regenerate the dependent file using additional
tools (e.g. help2man), which would not be needed otherwise.

In some cases, it's preferable to avoid that (e.g. see #21255). A way
to do that is to save the modification timestamps before patching and
restoring them afterwards. This PR introduces a context wrapper that
does that.
2021-01-27 11:41:07 -08:00
Mark C. Miller
f766585882
Doc default behavior of install tests (#21309)
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
2021-01-27 09:08:27 -08:00
Adam J. Stewart
aa8e026242
spack setup: remove the command for v0.17.0 (#20277)
spack setup was deprecated in 0.16 and will be removed in 0.17

Follow-up to #18240
2021-01-27 09:24:09 +01:00
Tom Scogland
12eb4a146f
use module and package flags to get more correct mypy behavior (#21225)
The first of my two upstream patches to mypy landed in the 0.800 tag that was released this morning, which lets us use module and package parameters with a .mypy.ini file that has a files key. This uses those parameters to check all of spack in style, but leaves the packages out for now since they are still very, very broken. If no package has been modified, the packages are not checked, but if one has they are. Includes some fixes for the log tests since they were not type checking.

Should also fix all failures related to "duplicate module named package" errors.

Hopefully the next drop of mypy will include my other patch so we can just specify the modules and packages in the config file to begin with, but for now we'll have to live with a bare mypy doing a check of the libs but not the packages.

* use module and package flags to check packages properly
* stop checking package files, use package flag for libs

The packages are not type checkable yet, need to finish out another PR
before they can be.  The previous commit also didn't check the libraries
properly, this one does.
2021-01-22 16:24:15 -08:00
AMD Toolchain Support
2c1bb64594
Added @property stdcxx_libs to return -lstdc++ for AOCC compiler (#21145) 2021-01-22 09:58:04 -06:00
Tamara Dahlgren
6b13909cc1
docs: Update the CudaPackage (build system) description (#20742)
Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja>
2021-01-21 12:43:21 -08:00
Tamara Dahlgren
25bab31bc8
Added ROCmPackage (build system) documentation (#20743) 2021-01-21 10:46:39 -08:00
eugeneswalker
7f86cc87b0
store sbang_install_path in buildinfo, use for subsequent relocation (#20768) 2021-01-20 12:43:07 -08:00
Nathan Hanford
ebc871abbf
[WIP] relocate.py: parallelize test replacement logic (#19690)
* sbang pushed back to callers;
star moved to util.lang

* updated unit test

* sbang test moved; local tests pass

Co-authored-by: Nathan Hanford <hanford1@llnl.gov>
2021-01-20 09:17:47 -08:00
Robert Cohn
09ee3f8d8e
add required libs for sycl programs (#20728) 2021-01-18 23:15:24 -06:00
Danny Taller
74b2c85b41
improve documentation for Rocm (hip amd builds) (#20812)
* improve documentation
2021-01-14 08:54:58 -08:00
Massimiliano Culpo
0bd76ac9e6 concretizer: require at least a dependency type to say the dependency holds
fixes #20784

Similarly to the previous bug, here we were deducing
conditions to be imposed on nodes that were not part
of the DAG.
2021-01-12 22:23:39 -08:00
Massimiliano Culpo
ed8fe68cf2 concretizer: dependency conditions cannot hold if package is external
fixes #20736

Before this one line fix we were erroneously deducing
that dependency conditions hold even if a package
was external.

This may result in answer sets that contain imposed
conditions on a node without the node being present
in the DAG, hence #20736.
2021-01-12 22:23:39 -08:00
Robert Underwood
823ab32c10
restore ability of dev-build to skip patches (#20351)
At some point in the past, the skip_patch argument was removed
from the call to package.do_install() this broke the --skip-patch
flag on the dev-build command.
2021-01-12 14:32:04 -06:00
Adam J. Stewart
881bdd5b9e
Package Repositories docs: num packages has grown (#20735) 2021-01-11 11:00:14 +01:00
Tamara Dahlgren
31f26daf10
Update the docs footer copyright (#20741) 2021-01-11 08:00:08 +01:00
eugeneswalker
6157854de6
fix gpg user rundir check (#20705) 2021-01-06 12:29:00 -08:00
Massimiliano Culpo
0111a18b23 concretizer: make rules on virtual packages more linear
fixes #20679

In this refactor we have a single cardinality rule on the
provider, which triggers a rule transforming a dependency
on a virtual package into a dependency on the provider of
the virtual.
2021-01-05 16:59:15 -08:00
Vanessasaurus
67ce1939a3
spack python: allow use of IPython (#20329)
This adds a -i option to "spack python" which allows use of the
IPython interpreter; it can be used with "spack python -i ipython".
This assumes it is available in the Python instance used to run
Spack (i.e. that you can "import IPython").
2021-01-05 16:54:47 -08:00
Greg Becker
61c1b71d38
bugfix for target adjustments on target ranges (#20537) 2021-01-05 12:27:13 -08:00
Todd Gamblin
5ac05db917
concretizer: use consistent naming for compiler predicates (#20677)
Every other predicate in the concretizer uses a `_set` suffix to
implement user- or package-supplied settings, but compiler settings use a
`_hard` suffix for this. There's no difference in how they're used, so
make the names the same.

- [x] change `node_compiler_hard` to `node_compiler_set`
- [x] change `node_compiler_version_hard` to `node_compiler_version_set`
2021-01-05 08:41:01 +01:00
Todd Gamblin
9c941bb706 concretizer: simplify handling of virtual version constraints
Previously, the concretizer handled version constraints by comparing all
pairs of constraints and ensuring they satisfied each other. This led to
INCONSISTENT ressults from clingo, due to ambiguous semantics like:

    version_constraint_satisfies("mpi", ":1", ":3")
    version_constraint_satisfies("mpi", ":3", ":1")

To get around this, we introduce possible (fake) versions for virtuals,
based on their constraints. Essentially, we add any Versions,
VersionRange endpoints, and all such Versions and endpoints from
VersionLists to the constraint. Virtuals will have one of these synthetic
versions "picked" by the solver. This also allows us to remove a special
case from handling of `version_satisfies/3` -- virtuals now work just
like regular packages.
2021-01-04 14:51:10 -08:00
Todd Gamblin
a1ed71f7e4 concretizer: remove rule generation code from concretizer
Our program only generates facts now, so remove all unused code related
to generating cardinality constraints and rules.
2021-01-04 14:51:10 -08:00