remove activate/deactivate support in favor of environments (#29317)
Environments and environment views have taken over the role of `spack activate/deactivate`, and we should deprecate these commands for several reasons: - Global activation is a really poor idea: - Install prefixes should be immutable; since they can have multiple, unrelated dependents; see below - Added complexity elsewhere: verification of installations, tarballs for build caches, creation of environment views of packages with unrelated extensions "globally activated"... by removing the feature, it gets easier for people to contribute, and we'd end up with fewer bugs due to edge cases. - Environment accomplish the same thing for non-global "activation" i.e. `spack view`, but better. Also we write in the docs: ``` However, Spack global activations have two potential drawbacks: #. Activated packages that involve compiled C extensions may still need their dependencies to be loaded manually. For example, ``spack load openblas`` might be required to make ``py-numpy`` work. #. Global activations "break" a core feature of Spack, which is that multiple versions of a package can co-exist side-by-side. For example, suppose you wish to run a Python package in two different environments but the same basic Python --- one with ``py-numpy@1.7`` and one with ``py-numpy@1.8``. Spack extensions will not support this potential debugging use case. ``` Now that environments are established and views can take over the role of activation non-destructively, we can remove global activation/deactivation.
This commit is contained in:
Harmen Stoppels
GitHub
parent
f11778bb02
commit
0f54a63dfd
24 changed files with 34 additions and 1708 deletions
|
|
@ -1114,21 +1114,21 @@ set of arbitrary versions, such as ``@1.0,1.5,1.7`` (``1.0``, ``1.5``,
|
|||
or ``1.7``). When you supply such a specifier to ``spack install``,
|
||||
it constrains the set of versions that Spack will install.
|
||||
|
||||
For packages with a ``git`` attribute, ``git`` references
|
||||
may be specified instead of a numerical version i.e. branches, tags
|
||||
and commits. Spack will stage and build based off the ``git``
|
||||
For packages with a ``git`` attribute, ``git`` references
|
||||
may be specified instead of a numerical version i.e. branches, tags
|
||||
and commits. Spack will stage and build based off the ``git``
|
||||
reference provided. Acceptable syntaxes for this are:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
|
||||
# branches and tags
|
||||
foo@git.develop # use the develop branch
|
||||
foo@git.0.19 # use the 0.19 tag
|
||||
|
||||
|
||||
# commit hashes
|
||||
foo@abcdef1234abcdef1234abcdef1234abcdef1234 # 40 character hashes are automatically treated as git commits
|
||||
foo@git.abcdef1234abcdef1234abcdef1234abcdef1234
|
||||
|
||||
|
||||
Spack versions from git reference either have an associated version supplied by the user,
|
||||
or infer a relationship to known versions from the structure of the git repository. If an
|
||||
associated version is supplied by the user, Spack treats the git version as equivalent to that
|
||||
|
|
@ -1745,13 +1745,13 @@ directly when you run ``python``:
|
|||
Using Extensions
|
||||
^^^^^^^^^^^^^^^^
|
||||
|
||||
There are four ways to get ``numpy`` working in Python. The first is
|
||||
There are multiple ways to get ``numpy`` working in Python. The first is
|
||||
to use :ref:`shell-support`. You can simply ``load`` the extension,
|
||||
and it will be added to the ``PYTHONPATH`` in your current shell:
|
||||
and it will be added to the ``PYTHONPATH`` in your current shell, and
|
||||
Python itself will be available in the ``PATH``:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
$ spack load python
|
||||
$ spack load py-numpy
|
||||
|
||||
Now ``import numpy`` will succeed for as long as you keep your current
|
||||
|
|
@ -1777,128 +1777,13 @@ load, you can use the ``spack module tcl|lmod loads`` command to get
|
|||
the name of the module from the Spack spec.
|
||||
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
Activating Extensions in a View
|
||||
Extensions in an Environment
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Another way to use extensions is to create a view, which merges the
|
||||
python installation along with the extensions into a single prefix.
|
||||
See :ref:`configuring_environment_views` for a more in-depth description
|
||||
of views.
|
||||
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
Activating Extensions Globally
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
As an alternative to creating a merged prefix with Python and its extensions,
|
||||
and prior to support for views, Spack has provided a means to install the
|
||||
extension into the Spack installation prefix for the extendee. This has
|
||||
typically been useful since extendable packages typically search their own
|
||||
installation path for addons by default.
|
||||
|
||||
Global activations are performed with the ``spack activate`` command:
|
||||
|
||||
.. _cmd-spack-activate:
|
||||
|
||||
^^^^^^^^^^^^^^^^^^
|
||||
``spack activate``
|
||||
^^^^^^^^^^^^^^^^^^
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
$ spack activate py-numpy
|
||||
==> Activated extension py-setuptools@11.3.1%gcc@4.4.7 arch=linux-debian7-x86_64-3c74eb69 for python@2.7.8%gcc@4.4.7.
|
||||
==> Activated extension py-nose@1.3.4%gcc@4.4.7 arch=linux-debian7-x86_64-5f70f816 for python@2.7.8%gcc@4.4.7.
|
||||
==> Activated extension py-numpy@1.9.1%gcc@4.4.7 arch=linux-debian7-x86_64-66733244 for python@2.7.8%gcc@4.4.7.
|
||||
|
||||
Several things have happened here. The user requested that
|
||||
``py-numpy`` be activated in the ``python`` installation it was built
|
||||
with. Spack knows that ``py-numpy`` depends on ``py-nose`` and
|
||||
``py-setuptools``, so it activated those packages first. Finally,
|
||||
once all dependencies were activated in the ``python`` installation,
|
||||
``py-numpy`` was activated as well.
|
||||
|
||||
If we run ``spack extensions`` again, we now see the three new
|
||||
packages listed as activated:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
$ spack extensions python
|
||||
==> python@2.7.8%gcc@4.4.7 arch=linux-debian7-x86_64-703c7a96
|
||||
==> 36 extensions:
|
||||
geos py-ipython py-pexpect py-pyside py-sip
|
||||
py-basemap py-libxml2 py-pil py-pytz py-six
|
||||
py-biopython py-mako py-pmw py-rpy2 py-sympy
|
||||
py-cython py-matplotlib py-pychecker py-scientificpython py-virtualenv
|
||||
py-dateutil py-mpi4py py-pygments py-scikit-learn
|
||||
py-epydoc py-mx py-pylint py-scipy
|
||||
py-gnuplot py-nose py-pyparsing py-setuptools
|
||||
py-h5py py-numpy py-pyqt py-shiboken
|
||||
|
||||
==> 12 installed:
|
||||
-- linux-debian7-x86_64 / gcc@4.4.7 --------------------------------
|
||||
py-dateutil@2.4.0 py-nose@1.3.4 py-pyside@1.2.2
|
||||
py-dateutil@2.4.0 py-numpy@1.9.1 py-pytz@2014.10
|
||||
py-ipython@2.3.1 py-pygments@2.0.1 py-setuptools@11.3.1
|
||||
py-matplotlib@1.4.2 py-pyparsing@2.0.3 py-six@1.9.0
|
||||
|
||||
==> 3 currently activated:
|
||||
-- linux-debian7-x86_64 / gcc@4.4.7 --------------------------------
|
||||
py-nose@1.3.4 py-numpy@1.9.1 py-setuptools@11.3.1
|
||||
|
||||
Now, when a user runs python, ``numpy`` will be available for import
|
||||
*without* the user having to explicitly load it. ``python@2.7.8`` now
|
||||
acts like a system Python installation with ``numpy`` installed inside
|
||||
of it.
|
||||
|
||||
Spack accomplishes this by symbolically linking the *entire* prefix of
|
||||
the ``py-numpy`` package into the prefix of the ``python`` package. To the
|
||||
python interpreter, it looks like ``numpy`` is installed in the
|
||||
``site-packages`` directory.
|
||||
|
||||
The only limitation of global activation is that you can only have a *single*
|
||||
version of an extension activated at a time. This is because multiple
|
||||
versions of the same extension would conflict if symbolically linked
|
||||
into the same prefix. Users who want a different version of a package
|
||||
can still get it by using environment modules or views, but they will have to
|
||||
explicitly load their preferred version.
|
||||
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
``spack activate --force``
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
If, for some reason, you want to activate a package *without* its
|
||||
dependencies, you can use ``spack activate --force``:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
$ spack activate --force py-numpy
|
||||
==> Activated extension py-numpy@1.9.1%gcc@4.4.7 arch=linux-debian7-x86_64-66733244 for python@2.7.8%gcc@4.4.7.
|
||||
|
||||
.. _cmd-spack-deactivate:
|
||||
|
||||
^^^^^^^^^^^^^^^^^^^^
|
||||
``spack deactivate``
|
||||
^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
We've seen how activating an extension can be used to set up a default
|
||||
version of a Python module. Obviously, you may want to change that at
|
||||
some point. ``spack deactivate`` is the command for this. There are
|
||||
several variants:
|
||||
|
||||
* ``spack deactivate <extension>`` will deactivate a single
|
||||
extension. If another activated extension depends on this one,
|
||||
Spack will warn you and exit with an error.
|
||||
* ``spack deactivate --force <extension>`` deactivates an extension
|
||||
regardless of packages that depend on it.
|
||||
* ``spack deactivate --all <extension>`` deactivates an extension and
|
||||
all of its dependencies. Use ``--force`` to disregard dependents.
|
||||
* ``spack deactivate --all <extendee>`` deactivates *all* activated
|
||||
extensions of a package. For example, to deactivate *all* python
|
||||
extensions, use:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
$ spack deactivate --all python
|
||||
See :ref:`environments` for a more in-depth description
|
||||
of environment views.
|
||||
|
||||
-----------------------
|
||||
Filesystem requirements
|
||||
|
|
|
|||
|
|
@ -253,27 +253,6 @@ to update them.
|
|||
multiple runs of ``spack style`` just to re-compute line numbers and
|
||||
makes it much easier to fix errors directly off of the CI output.
|
||||
|
||||
.. warning::
|
||||
|
||||
Flake8 and ``pep8-naming`` require a number of dependencies in order
|
||||
to run. If you installed ``py-flake8`` and ``py-pep8-naming``, the
|
||||
easiest way to ensure the right packages are on your ``PYTHONPATH`` is
|
||||
to run::
|
||||
|
||||
spack activate py-flake8
|
||||
spack activate pep8-naming
|
||||
|
||||
so that all of the dependencies are symlinked to a central
|
||||
location. If you see an error message like:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
Traceback (most recent call last):
|
||||
File: "/usr/bin/flake8", line 5, in <module>
|
||||
from pkg_resources import load_entry_point
|
||||
ImportError: No module named pkg_resources
|
||||
|
||||
that means Flake8 couldn't find setuptools in your ``PYTHONPATH``.
|
||||
|
||||
^^^^^^^^^^^^^^^^^^^
|
||||
Documentation Tests
|
||||
|
|
@ -309,13 +288,9 @@ All of these can be installed with Spack, e.g.
|
|||
|
||||
.. code-block:: console
|
||||
|
||||
$ spack activate py-sphinx
|
||||
$ spack activate py-sphinx-rtd-theme
|
||||
$ spack activate py-sphinxcontrib-programoutput
|
||||
$ spack load py-sphinx py-sphinx-rtd-theme py-sphinxcontrib-programoutput
|
||||
|
||||
so that all of the dependencies are symlinked into that Python's
|
||||
tree. Alternatively, you could arrange for their library
|
||||
directories to be added to PYTHONPATH. If you see an error message
|
||||
so that all of the dependencies are added to PYTHONPATH. If you see an error message
|
||||
like:
|
||||
|
||||
.. code-block:: console
|
||||
|
|
|
|||
|
|
@ -233,8 +233,8 @@ packages will be listed as roots of the Environment.
|
|||
|
||||
All of the Spack commands that act on the list of installed specs are
|
||||
Environment-sensitive in this way, including ``install``,
|
||||
``uninstall``, ``activate``, ``deactivate``, ``find``, ``extensions``,
|
||||
and more. In the :ref:`environment-configuration` section we will discuss
|
||||
``uninstall``, ``find``, ``extensions``, and more. In the
|
||||
:ref:`environment-configuration` section we will discuss
|
||||
Environment-sensitive commands further.
|
||||
|
||||
^^^^^^^^^^^^^^^^^^^^^
|
||||
|
|
|
|||
|
|
@ -2722,67 +2722,6 @@ extensions; as a consequence python extension packages (those inheriting from
|
|||
``PythonPackage``) likewise override ``add_files_to_view`` in order to rewrite
|
||||
shebang lines which point to the Python interpreter.
|
||||
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
Activation & deactivation
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Adding an extension to a view is referred to as an activation. If the view is
|
||||
maintained in the Spack installation prefix of the extendee this is called a
|
||||
global activation. Activations may involve updating some centralized state
|
||||
that is maintained by the extendee package, so there can be additional work
|
||||
for adding extensions compared with non-extension packages.
|
||||
|
||||
Spack's ``Package`` class has default ``activate`` and ``deactivate``
|
||||
implementations that handle symbolically linking extensions' prefixes
|
||||
into a specified view. Extendable packages can override these methods
|
||||
to add custom activate/deactivate logic of their own. For example,
|
||||
the ``activate`` and ``deactivate`` methods in the Python class handle
|
||||
symbolic linking of extensions, but they also handle details surrounding
|
||||
Python's ``.pth`` files, and other aspects of Python packaging.
|
||||
|
||||
Spack's extensions mechanism is designed to be extensible, so that
|
||||
other packages (like Ruby, R, Perl, etc.) can provide their own
|
||||
custom extension management logic, as they may not handle modules the
|
||||
same way that Python does.
|
||||
|
||||
Let's look at Python's activate function:
|
||||
|
||||
.. literalinclude:: _spack_root/var/spack/repos/builtin/packages/python/package.py
|
||||
:pyobject: Python.activate
|
||||
:linenos:
|
||||
|
||||
This function is called on the *extendee* (Python). It first calls
|
||||
``activate`` in the superclass, which handles symlinking the
|
||||
extension package's prefix into the specified view. It then does
|
||||
some special handling of the ``easy-install.pth`` file, part of
|
||||
Python's setuptools.
|
||||
|
||||
Deactivate behaves similarly to activate, but it unlinks files:
|
||||
|
||||
.. literalinclude:: _spack_root/var/spack/repos/builtin/packages/python/package.py
|
||||
:pyobject: Python.deactivate
|
||||
:linenos:
|
||||
|
||||
Both of these methods call some custom functions in the Python
|
||||
package. See the source for Spack's Python package for details.
|
||||
|
||||
^^^^^^^^^^^^^^^^^^^^
|
||||
Activation arguments
|
||||
^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
You may have noticed that the ``activate`` function defined above
|
||||
takes keyword arguments. These are the keyword arguments from
|
||||
``extends()``, and they are passed to both activate and deactivate.
|
||||
|
||||
This capability allows an extension to customize its own activation by
|
||||
passing arguments to the extendee. Extendees can likewise implement
|
||||
custom ``activate()`` and ``deactivate()`` functions to suit their
|
||||
needs.
|
||||
|
||||
The only keyword argument supported by default is the ``ignore``
|
||||
argument, which can take a regex, list of regexes, or a predicate to
|
||||
determine which files *not* to symlink during activation.
|
||||
|
||||
.. _virtual-dependencies:
|
||||
|
||||
--------------------
|
||||
|
|
|
|||
|
|
@ -1,53 +0,0 @@
|
|||
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
|
||||
# Spack Project Developers. See the top-level COPYRIGHT file for details.
|
||||
#
|
||||
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
|
||||
|
||||
import llnl.util.tty as tty
|
||||
|
||||
import spack.cmd
|
||||
import spack.cmd.common.arguments as arguments
|
||||
import spack.environment as ev
|
||||
from spack.filesystem_view import YamlFilesystemView
|
||||
|
||||
description = "activate a package extension"
|
||||
section = "extensions"
|
||||
level = "long"
|
||||
|
||||
|
||||
def setup_parser(subparser):
|
||||
subparser.add_argument(
|
||||
"-f", "--force", action="store_true", help="activate without first activating dependencies"
|
||||
)
|
||||
subparser.add_argument("-v", "--view", metavar="VIEW", type=str, help="the view to operate on")
|
||||
arguments.add_common_arguments(subparser, ["installed_spec"])
|
||||
|
||||
|
||||
def activate(parser, args):
|
||||
|
||||
tty.warn(
|
||||
"spack activate is deprecated in favor of " "environments and will be removed in v0.19.0"
|
||||
)
|
||||
|
||||
specs = spack.cmd.parse_specs(args.spec)
|
||||
if len(specs) != 1:
|
||||
tty.die("activate requires one spec. %d given." % len(specs))
|
||||
|
||||
spec = spack.cmd.disambiguate_spec(specs[0], ev.active_environment())
|
||||
if not spec.package.is_extension:
|
||||
tty.die("%s is not an extension." % spec.name)
|
||||
|
||||
if args.view:
|
||||
target = args.view
|
||||
else:
|
||||
target = spec.package.extendee_spec.prefix
|
||||
|
||||
view = YamlFilesystemView(target, spack.store.layout)
|
||||
|
||||
if spec.package.is_activated(view):
|
||||
tty.msg("Package %s is already activated." % specs[0].short_spec)
|
||||
return
|
||||
|
||||
# TODO: refactor FilesystemView.add_extension and use that here (so there
|
||||
# aren't two ways of activating extensions)
|
||||
spec.package.do_activate(view, with_dependencies=not args.force)
|
||||
|
|
@ -1,96 +0,0 @@
|
|||
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
|
||||
# Spack Project Developers. See the top-level COPYRIGHT file for details.
|
||||
#
|
||||
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
|
||||
|
||||
import llnl.util.tty as tty
|
||||
|
||||
import spack.cmd
|
||||
import spack.cmd.common.arguments as arguments
|
||||
import spack.environment as ev
|
||||
import spack.graph
|
||||
import spack.store
|
||||
from spack.filesystem_view import YamlFilesystemView
|
||||
|
||||
description = "deactivate a package extension"
|
||||
section = "extensions"
|
||||
level = "long"
|
||||
|
||||
|
||||
def setup_parser(subparser):
|
||||
subparser.add_argument(
|
||||
"-f",
|
||||
"--force",
|
||||
action="store_true",
|
||||
help="run deactivation even if spec is NOT currently activated",
|
||||
)
|
||||
subparser.add_argument("-v", "--view", metavar="VIEW", type=str, help="the view to operate on")
|
||||
subparser.add_argument(
|
||||
"-a",
|
||||
"--all",
|
||||
action="store_true",
|
||||
help="deactivate all extensions of an extendable package, or "
|
||||
"deactivate an extension AND its dependencies",
|
||||
)
|
||||
arguments.add_common_arguments(subparser, ["installed_spec"])
|
||||
|
||||
|
||||
def deactivate(parser, args):
|
||||
|
||||
tty.warn(
|
||||
"spack deactivate is deprecated in favor of " "environments and will be removed in v0.19.0"
|
||||
)
|
||||
|
||||
specs = spack.cmd.parse_specs(args.spec)
|
||||
if len(specs) != 1:
|
||||
tty.die("deactivate requires one spec. %d given." % len(specs))
|
||||
|
||||
env = ev.active_environment()
|
||||
spec = spack.cmd.disambiguate_spec(specs[0], env)
|
||||
pkg = spec.package
|
||||
|
||||
if args.view:
|
||||
target = args.view
|
||||
elif pkg.is_extension:
|
||||
target = pkg.extendee_spec.prefix
|
||||
elif pkg.extendable:
|
||||
target = spec.prefix
|
||||
|
||||
view = YamlFilesystemView(target, spack.store.layout)
|
||||
|
||||
if args.all:
|
||||
if pkg.extendable:
|
||||
tty.msg("Deactivating all extensions of %s" % pkg.spec.short_spec)
|
||||
ext_pkgs = spack.store.db.activated_extensions_for(spec, view.extensions_layout)
|
||||
|
||||
for ext_pkg in ext_pkgs:
|
||||
ext_pkg.spec.normalize()
|
||||
if ext_pkg.is_activated(view):
|
||||
ext_pkg.do_deactivate(view, force=True)
|
||||
|
||||
elif pkg.is_extension:
|
||||
if not args.force and not spec.package.is_activated(view):
|
||||
tty.die("%s is not activated." % pkg.spec.short_spec)
|
||||
|
||||
tty.msg("Deactivating %s and all dependencies." % pkg.spec.short_spec)
|
||||
|
||||
nodes_in_topological_order = spack.graph.topological_sort(spec)
|
||||
for espec in reversed(nodes_in_topological_order):
|
||||
epkg = espec.package
|
||||
if epkg.extends(pkg.extendee_spec):
|
||||
if epkg.is_activated(view) or args.force:
|
||||
epkg.do_deactivate(view, force=args.force)
|
||||
|
||||
else:
|
||||
tty.die("spack deactivate --all requires an extendable package " "or an extension.")
|
||||
|
||||
else:
|
||||
if not pkg.is_extension:
|
||||
tty.die(
|
||||
"spack deactivate requires an extension.", "Did you mean 'spack deactivate --all'?"
|
||||
)
|
||||
|
||||
if not args.force and not spec.package.is_activated(view):
|
||||
tty.die("Package %s is not activated." % spec.short_spec)
|
||||
|
||||
spec.package.do_deactivate(view, force=args.force)
|
||||
|
|
@ -14,7 +14,6 @@
|
|||
import spack.environment as ev
|
||||
import spack.repo
|
||||
import spack.store
|
||||
from spack.filesystem_view import YamlFilesystemView
|
||||
|
||||
description = "list extensions for package"
|
||||
section = "extensions"
|
||||
|
|
@ -38,10 +37,9 @@ def setup_parser(subparser):
|
|||
"--show",
|
||||
action="store",
|
||||
default="all",
|
||||
choices=("packages", "installed", "activated", "all"),
|
||||
choices=("packages", "installed", "all"),
|
||||
help="show only part of output",
|
||||
)
|
||||
subparser.add_argument("-v", "--view", metavar="VIEW", type=str, help="the view to operate on")
|
||||
|
||||
subparser.add_argument(
|
||||
"spec",
|
||||
|
|
@ -91,13 +89,6 @@ def extensions(parser, args):
|
|||
tty.msg("%d extensions:" % len(extensions))
|
||||
colify(ext.name for ext in extensions)
|
||||
|
||||
if args.view:
|
||||
target = args.view
|
||||
else:
|
||||
target = spec.prefix
|
||||
|
||||
view = YamlFilesystemView(target, spack.store.layout)
|
||||
|
||||
if args.show in ("installed", "all"):
|
||||
# List specs of installed extensions.
|
||||
installed = [s.spec for s in spack.store.db.installed_extensions_for(spec)]
|
||||
|
|
@ -109,14 +100,3 @@ def extensions(parser, args):
|
|||
else:
|
||||
tty.msg("%d installed:" % len(installed))
|
||||
cmd.display_specs(installed, args)
|
||||
|
||||
if args.show in ("activated", "all"):
|
||||
# List specs of activated extensions.
|
||||
activated = view.extensions_layout.extension_map(spec)
|
||||
if args.show == "all":
|
||||
print
|
||||
if not activated:
|
||||
tty.msg("None activated.")
|
||||
else:
|
||||
tty.msg("%d activated:" % len(activated))
|
||||
cmd.display_specs(activated.values(), args)
|
||||
|
|
|
|||
|
|
@ -53,7 +53,6 @@
|
|||
InconsistentInstallDirectoryError,
|
||||
)
|
||||
from spack.error import SpackError
|
||||
from spack.filesystem_view import YamlFilesystemView
|
||||
from spack.util.crypto import bit_length
|
||||
from spack.version import Version
|
||||
|
||||
|
|
@ -1379,23 +1378,6 @@ def installed_extensions_for(self, extendee_spec):
|
|||
if spec.package.extends(extendee_spec):
|
||||
yield spec.package
|
||||
|
||||
@_autospec
|
||||
def activated_extensions_for(self, extendee_spec, extensions_layout=None):
|
||||
"""
|
||||
Return the specs of all packages that extend
|
||||
the given spec
|
||||
"""
|
||||
if extensions_layout is None:
|
||||
view = YamlFilesystemView(extendee_spec.prefix, spack.store.layout)
|
||||
extensions_layout = view.extensions_layout
|
||||
for spec in self.query():
|
||||
try:
|
||||
extensions_layout.check_activated(extendee_spec, spec)
|
||||
yield spec.package
|
||||
except spack.directory_layout.NoSuchExtensionError:
|
||||
continue
|
||||
# TODO: conditional way to do this instead of catching exceptions
|
||||
|
||||
def _get_by_hash_local(self, dag_hash, default=None, installed=any):
|
||||
# hash is a full hash and is in the data somewhere
|
||||
if dag_hash in self._data:
|
||||
|
|
|
|||
|
|
@ -468,14 +468,7 @@ def _execute_depends_on(pkg):
|
|||
|
||||
@directive(("extendees", "dependencies"))
|
||||
def extends(spec, type=("build", "run"), **kwargs):
|
||||
"""Same as depends_on, but allows symlinking into dependency's
|
||||
prefix tree.
|
||||
|
||||
This is for Python and other language modules where the module
|
||||
needs to be installed into the prefix of the Python installation.
|
||||
Spack handles this by installing modules into their own prefix,
|
||||
but allowing ONE module version to be symlinked into a parent
|
||||
Python install at a time, using ``spack activate``.
|
||||
"""Same as depends_on, but also adds this package to the extendee list.
|
||||
|
||||
keyword arguments can be passed to extends() so that extension
|
||||
packages can pass parameters to the extendee's extension
|
||||
|
|
|
|||
|
|
@ -10,10 +10,8 @@
|
|||
import re
|
||||
import shutil
|
||||
import sys
|
||||
import tempfile
|
||||
from contextlib import contextmanager
|
||||
|
||||
import ruamel.yaml as yaml
|
||||
import six
|
||||
|
||||
import llnl.util.filesystem as fs
|
||||
|
|
@ -389,205 +387,6 @@ def remove_install_directory(self, spec, deprecated=False):
|
|||
path = os.path.dirname(path)
|
||||
|
||||
|
||||
class ExtensionsLayout(object):
|
||||
"""A directory layout is used to associate unique paths with specs for
|
||||
package extensions.
|
||||
Keeps track of which extensions are activated for what package.
|
||||
Depending on the use case, this can mean globally activated extensions
|
||||
directly in the installation folder - or extensions activated in
|
||||
filesystem views.
|
||||
"""
|
||||
|
||||
def __init__(self, view, **kwargs):
|
||||
self.view = view
|
||||
|
||||
def add_extension(self, spec, ext_spec):
|
||||
"""Add to the list of currently installed extensions."""
|
||||
raise NotImplementedError()
|
||||
|
||||
def check_activated(self, spec, ext_spec):
|
||||
"""Ensure that ext_spec can be removed from spec.
|
||||
|
||||
If not, raise NoSuchExtensionError.
|
||||
"""
|
||||
raise NotImplementedError()
|
||||
|
||||
def check_extension_conflict(self, spec, ext_spec):
|
||||
"""Ensure that ext_spec can be activated in spec.
|
||||
|
||||
If not, raise ExtensionAlreadyInstalledError or
|
||||
ExtensionConflictError.
|
||||
"""
|
||||
raise NotImplementedError()
|
||||
|
||||
def extension_map(self, spec):
|
||||
"""Get a dict of currently installed extension packages for a spec.
|
||||
|
||||
Dict maps { name : extension_spec }
|
||||
Modifying dict does not affect internals of this layout.
|
||||
"""
|
||||
raise NotImplementedError()
|
||||
|
||||
def extendee_target_directory(self, extendee):
|
||||
"""Specify to which full path extendee should link all files
|
||||
from extensions."""
|
||||
raise NotImplementedError
|
||||
|
||||
def remove_extension(self, spec, ext_spec):
|
||||
"""Remove from the list of currently installed extensions."""
|
||||
raise NotImplementedError()
|
||||
|
||||
|
||||
class YamlViewExtensionsLayout(ExtensionsLayout):
|
||||
"""Maintain extensions within a view."""
|
||||
|
||||
def __init__(self, view, layout):
|
||||
"""layout is the corresponding YamlDirectoryLayout object for which
|
||||
we implement extensions.
|
||||
"""
|
||||
super(YamlViewExtensionsLayout, self).__init__(view)
|
||||
self.layout = layout
|
||||
self.extension_file_name = "extensions.yaml"
|
||||
|
||||
# Cache of already written/read extension maps.
|
||||
self._extension_maps = {}
|
||||
|
||||
def add_extension(self, spec, ext_spec):
|
||||
_check_concrete(spec)
|
||||
_check_concrete(ext_spec)
|
||||
|
||||
# Check whether it's already installed or if it's a conflict.
|
||||
exts = self._extension_map(spec)
|
||||
self.check_extension_conflict(spec, ext_spec)
|
||||
|
||||
# do the actual adding.
|
||||
exts[ext_spec.name] = ext_spec
|
||||
self._write_extensions(spec, exts)
|
||||
|
||||
def check_extension_conflict(self, spec, ext_spec):
|
||||
exts = self._extension_map(spec)
|
||||
if ext_spec.name in exts:
|
||||
installed_spec = exts[ext_spec.name]
|
||||
if ext_spec.dag_hash() == installed_spec.dag_hash():
|
||||
raise ExtensionAlreadyInstalledError(spec, ext_spec)
|
||||
else:
|
||||
raise ExtensionConflictError(spec, ext_spec, installed_spec)
|
||||
|
||||
def check_activated(self, spec, ext_spec):
|
||||
exts = self._extension_map(spec)
|
||||
if (ext_spec.name not in exts) or (ext_spec != exts[ext_spec.name]):
|
||||
raise NoSuchExtensionError(spec, ext_spec)
|
||||
|
||||
def extension_file_path(self, spec):
|
||||
"""Gets full path to an installed package's extension file, which
|
||||
keeps track of all the extensions for that package which have been
|
||||
added to this view.
|
||||
"""
|
||||
_check_concrete(spec)
|
||||
normalize_path = lambda p: (os.path.abspath(p).rstrip(os.path.sep))
|
||||
|
||||
view_prefix = self.view.get_projection_for_spec(spec)
|
||||
if normalize_path(spec.prefix) == normalize_path(view_prefix):
|
||||
# For backwards compatibility, when the view is the extended
|
||||
# package's installation directory, do not include the spec name
|
||||
# as a subdirectory.
|
||||
components = [view_prefix, self.layout.metadata_dir, self.extension_file_name]
|
||||
else:
|
||||
components = [
|
||||
view_prefix,
|
||||
self.layout.metadata_dir,
|
||||
spec.name,
|
||||
self.extension_file_name,
|
||||
]
|
||||
|
||||
return os.path.join(*components)
|
||||
|
||||
def extension_map(self, spec):
|
||||
"""Defensive copying version of _extension_map() for external API."""
|
||||
_check_concrete(spec)
|
||||
return self._extension_map(spec).copy()
|
||||
|
||||
def remove_extension(self, spec, ext_spec):
|
||||
_check_concrete(spec)
|
||||
_check_concrete(ext_spec)
|
||||
|
||||
# Make sure it's installed before removing.
|
||||
exts = self._extension_map(spec)
|
||||
self.check_activated(spec, ext_spec)
|
||||
|
||||
# do the actual removing.
|
||||
del exts[ext_spec.name]
|
||||
self._write_extensions(spec, exts)
|
||||
|
||||
def _extension_map(self, spec):
|
||||
"""Get a dict<name -> spec> for all extensions currently
|
||||
installed for this package."""
|
||||
_check_concrete(spec)
|
||||
|
||||
if spec not in self._extension_maps:
|
||||
path = self.extension_file_path(spec)
|
||||
if not os.path.exists(path):
|
||||
self._extension_maps[spec] = {}
|
||||
|
||||
else:
|
||||
by_hash = self.layout.specs_by_hash()
|
||||
exts = {}
|
||||
with open(path) as ext_file:
|
||||
yaml_file = yaml.load(ext_file)
|
||||
for entry in yaml_file["extensions"]:
|
||||
name = next(iter(entry))
|
||||
dag_hash = entry[name]["hash"]
|
||||
prefix = entry[name]["path"]
|
||||
|
||||
if dag_hash not in by_hash:
|
||||
raise InvalidExtensionSpecError(
|
||||
"Spec %s not found in %s" % (dag_hash, prefix)
|
||||
)
|
||||
|
||||
ext_spec = by_hash[dag_hash]
|
||||
if prefix != ext_spec.prefix:
|
||||
raise InvalidExtensionSpecError(
|
||||
"Prefix %s does not match spec hash %s: %s"
|
||||
% (prefix, dag_hash, ext_spec)
|
||||
)
|
||||
|
||||
exts[ext_spec.name] = ext_spec
|
||||
self._extension_maps[spec] = exts
|
||||
|
||||
return self._extension_maps[spec]
|
||||
|
||||
def _write_extensions(self, spec, extensions):
|
||||
path = self.extension_file_path(spec)
|
||||
|
||||
if not extensions:
|
||||
# Remove the empty extensions file
|
||||
os.remove(path)
|
||||
return
|
||||
|
||||
# Create a temp file in the same directory as the actual file.
|
||||
dirname, basename = os.path.split(path)
|
||||
fs.mkdirp(dirname)
|
||||
|
||||
tmp = tempfile.NamedTemporaryFile(prefix=basename, dir=dirname, delete=False)
|
||||
|
||||
# write tmp file
|
||||
with tmp:
|
||||
yaml.dump(
|
||||
{
|
||||
"extensions": [
|
||||
{ext.name: {"hash": ext.dag_hash(), "path": str(ext.prefix)}}
|
||||
for ext in sorted(extensions.values())
|
||||
]
|
||||
},
|
||||
tmp,
|
||||
default_flow_style=False,
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
# Atomic update by moving tmpfile on top of old one.
|
||||
fs.rename(tmp.name, path)
|
||||
|
||||
|
||||
class DirectoryLayoutError(SpackError):
|
||||
"""Superclass for directory layout errors."""
|
||||
|
||||
|
|
@ -644,13 +443,3 @@ def __init__(self, spec, ext_spec, conflict):
|
|||
"%s cannot be installed in %s because it conflicts with %s"
|
||||
% (ext_spec.short_spec, spec.short_spec, conflict.short_spec)
|
||||
)
|
||||
|
||||
|
||||
class NoSuchExtensionError(DirectoryLayoutError):
|
||||
"""Raised when an extension isn't there on deactivate."""
|
||||
|
||||
def __init__(self, spec, ext_spec):
|
||||
super(NoSuchExtensionError, self).__init__(
|
||||
"%s cannot be removed from %s because it's not activated."
|
||||
% (ext_spec.short_spec, spec.short_spec)
|
||||
)
|
||||
|
|
|
|||
|
|
@ -37,10 +37,6 @@
|
|||
import spack.store
|
||||
import spack.util.spack_json as s_json
|
||||
import spack.util.spack_yaml as s_yaml
|
||||
from spack.directory_layout import (
|
||||
ExtensionAlreadyInstalledError,
|
||||
YamlViewExtensionsLayout,
|
||||
)
|
||||
from spack.error import SpackError
|
||||
|
||||
__all__ = ["FilesystemView", "YamlFilesystemView"]
|
||||
|
|
@ -166,9 +162,6 @@ def add_specs(self, *specs, **kwargs):
|
|||
"""
|
||||
Add given specs to view.
|
||||
|
||||
The supplied specs might be standalone packages or extensions of
|
||||
other packages.
|
||||
|
||||
Should accept `with_dependencies` as keyword argument (default
|
||||
True) to indicate wether or not dependencies should be activated as
|
||||
well.
|
||||
|
|
@ -176,13 +169,7 @@ def add_specs(self, *specs, **kwargs):
|
|||
Should except an `exclude` keyword argument containing a list of
|
||||
regexps that filter out matching spec names.
|
||||
|
||||
This method should make use of `activate_{extension,standalone}`.
|
||||
"""
|
||||
raise NotImplementedError
|
||||
|
||||
def add_extension(self, spec):
|
||||
"""
|
||||
Add (link) an extension in this view. Does not add dependencies.
|
||||
This method should make use of `activate_standalone`.
|
||||
"""
|
||||
raise NotImplementedError
|
||||
|
||||
|
|
@ -202,9 +189,6 @@ def remove_specs(self, *specs, **kwargs):
|
|||
"""
|
||||
Removes given specs from view.
|
||||
|
||||
The supplied spec might be a standalone package or an extension of
|
||||
another package.
|
||||
|
||||
Should accept `with_dependencies` as keyword argument (default
|
||||
True) to indicate wether or not dependencies should be deactivated
|
||||
as well.
|
||||
|
|
@ -216,13 +200,7 @@ def remove_specs(self, *specs, **kwargs):
|
|||
Should except an `exclude` keyword argument containing a list of
|
||||
regexps that filter out matching spec names.
|
||||
|
||||
This method should make use of `deactivate_{extension,standalone}`.
|
||||
"""
|
||||
raise NotImplementedError
|
||||
|
||||
def remove_extension(self, spec):
|
||||
"""
|
||||
Remove (unlink) an extension from this view.
|
||||
This method should make use of `deactivate_standalone`.
|
||||
"""
|
||||
raise NotImplementedError
|
||||
|
||||
|
|
@ -296,8 +274,6 @@ def __init__(self, root, layout, **kwargs):
|
|||
msg += " which does not match projections passed manually."
|
||||
raise ConflictingProjectionsError(msg)
|
||||
|
||||
self.extensions_layout = YamlViewExtensionsLayout(self, layout)
|
||||
|
||||
self._croot = colorize_root(self._root) + " "
|
||||
|
||||
def write_projections(self):
|
||||
|
|
@ -332,38 +308,10 @@ def add_specs(self, *specs, **kwargs):
|
|||
self.print_conflict(v, s)
|
||||
return
|
||||
|
||||
extensions = set(filter(lambda s: s.package.is_extension, specs))
|
||||
standalones = specs - extensions
|
||||
|
||||
set(map(self._check_no_ext_conflicts, extensions))
|
||||
# fail on first error, otherwise link extensions as well
|
||||
if all(map(self.add_standalone, standalones)):
|
||||
all(map(self.add_extension, extensions))
|
||||
|
||||
def add_extension(self, spec):
|
||||
if not spec.package.is_extension:
|
||||
tty.error(self._croot + "Package %s is not an extension." % spec.name)
|
||||
return False
|
||||
|
||||
if spec.external:
|
||||
tty.warn(self._croot + "Skipping external package: %s" % colorize_spec(spec))
|
||||
return True
|
||||
|
||||
if not spec.package.is_activated(self):
|
||||
spec.package.do_activate(self, verbose=self.verbose, with_dependencies=False)
|
||||
|
||||
# make sure the meta folder is linked as well (this is not done by the
|
||||
# extension-activation mechnism)
|
||||
if not self.check_added(spec):
|
||||
self.link_meta_folder(spec)
|
||||
|
||||
return True
|
||||
for s in specs:
|
||||
self.add_standalone(s)
|
||||
|
||||
def add_standalone(self, spec):
|
||||
if spec.package.is_extension:
|
||||
tty.error(self._croot + "Package %s is an extension." % spec.name)
|
||||
return False
|
||||
|
||||
if spec.external:
|
||||
tty.warn(self._croot + "Skipping external package: %s" % colorize_spec(spec))
|
||||
return True
|
||||
|
|
@ -372,19 +320,6 @@ def add_standalone(self, spec):
|
|||
tty.warn(self._croot + "Skipping already linked package: %s" % colorize_spec(spec))
|
||||
return True
|
||||
|
||||
if spec.package.extendable:
|
||||
# Check for globally activated extensions in the extendee that
|
||||
# we're looking at.
|
||||
activated = [p.spec for p in spack.store.db.activated_extensions_for(spec)]
|
||||
if activated:
|
||||
tty.error(
|
||||
"Globally activated extensions cannot be used in "
|
||||
"conjunction with filesystem views. "
|
||||
"Please deactivate the following specs: "
|
||||
)
|
||||
spack.cmd.display_specs(activated, flags=True, variants=True, long=False)
|
||||
return False
|
||||
|
||||
self.merge(spec)
|
||||
|
||||
self.link_meta_folder(spec)
|
||||
|
|
@ -533,27 +468,10 @@ def remove_specs(self, *specs, **kwargs):
|
|||
|
||||
# Remove the packages from the view
|
||||
for spec in to_deactivate_sorted:
|
||||
if spec.package.is_extension:
|
||||
self.remove_extension(spec, with_dependents=with_dependents)
|
||||
else:
|
||||
self.remove_standalone(spec)
|
||||
self.remove_standalone(spec)
|
||||
|
||||
self._purge_empty_directories()
|
||||
|
||||
def remove_extension(self, spec, with_dependents=True):
|
||||
"""
|
||||
Remove (unlink) an extension from this view.
|
||||
"""
|
||||
if not self.check_added(spec):
|
||||
tty.warn(self._croot + "Skipping package not linked in view: %s" % spec.name)
|
||||
return
|
||||
|
||||
if spec.package.is_activated(self):
|
||||
spec.package.do_deactivate(
|
||||
self, verbose=self.verbose, remove_dependents=with_dependents
|
||||
)
|
||||
self.unlink_meta_folder(spec)
|
||||
|
||||
def remove_standalone(self, spec):
|
||||
"""
|
||||
Remove (unlink) a standalone package from this view.
|
||||
|
|
@ -575,14 +493,9 @@ def get_projection_for_spec(self, spec):
|
|||
Relies on the ordering of projections to avoid ambiguity.
|
||||
"""
|
||||
spec = spack.spec.Spec(spec)
|
||||
# Extensions are placed by their extendee, not by their own spec
|
||||
locator_spec = spec
|
||||
if spec.package.extendee_spec:
|
||||
locator_spec = spec.package.extendee_spec
|
||||
|
||||
proj = spack.projections.get_projection(self.projections, locator_spec)
|
||||
proj = spack.projections.get_projection(self.projections, spec)
|
||||
if proj:
|
||||
return os.path.join(self._root, locator_spec.format(proj))
|
||||
return os.path.join(self._root, spec.format(proj))
|
||||
return self._root
|
||||
|
||||
def get_all_specs(self):
|
||||
|
|
@ -712,18 +625,6 @@ def unlink_meta_folder(self, spec):
|
|||
assert os.path.exists(path)
|
||||
shutil.rmtree(path)
|
||||
|
||||
def _check_no_ext_conflicts(self, spec):
|
||||
"""
|
||||
Check that there is no extension conflict for specs.
|
||||
"""
|
||||
extendee = spec.package.extendee_spec
|
||||
try:
|
||||
self.extensions_layout.check_extension_conflict(extendee, spec)
|
||||
except ExtensionAlreadyInstalledError:
|
||||
# we print the warning here because later on the order in which
|
||||
# packages get activated is not clear (set-sorting)
|
||||
tty.warn(self._croot + "Skipping already activated package: %s" % spec.name)
|
||||
|
||||
|
||||
class SimpleFilesystemView(FilesystemView):
|
||||
"""A simple and partial implementation of FilesystemView focused on
|
||||
|
|
@ -842,14 +743,9 @@ def get_projection_for_spec(self, spec):
|
|||
Relies on the ordering of projections to avoid ambiguity.
|
||||
"""
|
||||
spec = spack.spec.Spec(spec)
|
||||
# Extensions are placed by their extendee, not by their own spec
|
||||
locator_spec = spec
|
||||
if spec.package.extendee_spec:
|
||||
locator_spec = spec.package.extendee_spec
|
||||
|
||||
proj = spack.projections.get_projection(self.projections, locator_spec)
|
||||
proj = spack.projections.get_projection(self.projections, spec)
|
||||
if proj:
|
||||
return os.path.join(self._root, locator_spec.format(proj))
|
||||
return os.path.join(self._root, spec.format(proj))
|
||||
return self._root
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -1,20 +0,0 @@
|
|||
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
|
||||
# Spack Project Developers. See the top-level COPYRIGHT file for details.
|
||||
#
|
||||
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
|
||||
|
||||
import spack
|
||||
from spack.filesystem_view import YamlFilesystemView
|
||||
|
||||
|
||||
def pre_uninstall(spec):
|
||||
pkg = spec.package
|
||||
assert spec.concrete
|
||||
|
||||
if pkg.is_extension:
|
||||
target = pkg.extendee_spec.prefix
|
||||
view = YamlFilesystemView(target, spack.store.layout)
|
||||
|
||||
if pkg.is_activated(view):
|
||||
# deactivate globally
|
||||
pkg.do_deactivate(force=True)
|
||||
|
|
@ -1313,19 +1313,6 @@ def extends(self, spec):
|
|||
s = self.extendee_spec
|
||||
return s and spec.satisfies(s)
|
||||
|
||||
def is_activated(self, view):
|
||||
"""Return True if package is activated."""
|
||||
if not self.is_extension:
|
||||
raise ValueError("is_activated called on package that is not an extension.")
|
||||
if self.extendee_spec.installed_upstream:
|
||||
# If this extends an upstream package, it cannot be activated for
|
||||
# it. This bypasses construction of the extension map, which can
|
||||
# can fail when run in the context of a downstream Spack instance
|
||||
return False
|
||||
extensions_layout = view.extensions_layout
|
||||
exts = extensions_layout.extension_map(self.extendee_spec)
|
||||
return (self.name in exts) and (exts[self.name] == self.spec)
|
||||
|
||||
def provides(self, vpkg_name):
|
||||
"""
|
||||
True if this package provides a virtual package with the specified name
|
||||
|
|
@ -2325,30 +2312,6 @@ def do_deprecate(self, deprecator, link_fn):
|
|||
"""Deprecate this package in favor of deprecator spec"""
|
||||
spec = self.spec
|
||||
|
||||
# Check whether package to deprecate has active extensions
|
||||
if self.extendable:
|
||||
view = spack.filesystem_view.YamlFilesystemView(spec.prefix, spack.store.layout)
|
||||
active_exts = view.extensions_layout.extension_map(spec).values()
|
||||
if active_exts:
|
||||
short = spec.format("{name}/{hash:7}")
|
||||
m = "Spec %s has active extensions\n" % short
|
||||
for active in active_exts:
|
||||
m += " %s\n" % active.format("{name}/{hash:7}")
|
||||
m += "Deactivate extensions before deprecating %s" % short
|
||||
tty.die(m)
|
||||
|
||||
# Check whether package to deprecate is an active extension
|
||||
if self.is_extension:
|
||||
extendee = self.extendee_spec
|
||||
view = spack.filesystem_view.YamlFilesystemView(extendee.prefix, spack.store.layout)
|
||||
|
||||
if self.is_activated(view):
|
||||
short = spec.format("{name}/{hash:7}")
|
||||
short_ext = extendee.format("{name}/{hash:7}")
|
||||
msg = "Spec %s is an active extension of %s\n" % (short, short_ext)
|
||||
msg += "Deactivate %s to be able to deprecate it" % short
|
||||
tty.die(msg)
|
||||
|
||||
# Install deprecator if it isn't installed already
|
||||
if not spack.store.db.query(deprecator):
|
||||
deprecator.package.do_install()
|
||||
|
|
@ -2378,155 +2341,6 @@ def _check_extendable(self):
|
|||
if not self.extendable:
|
||||
raise ValueError("Package %s is not extendable!" % self.name)
|
||||
|
||||
def _sanity_check_extension(self):
|
||||
if not self.is_extension:
|
||||
raise ActivationError("This package is not an extension.")
|
||||
|
||||
extendee_package = self.extendee_spec.package
|
||||
extendee_package._check_extendable()
|
||||
|
||||
if not self.extendee_spec.installed:
|
||||
raise ActivationError("Can only (de)activate extensions for installed packages.")
|
||||
if not self.spec.installed:
|
||||
raise ActivationError("Extensions must first be installed.")
|
||||
if self.extendee_spec.name not in self.extendees:
|
||||
raise ActivationError("%s does not extend %s!" % (self.name, self.extendee.name))
|
||||
|
||||
def do_activate(self, view=None, with_dependencies=True, verbose=True):
|
||||
"""Called on an extension to invoke the extendee's activate method.
|
||||
|
||||
Commands should call this routine, and should not call
|
||||
activate() directly.
|
||||
"""
|
||||
if verbose:
|
||||
tty.msg(
|
||||
"Activating extension {0} for {1}".format(
|
||||
self.spec.cshort_spec, self.extendee_spec.cshort_spec
|
||||
)
|
||||
)
|
||||
|
||||
self._sanity_check_extension()
|
||||
if not view:
|
||||
view = YamlFilesystemView(self.extendee_spec.prefix, spack.store.layout)
|
||||
|
||||
extensions_layout = view.extensions_layout
|
||||
|
||||
try:
|
||||
extensions_layout.check_extension_conflict(self.extendee_spec, self.spec)
|
||||
except spack.directory_layout.ExtensionAlreadyInstalledError as e:
|
||||
# already installed, let caller know
|
||||
tty.msg(e.message)
|
||||
return
|
||||
|
||||
# Activate any package dependencies that are also extensions.
|
||||
if with_dependencies:
|
||||
for spec in self.dependency_activations():
|
||||
if not spec.package.is_activated(view):
|
||||
spec.package.do_activate(
|
||||
view, with_dependencies=with_dependencies, verbose=verbose
|
||||
)
|
||||
|
||||
self.extendee_spec.package.activate(self, view, **self.extendee_args)
|
||||
|
||||
extensions_layout.add_extension(self.extendee_spec, self.spec)
|
||||
|
||||
if verbose:
|
||||
tty.debug(
|
||||
"Activated extension {0} for {1}".format(
|
||||
self.spec.cshort_spec, self.extendee_spec.cshort_spec
|
||||
)
|
||||
)
|
||||
|
||||
def dependency_activations(self):
|
||||
return (
|
||||
spec
|
||||
for spec in self.spec.traverse(root=False, deptype="run")
|
||||
if spec.package.extends(self.extendee_spec)
|
||||
)
|
||||
|
||||
def activate(self, extension, view, **kwargs):
|
||||
"""
|
||||
Add the extension to the specified view.
|
||||
|
||||
Package authors can override this function to maintain some
|
||||
centralized state related to the set of activated extensions
|
||||
for a package.
|
||||
|
||||
Spack internals (commands, hooks, etc.) should call
|
||||
do_activate() method so that proper checks are always executed.
|
||||
"""
|
||||
view.merge(extension.spec, ignore=kwargs.get("ignore", None))
|
||||
|
||||
def do_deactivate(self, view=None, **kwargs):
|
||||
"""Remove this extension package from the specified view. Called
|
||||
on the extension to invoke extendee's deactivate() method.
|
||||
|
||||
`remove_dependents=True` deactivates extensions depending on this
|
||||
package instead of raising an error.
|
||||
"""
|
||||
self._sanity_check_extension()
|
||||
force = kwargs.get("force", False)
|
||||
verbose = kwargs.get("verbose", True)
|
||||
remove_dependents = kwargs.get("remove_dependents", False)
|
||||
|
||||
if verbose:
|
||||
tty.msg(
|
||||
"Deactivating extension {0} for {1}".format(
|
||||
self.spec.cshort_spec, self.extendee_spec.cshort_spec
|
||||
)
|
||||
)
|
||||
|
||||
if not view:
|
||||
view = YamlFilesystemView(self.extendee_spec.prefix, spack.store.layout)
|
||||
extensions_layout = view.extensions_layout
|
||||
|
||||
# Allow a force deactivate to happen. This can unlink
|
||||
# spurious files if something was corrupted.
|
||||
if not force:
|
||||
extensions_layout.check_activated(self.extendee_spec, self.spec)
|
||||
|
||||
activated = extensions_layout.extension_map(self.extendee_spec)
|
||||
for name, aspec in activated.items():
|
||||
if aspec == self.spec:
|
||||
continue
|
||||
for dep in aspec.traverse(deptype="run"):
|
||||
if self.spec == dep:
|
||||
if remove_dependents:
|
||||
aspec.package.do_deactivate(**kwargs)
|
||||
else:
|
||||
msg = (
|
||||
"Cannot deactivate {0} because {1} is "
|
||||
"activated and depends on it"
|
||||
)
|
||||
raise ActivationError(
|
||||
msg.format(self.spec.cshort_spec, aspec.cshort_spec)
|
||||
)
|
||||
|
||||
self.extendee_spec.package.deactivate(self, view, **self.extendee_args)
|
||||
|
||||
# redundant activation check -- makes SURE the spec is not
|
||||
# still activated even if something was wrong above.
|
||||
if self.is_activated(view):
|
||||
extensions_layout.remove_extension(self.extendee_spec, self.spec)
|
||||
|
||||
if verbose:
|
||||
tty.debug(
|
||||
"Deactivated extension {0} for {1}".format(
|
||||
self.spec.cshort_spec, self.extendee_spec.cshort_spec
|
||||
)
|
||||
)
|
||||
|
||||
def deactivate(self, extension, view, **kwargs):
|
||||
"""
|
||||
Remove all extension files from the specified view.
|
||||
|
||||
Package authors can override this method to support other
|
||||
extension mechanisms. Spack internals (commands, hooks, etc.)
|
||||
should call do_deactivate() method so that proper checks are
|
||||
always executed.
|
||||
"""
|
||||
view.unmerge(extension.spec, ignore=kwargs.get("ignore", None))
|
||||
|
||||
def view(self):
|
||||
"""Create a view with the prefix of this package as the root.
|
||||
Extensions added to this view will modify the installation prefix of
|
||||
|
|
|
|||
|
|
@ -1,41 +0,0 @@
|
|||
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
|
||||
# Spack Project Developers. See the top-level COPYRIGHT file for details.
|
||||
#
|
||||
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
from spack.main import SpackCommand
|
||||
|
||||
activate = SpackCommand("activate")
|
||||
deactivate = SpackCommand("deactivate")
|
||||
install = SpackCommand("install")
|
||||
extensions = SpackCommand("extensions")
|
||||
|
||||
pytestmark = pytest.mark.skipif(sys.platform == "win32", reason="does not run on windows")
|
||||
|
||||
|
||||
def test_activate(mock_packages, mock_archive, mock_fetch, config, install_mockery):
|
||||
install("extension1")
|
||||
activate("extension1")
|
||||
output = extensions("--show", "activated", "extendee")
|
||||
assert "extension1" in output
|
||||
|
||||
|
||||
def test_deactivate(mock_packages, mock_archive, mock_fetch, config, install_mockery):
|
||||
install("extension1")
|
||||
activate("extension1")
|
||||
deactivate("extension1")
|
||||
output = extensions("--show", "activated", "extendee")
|
||||
assert "extension1" not in output
|
||||
|
||||
|
||||
def test_deactivate_all(mock_packages, mock_archive, mock_fetch, config, install_mockery):
|
||||
install("extension1")
|
||||
install("extension2")
|
||||
activate("extension1")
|
||||
activate("extension2")
|
||||
deactivate("--all", "extendee")
|
||||
output = extensions("--show", "activated", "extendee")
|
||||
assert "extension1" not in output
|
||||
|
|
@ -15,7 +15,6 @@
|
|||
uninstall = SpackCommand("uninstall")
|
||||
deprecate = SpackCommand("deprecate")
|
||||
find = SpackCommand("find")
|
||||
activate = SpackCommand("activate")
|
||||
|
||||
pytestmark = pytest.mark.skipif(sys.platform == "win32", reason="does not run on windows")
|
||||
|
||||
|
|
@ -89,24 +88,6 @@ def test_deprecate_deps(mock_packages, mock_archive, mock_fetch, install_mockery
|
|||
assert sorted(deprecated) == sorted(list(old_spec.traverse()))
|
||||
|
||||
|
||||
def test_deprecate_fails_active_extensions(
|
||||
mock_packages, mock_archive, mock_fetch, install_mockery
|
||||
):
|
||||
"""Tests that active extensions and their extendees cannot be
|
||||
deprecated."""
|
||||
install("extendee")
|
||||
install("extension1")
|
||||
activate("extension1")
|
||||
|
||||
output = deprecate("-yi", "extendee", "extendee@nonexistent", fail_on_error=False)
|
||||
assert "extension1" in output
|
||||
assert "Deactivate extensions before deprecating" in output
|
||||
|
||||
output = deprecate("-yiD", "extension1", "extension1@notaversion", fail_on_error=False)
|
||||
assert "extendee" in output
|
||||
assert "is an active extension of" in output
|
||||
|
||||
|
||||
def test_uninstall_deprecated(mock_packages, mock_archive, mock_fetch, install_mockery):
|
||||
"""Tests that we can still uninstall deprecated packages."""
|
||||
install("libelf@0.8.13")
|
||||
|
|
|
|||
|
|
@ -35,12 +35,11 @@ def python_database(mock_packages, mutable_database):
|
|||
def test_extensions(mock_packages, python_database, config, capsys):
|
||||
ext2 = Spec("py-extension2").concretized()
|
||||
|
||||
def check_output(ni, na):
|
||||
def check_output(ni):
|
||||
with capsys.disabled():
|
||||
output = extensions("python")
|
||||
packages = extensions("-s", "packages", "python")
|
||||
installed = extensions("-s", "installed", "python")
|
||||
activated = extensions("-s", "activated", "python")
|
||||
assert "==> python@2.7.11" in output
|
||||
assert "==> 2 extensions" in output
|
||||
assert "py-extension1" in output
|
||||
|
|
@ -50,26 +49,13 @@ def check_output(ni, na):
|
|||
assert "py-extension1" in packages
|
||||
assert "py-extension2" in packages
|
||||
assert "installed" not in packages
|
||||
assert "activated" not in packages
|
||||
|
||||
assert ("%s installed" % (ni if ni else "None")) in output
|
||||
assert ("%s activated" % (na if na else "None")) in output
|
||||
assert ("%s installed" % (ni if ni else "None")) in installed
|
||||
assert ("%s activated" % (na if na else "None")) in activated
|
||||
|
||||
check_output(2, 0)
|
||||
|
||||
ext2.package.do_activate()
|
||||
check_output(2, 2)
|
||||
|
||||
ext2.package.do_deactivate(force=True)
|
||||
check_output(2, 1)
|
||||
|
||||
ext2.package.do_activate()
|
||||
check_output(2, 2)
|
||||
|
||||
check_output(2)
|
||||
ext2.package.do_uninstall(force=True)
|
||||
check_output(1, 1)
|
||||
check_output(1)
|
||||
|
||||
|
||||
def test_extensions_no_arguments(mock_packages):
|
||||
|
|
|
|||
|
|
@ -12,7 +12,6 @@
|
|||
from spack.main import SpackCommand
|
||||
from spack.spec import Spec
|
||||
|
||||
activate = SpackCommand("activate")
|
||||
extensions = SpackCommand("extensions")
|
||||
install = SpackCommand("install")
|
||||
view = SpackCommand("view")
|
||||
|
|
@ -135,46 +134,9 @@ def test_view_extension(tmpdir, mock_packages, mock_archive, mock_fetch, config,
|
|||
assert "extension1@1.0" in all_installed
|
||||
assert "extension1@2.0" in all_installed
|
||||
assert "extension2@1.0" in all_installed
|
||||
global_activated = extensions("--show", "activated", "extendee")
|
||||
assert "extension1@1.0" not in global_activated
|
||||
assert "extension1@2.0" not in global_activated
|
||||
assert "extension2@1.0" not in global_activated
|
||||
view_activated = extensions("--show", "activated", "-v", viewpath, "extendee")
|
||||
assert "extension1@1.0" in view_activated
|
||||
assert "extension1@2.0" not in view_activated
|
||||
assert "extension2@1.0" not in view_activated
|
||||
assert os.path.exists(os.path.join(viewpath, "bin", "extension1"))
|
||||
|
||||
|
||||
def test_view_extension_projection(
|
||||
tmpdir, mock_packages, mock_archive, mock_fetch, config, install_mockery
|
||||
):
|
||||
install("extendee@1.0")
|
||||
install("extension1@1.0")
|
||||
install("extension1@2.0")
|
||||
install("extension2@1.0")
|
||||
|
||||
viewpath = str(tmpdir.mkdir("view"))
|
||||
view_projection = {"all": "{name}-{version}"}
|
||||
projection_file = create_projection_file(tmpdir, view_projection)
|
||||
view("symlink", viewpath, "--projection-file={0}".format(projection_file), "extension1@1.0")
|
||||
|
||||
all_installed = extensions("--show", "installed", "extendee")
|
||||
assert "extension1@1.0" in all_installed
|
||||
assert "extension1@2.0" in all_installed
|
||||
assert "extension2@1.0" in all_installed
|
||||
global_activated = extensions("--show", "activated", "extendee")
|
||||
assert "extension1@1.0" not in global_activated
|
||||
assert "extension1@2.0" not in global_activated
|
||||
assert "extension2@1.0" not in global_activated
|
||||
view_activated = extensions("--show", "activated", "-v", viewpath, "extendee")
|
||||
assert "extension1@1.0" in view_activated
|
||||
assert "extension1@2.0" not in view_activated
|
||||
assert "extension2@1.0" not in view_activated
|
||||
|
||||
assert os.path.exists(os.path.join(viewpath, "extendee-1.0", "bin", "extension1"))
|
||||
|
||||
|
||||
def test_view_extension_remove(
|
||||
tmpdir, mock_packages, mock_archive, mock_fetch, config, install_mockery
|
||||
):
|
||||
|
|
@ -185,10 +147,6 @@ def test_view_extension_remove(
|
|||
view("remove", viewpath, "extension1@1.0")
|
||||
all_installed = extensions("--show", "installed", "extendee")
|
||||
assert "extension1@1.0" in all_installed
|
||||
global_activated = extensions("--show", "activated", "extendee")
|
||||
assert "extension1@1.0" not in global_activated
|
||||
view_activated = extensions("--show", "activated", "-v", viewpath, "extendee")
|
||||
assert "extension1@1.0" not in view_activated
|
||||
assert not os.path.exists(os.path.join(viewpath, "bin", "extension1"))
|
||||
|
||||
|
||||
|
|
@ -217,46 +175,6 @@ def test_view_extension_conflict_ignored(
|
|||
assert fin.read() == "1.0"
|
||||
|
||||
|
||||
def test_view_extension_global_activation(
|
||||
tmpdir, mock_packages, mock_archive, mock_fetch, config, install_mockery
|
||||
):
|
||||
install("extendee")
|
||||
install("extension1@1.0")
|
||||
install("extension1@2.0")
|
||||
install("extension2@1.0")
|
||||
viewpath = str(tmpdir.mkdir("view"))
|
||||
view("symlink", viewpath, "extension1@1.0")
|
||||
activate("extension1@2.0")
|
||||
activate("extension2@1.0")
|
||||
all_installed = extensions("--show", "installed", "extendee")
|
||||
assert "extension1@1.0" in all_installed
|
||||
assert "extension1@2.0" in all_installed
|
||||
assert "extension2@1.0" in all_installed
|
||||
global_activated = extensions("--show", "activated", "extendee")
|
||||
assert "extension1@1.0" not in global_activated
|
||||
assert "extension1@2.0" in global_activated
|
||||
assert "extension2@1.0" in global_activated
|
||||
view_activated = extensions("--show", "activated", "-v", viewpath, "extendee")
|
||||
assert "extension1@1.0" in view_activated
|
||||
assert "extension1@2.0" not in view_activated
|
||||
assert "extension2@1.0" not in view_activated
|
||||
assert os.path.exists(os.path.join(viewpath, "bin", "extension1"))
|
||||
assert not os.path.exists(os.path.join(viewpath, "bin", "extension2"))
|
||||
|
||||
|
||||
def test_view_extendee_with_global_activations(
|
||||
tmpdir, mock_packages, mock_archive, mock_fetch, config, install_mockery
|
||||
):
|
||||
install("extendee")
|
||||
install("extension1@1.0")
|
||||
install("extension1@2.0")
|
||||
install("extension2@1.0")
|
||||
viewpath = str(tmpdir.mkdir("view"))
|
||||
activate("extension1@2.0")
|
||||
output = view("symlink", viewpath, "extension1@1.0")
|
||||
assert "Error: Globally activated extensions cannot be used" in output
|
||||
|
||||
|
||||
def test_view_fails_with_missing_projections_file(tmpdir):
|
||||
viewpath = str(tmpdir.mkdir("view"))
|
||||
projection_file = os.path.join(str(tmpdir), "nonexistent")
|
||||
|
|
|
|||
|
|
@ -84,12 +84,6 @@ def test_inheritance_of_patches(self):
|
|||
# Will error if inheritor package cannot find inherited patch files
|
||||
s.concretize()
|
||||
|
||||
def test_dependency_extensions(self):
|
||||
s = Spec("extension2")
|
||||
s.concretize()
|
||||
deps = set(x.name for x in s.package.dependency_activations())
|
||||
assert deps == set(["extension1"])
|
||||
|
||||
def test_import_class_from_package(self):
|
||||
from spack.pkg.builtin.mock.mpich import Mpich # noqa: F401
|
||||
|
||||
|
|
|
|||
|
|
@ -1,402 +0,0 @@
|
|||
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
|
||||
# Spack Project Developers. See the top-level COPYRIGHT file for details.
|
||||
#
|
||||
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
|
||||
|
||||
"""This includes tests for customized activation logic for specific packages
|
||||
(e.g. python and perl).
|
||||
"""
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
from llnl.util.link_tree import MergeConflictError
|
||||
|
||||
import spack.package_base
|
||||
import spack.spec
|
||||
from spack.directory_layout import DirectoryLayout
|
||||
from spack.filesystem_view import YamlFilesystemView
|
||||
|
||||
pytestmark = pytest.mark.skipif(
|
||||
sys.platform == "win32",
|
||||
reason="Python activation not currently supported on Windows",
|
||||
)
|
||||
|
||||
|
||||
def create_ext_pkg(name, prefix, extendee_spec, monkeypatch):
|
||||
ext_spec = spack.spec.Spec(name)
|
||||
ext_spec._concrete = True
|
||||
|
||||
ext_spec.package.spec.prefix = prefix
|
||||
ext_pkg = ext_spec.package
|
||||
|
||||
# temporarily override extendee_spec property on the package
|
||||
monkeypatch.setattr(ext_pkg.__class__, "extendee_spec", extendee_spec)
|
||||
|
||||
return ext_pkg
|
||||
|
||||
|
||||
def create_python_ext_pkg(name, prefix, python_spec, monkeypatch, namespace=None):
|
||||
ext_pkg = create_ext_pkg(name, prefix, python_spec, monkeypatch)
|
||||
ext_pkg.py_namespace = namespace
|
||||
return ext_pkg
|
||||
|
||||
|
||||
def create_dir_structure(tmpdir, dir_structure):
|
||||
for fname, children in dir_structure.items():
|
||||
tmpdir.ensure(fname, dir=fname.endswith("/"))
|
||||
if children:
|
||||
create_dir_structure(tmpdir.join(fname), children)
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def builtin_and_mock_packages():
|
||||
# These tests use mock_repo packages to test functionality of builtin
|
||||
# packages for python and perl. To test this we put the mock repo at lower
|
||||
# precedence than the builtin repo, so we test builtin.perl against
|
||||
# builtin.mock.perl-extension.
|
||||
repo_dirs = [spack.paths.packages_path, spack.paths.mock_packages_path]
|
||||
with spack.repo.use_repositories(*repo_dirs):
|
||||
yield
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def python_and_extension_dirs(tmpdir, builtin_and_mock_packages):
|
||||
python_dirs = {"bin/": {"python": None}, "lib/": {"python2.7/": {"site-packages/": None}}}
|
||||
|
||||
python_name = "python"
|
||||
python_prefix = tmpdir.join(python_name)
|
||||
create_dir_structure(python_prefix, python_dirs)
|
||||
|
||||
python_spec = spack.spec.Spec("python@2.7.12")
|
||||
python_spec._concrete = True
|
||||
python_spec.package.spec.prefix = str(python_prefix)
|
||||
|
||||
ext_dirs = {
|
||||
"bin/": {"py-ext-tool": None},
|
||||
"lib/": {"python2.7/": {"site-packages/": {"py-extension1/": {"sample.py": None}}}},
|
||||
}
|
||||
|
||||
ext_name = "py-extension1"
|
||||
ext_prefix = tmpdir.join(ext_name)
|
||||
create_dir_structure(ext_prefix, ext_dirs)
|
||||
|
||||
easy_install_location = "lib/python2.7/site-packages/easy-install.pth"
|
||||
with open(str(ext_prefix.join(easy_install_location)), "w") as f:
|
||||
f.write(
|
||||
"""path/to/ext1.egg
|
||||
path/to/setuptools.egg"""
|
||||
)
|
||||
|
||||
return str(python_prefix), str(ext_prefix)
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def namespace_extensions(tmpdir, builtin_and_mock_packages):
|
||||
ext1_dirs = {
|
||||
"bin/": {"py-ext-tool1": None},
|
||||
"lib/": {
|
||||
"python2.7/": {
|
||||
"site-packages/": {
|
||||
"examplenamespace/": {"__init__.py": None, "ext1_sample.py": None}
|
||||
}
|
||||
}
|
||||
},
|
||||
}
|
||||
|
||||
ext2_dirs = {
|
||||
"bin/": {"py-ext-tool2": None},
|
||||
"lib/": {
|
||||
"python2.7/": {
|
||||
"site-packages/": {
|
||||
"examplenamespace/": {"__init__.py": None, "ext2_sample.py": None}
|
||||
}
|
||||
}
|
||||
},
|
||||
}
|
||||
|
||||
ext1_name = "py-extension1"
|
||||
ext1_prefix = tmpdir.join(ext1_name)
|
||||
create_dir_structure(ext1_prefix, ext1_dirs)
|
||||
|
||||
ext2_name = "py-extension2"
|
||||
ext2_prefix = tmpdir.join(ext2_name)
|
||||
create_dir_structure(ext2_prefix, ext2_dirs)
|
||||
|
||||
return str(ext1_prefix), str(ext2_prefix), "examplenamespace"
|
||||
|
||||
|
||||
def test_python_activation_with_files(
|
||||
tmpdir, python_and_extension_dirs, monkeypatch, builtin_and_mock_packages
|
||||
):
|
||||
python_prefix, ext_prefix = python_and_extension_dirs
|
||||
|
||||
python_spec = spack.spec.Spec("python@2.7.12")
|
||||
python_spec._concrete = True
|
||||
python_spec.package.spec.prefix = python_prefix
|
||||
|
||||
ext_pkg = create_python_ext_pkg("py-extension1", ext_prefix, python_spec, monkeypatch)
|
||||
|
||||
python_pkg = python_spec.package
|
||||
python_pkg.activate(ext_pkg, python_pkg.view())
|
||||
|
||||
assert os.path.exists(os.path.join(python_prefix, "bin/py-ext-tool"))
|
||||
|
||||
easy_install_location = "lib/python2.7/site-packages/easy-install.pth"
|
||||
with open(os.path.join(python_prefix, easy_install_location), "r") as f:
|
||||
easy_install_contents = f.read()
|
||||
|
||||
assert "ext1.egg" in easy_install_contents
|
||||
assert "setuptools.egg" not in easy_install_contents
|
||||
|
||||
|
||||
def test_python_activation_view(
|
||||
tmpdir, python_and_extension_dirs, builtin_and_mock_packages, monkeypatch
|
||||
):
|
||||
python_prefix, ext_prefix = python_and_extension_dirs
|
||||
|
||||
python_spec = spack.spec.Spec("python@2.7.12")
|
||||
python_spec._concrete = True
|
||||
python_spec.package.spec.prefix = python_prefix
|
||||
|
||||
ext_pkg = create_python_ext_pkg("py-extension1", ext_prefix, python_spec, monkeypatch)
|
||||
|
||||
view_dir = str(tmpdir.join("view"))
|
||||
layout = DirectoryLayout(view_dir)
|
||||
view = YamlFilesystemView(view_dir, layout)
|
||||
|
||||
python_pkg = python_spec.package
|
||||
python_pkg.activate(ext_pkg, view)
|
||||
|
||||
assert not os.path.exists(os.path.join(python_prefix, "bin/py-ext-tool"))
|
||||
|
||||
assert os.path.exists(os.path.join(view_dir, "bin/py-ext-tool"))
|
||||
|
||||
|
||||
def test_python_ignore_namespace_init_conflict(
|
||||
tmpdir, namespace_extensions, builtin_and_mock_packages, monkeypatch
|
||||
):
|
||||
"""Test the view update logic in PythonPackage ignores conflicting
|
||||
instances of __init__ for packages which are in the same namespace.
|
||||
"""
|
||||
ext1_prefix, ext2_prefix, py_namespace = namespace_extensions
|
||||
|
||||
python_spec = spack.spec.Spec("python@2.7.12")
|
||||
python_spec._concrete = True
|
||||
|
||||
ext1_pkg = create_python_ext_pkg(
|
||||
"py-extension1", ext1_prefix, python_spec, monkeypatch, py_namespace
|
||||
)
|
||||
ext2_pkg = create_python_ext_pkg(
|
||||
"py-extension2", ext2_prefix, python_spec, monkeypatch, py_namespace
|
||||
)
|
||||
|
||||
view_dir = str(tmpdir.join("view"))
|
||||
layout = DirectoryLayout(view_dir)
|
||||
view = YamlFilesystemView(view_dir, layout)
|
||||
|
||||
python_pkg = python_spec.package
|
||||
python_pkg.activate(ext1_pkg, view)
|
||||
# Normally handled by Package.do_activate, but here we activate directly
|
||||
view.extensions_layout.add_extension(python_spec, ext1_pkg.spec)
|
||||
python_pkg.activate(ext2_pkg, view)
|
||||
|
||||
f1 = "lib/python2.7/site-packages/examplenamespace/ext1_sample.py"
|
||||
f2 = "lib/python2.7/site-packages/examplenamespace/ext2_sample.py"
|
||||
init_file = "lib/python2.7/site-packages/examplenamespace/__init__.py"
|
||||
|
||||
assert os.path.exists(os.path.join(view_dir, f1))
|
||||
assert os.path.exists(os.path.join(view_dir, f2))
|
||||
assert os.path.exists(os.path.join(view_dir, init_file))
|
||||
|
||||
|
||||
def test_python_keep_namespace_init(
|
||||
tmpdir, namespace_extensions, builtin_and_mock_packages, monkeypatch
|
||||
):
|
||||
"""Test the view update logic in PythonPackage keeps the namespace
|
||||
__init__ file as long as one package in the namespace still
|
||||
exists.
|
||||
"""
|
||||
ext1_prefix, ext2_prefix, py_namespace = namespace_extensions
|
||||
|
||||
python_spec = spack.spec.Spec("python@2.7.12")
|
||||
python_spec._concrete = True
|
||||
|
||||
ext1_pkg = create_python_ext_pkg(
|
||||
"py-extension1", ext1_prefix, python_spec, monkeypatch, py_namespace
|
||||
)
|
||||
ext2_pkg = create_python_ext_pkg(
|
||||
"py-extension2", ext2_prefix, python_spec, monkeypatch, py_namespace
|
||||
)
|
||||
|
||||
view_dir = str(tmpdir.join("view"))
|
||||
layout = DirectoryLayout(view_dir)
|
||||
view = YamlFilesystemView(view_dir, layout)
|
||||
|
||||
python_pkg = python_spec.package
|
||||
python_pkg.activate(ext1_pkg, view)
|
||||
# Normally handled by Package.do_activate, but here we activate directly
|
||||
view.extensions_layout.add_extension(python_spec, ext1_pkg.spec)
|
||||
python_pkg.activate(ext2_pkg, view)
|
||||
view.extensions_layout.add_extension(python_spec, ext2_pkg.spec)
|
||||
|
||||
f1 = "lib/python2.7/site-packages/examplenamespace/ext1_sample.py"
|
||||
init_file = "lib/python2.7/site-packages/examplenamespace/__init__.py"
|
||||
|
||||
python_pkg.deactivate(ext1_pkg, view)
|
||||
view.extensions_layout.remove_extension(python_spec, ext1_pkg.spec)
|
||||
|
||||
assert not os.path.exists(os.path.join(view_dir, f1))
|
||||
assert os.path.exists(os.path.join(view_dir, init_file))
|
||||
|
||||
python_pkg.deactivate(ext2_pkg, view)
|
||||
view.extensions_layout.remove_extension(python_spec, ext2_pkg.spec)
|
||||
|
||||
assert not os.path.exists(os.path.join(view_dir, init_file))
|
||||
|
||||
|
||||
def test_python_namespace_conflict(
|
||||
tmpdir, namespace_extensions, monkeypatch, builtin_and_mock_packages
|
||||
):
|
||||
"""Test the view update logic in PythonPackage reports an error when two
|
||||
python extensions with different namespaces have a conflicting __init__
|
||||
file.
|
||||
"""
|
||||
ext1_prefix, ext2_prefix, py_namespace = namespace_extensions
|
||||
other_namespace = py_namespace + "other"
|
||||
|
||||
python_spec = spack.spec.Spec("python@2.7.12")
|
||||
python_spec._concrete = True
|
||||
|
||||
ext1_pkg = create_python_ext_pkg(
|
||||
"py-extension1", ext1_prefix, python_spec, monkeypatch, py_namespace
|
||||
)
|
||||
ext2_pkg = create_python_ext_pkg(
|
||||
"py-extension2", ext2_prefix, python_spec, monkeypatch, other_namespace
|
||||
)
|
||||
|
||||
view_dir = str(tmpdir.join("view"))
|
||||
layout = DirectoryLayout(view_dir)
|
||||
view = YamlFilesystemView(view_dir, layout)
|
||||
|
||||
python_pkg = python_spec.package
|
||||
python_pkg.activate(ext1_pkg, view)
|
||||
view.extensions_layout.add_extension(python_spec, ext1_pkg.spec)
|
||||
with pytest.raises(MergeConflictError):
|
||||
python_pkg.activate(ext2_pkg, view)
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def perl_and_extension_dirs(tmpdir, builtin_and_mock_packages):
|
||||
perl_dirs = {
|
||||
"bin/": {"perl": None},
|
||||
"lib/": {"site_perl/": {"5.24.1/": {"x86_64-linux/": None}}},
|
||||
}
|
||||
|
||||
perl_name = "perl"
|
||||
perl_prefix = tmpdir.join(perl_name)
|
||||
create_dir_structure(perl_prefix, perl_dirs)
|
||||
|
||||
perl_spec = spack.spec.Spec("perl@5.24.1")
|
||||
perl_spec._concrete = True
|
||||
perl_spec.package.spec.prefix = str(perl_prefix)
|
||||
|
||||
ext_dirs = {
|
||||
"bin/": {"perl-ext-tool": None},
|
||||
"lib/": {"site_perl/": {"5.24.1/": {"x86_64-linux/": {"TestExt/": {}}}}},
|
||||
}
|
||||
|
||||
ext_name = "perl-extension"
|
||||
ext_prefix = tmpdir.join(ext_name)
|
||||
create_dir_structure(ext_prefix, ext_dirs)
|
||||
|
||||
return str(perl_prefix), str(ext_prefix)
|
||||
|
||||
|
||||
def test_perl_activation(tmpdir, builtin_and_mock_packages, monkeypatch):
|
||||
# Note the lib directory is based partly on the perl version
|
||||
perl_spec = spack.spec.Spec("perl@5.24.1")
|
||||
perl_spec._concrete = True
|
||||
|
||||
perl_name = "perl"
|
||||
tmpdir.ensure(perl_name, dir=True)
|
||||
|
||||
perl_prefix = str(tmpdir.join(perl_name))
|
||||
# Set the prefix on the package's spec reference because that is a copy of
|
||||
# the original spec
|
||||
perl_spec.package.spec.prefix = perl_prefix
|
||||
|
||||
ext_name = "perl-extension"
|
||||
tmpdir.ensure(ext_name, dir=True)
|
||||
ext_pkg = create_ext_pkg(ext_name, str(tmpdir.join(ext_name)), perl_spec, monkeypatch)
|
||||
|
||||
perl_pkg = perl_spec.package
|
||||
perl_pkg.activate(ext_pkg, perl_pkg.view())
|
||||
|
||||
|
||||
def test_perl_activation_with_files(
|
||||
tmpdir, perl_and_extension_dirs, monkeypatch, builtin_and_mock_packages
|
||||
):
|
||||
perl_prefix, ext_prefix = perl_and_extension_dirs
|
||||
|
||||
perl_spec = spack.spec.Spec("perl@5.24.1")
|
||||
perl_spec._concrete = True
|
||||
perl_spec.package.spec.prefix = perl_prefix
|
||||
|
||||
ext_pkg = create_ext_pkg("perl-extension", ext_prefix, perl_spec, monkeypatch)
|
||||
|
||||
perl_pkg = perl_spec.package
|
||||
perl_pkg.activate(ext_pkg, perl_pkg.view())
|
||||
|
||||
assert os.path.exists(os.path.join(perl_prefix, "bin/perl-ext-tool"))
|
||||
|
||||
|
||||
def test_perl_activation_view(
|
||||
tmpdir, perl_and_extension_dirs, monkeypatch, builtin_and_mock_packages
|
||||
):
|
||||
perl_prefix, ext_prefix = perl_and_extension_dirs
|
||||
|
||||
perl_spec = spack.spec.Spec("perl@5.24.1")
|
||||
perl_spec._concrete = True
|
||||
perl_spec.package.spec.prefix = perl_prefix
|
||||
|
||||
ext_pkg = create_ext_pkg("perl-extension", ext_prefix, perl_spec, monkeypatch)
|
||||
|
||||
view_dir = str(tmpdir.join("view"))
|
||||
layout = DirectoryLayout(view_dir)
|
||||
view = YamlFilesystemView(view_dir, layout)
|
||||
|
||||
perl_pkg = perl_spec.package
|
||||
perl_pkg.activate(ext_pkg, view)
|
||||
|
||||
assert not os.path.exists(os.path.join(perl_prefix, "bin/perl-ext-tool"))
|
||||
|
||||
assert os.path.exists(os.path.join(view_dir, "bin/perl-ext-tool"))
|
||||
|
||||
|
||||
def test_is_activated_upstream_extendee(tmpdir, builtin_and_mock_packages, monkeypatch):
|
||||
"""When an extendee is installed upstream, make sure that the extension
|
||||
spec is never considered to be globally activated for it.
|
||||
"""
|
||||
extendee_spec = spack.spec.Spec("python")
|
||||
extendee_spec._concrete = True
|
||||
|
||||
python_name = "python"
|
||||
tmpdir.ensure(python_name, dir=True)
|
||||
|
||||
python_prefix = str(tmpdir.join(python_name))
|
||||
# Set the prefix on the package's spec reference because that is a copy of
|
||||
# the original spec
|
||||
extendee_spec.package.spec.prefix = python_prefix
|
||||
monkeypatch.setattr(extendee_spec.__class__, "installed_upstream", True)
|
||||
|
||||
ext_name = "py-extension1"
|
||||
tmpdir.ensure(ext_name, dir=True)
|
||||
ext_pkg = create_ext_pkg(ext_name, str(tmpdir.join(ext_name)), extendee_spec, monkeypatch)
|
||||
|
||||
# The view should not be checked at all if the extendee is installed
|
||||
# upstream, so use 'None' here
|
||||
mock_view = None
|
||||
assert not ext_pkg.is_activated(mock_view)
|
||||
|
|
@ -3,7 +3,6 @@
|
|||
#
|
||||
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
|
@ -13,26 +12,6 @@
|
|||
from spack.spec import Spec
|
||||
|
||||
|
||||
@pytest.mark.skipif(sys.platform == "win32", reason="Not supported on Windows (yet)")
|
||||
def test_global_activation(install_mockery, mock_fetch):
|
||||
"""This test ensures that views which are maintained inside of an extendee
|
||||
package's prefix are maintained as expected and are compatible with
|
||||
global activations prior to #7152.
|
||||
"""
|
||||
spec = Spec("extension1").concretized()
|
||||
pkg = spec.package
|
||||
pkg.do_install()
|
||||
pkg.do_activate()
|
||||
|
||||
extendee_spec = spec["extendee"]
|
||||
extendee_pkg = spec["extendee"].package
|
||||
view = extendee_pkg.view()
|
||||
assert pkg.is_activated(view)
|
||||
|
||||
expected_path = os.path.join(extendee_spec.prefix, ".spack", "extensions.yaml")
|
||||
assert view.extensions_layout.extension_file_path(extendee_spec) == expected_path
|
||||
|
||||
|
||||
@pytest.mark.skipif(sys.platform == "win32", reason="Not supported on Windows (yet)")
|
||||
def test_remove_extensions_ordered(install_mockery, mock_fetch, tmpdir):
|
||||
view_dir = str(tmpdir.join("view"))
|
||||
|
|
|
|||
|
|
@ -162,39 +162,12 @@ def check_spec_manifest(spec):
|
|||
results.add_error(prefix, "manifest corrupted")
|
||||
return results
|
||||
|
||||
# Get extensions active in spec
|
||||
view = spack.filesystem_view.YamlFilesystemView(prefix, spack.store.layout)
|
||||
active_exts = view.extensions_layout.extension_map(spec).values()
|
||||
ext_file = ""
|
||||
if active_exts:
|
||||
# No point checking contents of this file as it is the only source of
|
||||
# truth for that information.
|
||||
ext_file = view.extensions_layout.extension_file_path(spec)
|
||||
|
||||
def is_extension_artifact(p):
|
||||
if os.path.islink(p):
|
||||
if any(os.readlink(p).startswith(e.prefix) for e in active_exts):
|
||||
# This file is linked in by an extension. Belongs to extension
|
||||
return True
|
||||
elif os.path.isdir(p) and p not in manifest:
|
||||
if all(is_extension_artifact(os.path.join(p, f)) for f in os.listdir(p)):
|
||||
return True
|
||||
return False
|
||||
|
||||
for root, dirs, files in os.walk(prefix):
|
||||
for entry in list(dirs + files):
|
||||
path = os.path.join(root, entry)
|
||||
|
||||
# Do not check links from prefix to active extension
|
||||
# TODO: make this stricter for non-linux systems that use symlink
|
||||
# permissions
|
||||
# Do not check directories that only exist for extensions
|
||||
if is_extension_artifact(path):
|
||||
continue
|
||||
|
||||
# Do not check manifest file. Can't store your own hash
|
||||
# Nothing to check for ext_file
|
||||
if path == manifest_file or path == ext_file:
|
||||
if path == manifest_file:
|
||||
continue
|
||||
|
||||
data = manifest.pop(path, {})
|
||||
|
|
|
|||
|
|
@ -337,16 +337,7 @@ _spack() {
|
|||
then
|
||||
SPACK_COMPREPLY="-h --help -H --all-help --color -c --config -C --config-scope -d --debug --timestamp --pdb -e --env -D --env-dir -E --no-env --use-env-repo -k --insecure -l --enable-locks -L --disable-locks -m --mock -b --bootstrap -p --profile --sorted-profile --lines -v --verbose --stacktrace --backtrace -V --version --print-shell-vars"
|
||||
else
|
||||
SPACK_COMPREPLY="activate add arch audit blame bootstrap build-env buildcache cd change checksum ci clean clone commands compiler compilers concretize config containerize create deactivate debug dependencies dependents deprecate dev-build develop diff docs edit env extensions external fetch find gc gpg graph help info install license list load location log-parse maintainers make-installer mark mirror module patch pkg providers pydoc python reindex remove rm repo resource restage solve spec stage style tags test test-env tutorial undevelop uninstall unit-test unload url verify versions view"
|
||||
fi
|
||||
}
|
||||
|
||||
_spack_activate() {
|
||||
if $list_options
|
||||
then
|
||||
SPACK_COMPREPLY="-h --help -f --force -v --view"
|
||||
else
|
||||
_installed_packages
|
||||
SPACK_COMPREPLY="add arch audit blame bootstrap build-env buildcache cd change checksum ci clean clone commands compiler compilers concretize config containerize create debug dependencies dependents deprecate dev-build develop diff docs edit env extensions external fetch find gc gpg graph help info install license list load location log-parse maintainers make-installer mark mirror module patch pkg providers pydoc python reindex remove rm repo resource restage solve spec stage style tags test test-env tutorial undevelop uninstall unit-test unload url verify versions view"
|
||||
fi
|
||||
}
|
||||
|
||||
|
|
@ -838,15 +829,6 @@ _spack_create() {
|
|||
fi
|
||||
}
|
||||
|
||||
_spack_deactivate() {
|
||||
if $list_options
|
||||
then
|
||||
SPACK_COMPREPLY="-h --help -f --force -v --view -a --all"
|
||||
else
|
||||
_installed_packages
|
||||
fi
|
||||
}
|
||||
|
||||
_spack_debug() {
|
||||
if $list_options
|
||||
then
|
||||
|
|
@ -1039,7 +1021,7 @@ _spack_env_depfile() {
|
|||
_spack_extensions() {
|
||||
if $list_options
|
||||
then
|
||||
SPACK_COMPREPLY="-h --help -l --long -L --very-long -d --deps -p --paths -s --show -v --view"
|
||||
SPACK_COMPREPLY="-h --help -l --long -L --very-long -d --deps -p --paths -s --show"
|
||||
else
|
||||
_extensions
|
||||
fi
|
||||
|
|
|
|||
|
|
@ -482,28 +482,6 @@ def perl_ignore(self, ext_pkg, args):
|
|||
|
||||
return match_predicate(ignore_arg, patterns)
|
||||
|
||||
def activate(self, ext_pkg, view, **args):
|
||||
ignore = self.perl_ignore(ext_pkg, args)
|
||||
args.update(ignore=ignore)
|
||||
|
||||
super(Perl, self).activate(ext_pkg, view, **args)
|
||||
|
||||
extensions_layout = view.extensions_layout
|
||||
exts = extensions_layout.extension_map(self.spec)
|
||||
exts[ext_pkg.name] = ext_pkg.spec
|
||||
|
||||
def deactivate(self, ext_pkg, view, **args):
|
||||
ignore = self.perl_ignore(ext_pkg, args)
|
||||
args.update(ignore=ignore)
|
||||
|
||||
super(Perl, self).deactivate(ext_pkg, view, **args)
|
||||
|
||||
extensions_layout = view.extensions_layout
|
||||
exts = extensions_layout.extension_map(self.spec)
|
||||
# Make deactivate idempotent
|
||||
if ext_pkg.name in exts:
|
||||
del exts[ext_pkg.name]
|
||||
|
||||
@property
|
||||
def command(self):
|
||||
"""Returns the Perl command, which may vary depending on the version
|
||||
|
|
|
|||
|
|
@ -18,7 +18,7 @@
|
|||
is_nonsymlink_exe_with_shebang,
|
||||
path_contains_subdirectory,
|
||||
)
|
||||
from llnl.util.lang import dedupe, match_predicate
|
||||
from llnl.util.lang import dedupe
|
||||
|
||||
from spack.build_environment import dso_suffix, stat_suffix
|
||||
from spack.package import *
|
||||
|
|
@ -1401,10 +1401,6 @@ def include(self):
|
|||
return path.replace(prefix, "")
|
||||
return os.path.join("include", "python{}".format(self.version.up_to(2)))
|
||||
|
||||
@property
|
||||
def easy_install_file(self):
|
||||
return join_path(self.purelib, "easy-install.pth")
|
||||
|
||||
def setup_run_environment(self, env):
|
||||
env.prepend_path("CPATH", os.pathsep.join(self.spec["python"].headers.directories))
|
||||
|
||||
|
|
@ -1526,108 +1522,6 @@ def setup_dependent_package(self, module, dependent_spec):
|
|||
mkdirp(module.python_platlib)
|
||||
mkdirp(module.python_purelib)
|
||||
|
||||
# ========================================================================
|
||||
# Handle specifics of activating and deactivating python modules.
|
||||
# ========================================================================
|
||||
|
||||
def python_ignore(self, ext_pkg, args):
|
||||
"""Add some ignore files to activate/deactivate args."""
|
||||
ignore_arg = args.get("ignore", lambda f: False)
|
||||
|
||||
# Always ignore easy-install.pth, as it needs to be merged.
|
||||
patterns = [r"(site|dist)-packages/easy-install\.pth$"]
|
||||
|
||||
# Ignore pieces of setuptools installed by other packages.
|
||||
# Must include directory name or it will remove all site*.py files.
|
||||
if ext_pkg.name != "py-setuptools":
|
||||
patterns.extend(
|
||||
[
|
||||
r"bin/easy_install[^/]*$",
|
||||
r"(site|dist)-packages/setuptools[^/]*\.egg$",
|
||||
r"(site|dist)-packages/setuptools\.pth$",
|
||||
r"(site|dist)-packages/site[^/]*\.pyc?$",
|
||||
r"(site|dist)-packages/__pycache__/site[^/]*\.pyc?$",
|
||||
]
|
||||
)
|
||||
if ext_pkg.name != "py-pygments":
|
||||
patterns.append(r"bin/pygmentize$")
|
||||
if ext_pkg.name != "py-numpy":
|
||||
patterns.append(r"bin/f2py[0-9.]*$")
|
||||
|
||||
return match_predicate(ignore_arg, patterns)
|
||||
|
||||
def write_easy_install_pth(self, exts, prefix=None):
|
||||
if not prefix:
|
||||
prefix = self.prefix
|
||||
|
||||
paths = []
|
||||
unique_paths = set()
|
||||
|
||||
for ext in sorted(exts.values()):
|
||||
easy_pth = join_path(ext.prefix, self.easy_install_file)
|
||||
|
||||
if not os.path.isfile(easy_pth):
|
||||
continue
|
||||
|
||||
with open(easy_pth) as f:
|
||||
for line in f:
|
||||
line = line.rstrip()
|
||||
|
||||
# Skip lines matching these criteria
|
||||
if not line:
|
||||
continue
|
||||
if re.search(r"^(import|#)", line):
|
||||
continue
|
||||
if ext.name != "py-setuptools" and re.search(r"setuptools.*egg$", line):
|
||||
continue
|
||||
|
||||
if line not in unique_paths:
|
||||
unique_paths.add(line)
|
||||
paths.append(line)
|
||||
|
||||
main_pth = join_path(prefix, self.easy_install_file)
|
||||
|
||||
if not paths:
|
||||
if os.path.isfile(main_pth):
|
||||
os.remove(main_pth)
|
||||
|
||||
else:
|
||||
with open(main_pth, "w") as f:
|
||||
f.write("import sys; sys.__plen = len(sys.path)\n")
|
||||
for path in paths:
|
||||
f.write("{0}\n".format(path))
|
||||
f.write(
|
||||
"import sys; new=sys.path[sys.__plen:]; "
|
||||
"del sys.path[sys.__plen:]; "
|
||||
"p=getattr(sys,'__egginsert',0); "
|
||||
"sys.path[p:p]=new; "
|
||||
"sys.__egginsert = p+len(new)\n"
|
||||
)
|
||||
|
||||
def activate(self, ext_pkg, view, **args):
|
||||
ignore = self.python_ignore(ext_pkg, args)
|
||||
args.update(ignore=ignore)
|
||||
|
||||
super(Python, self).activate(ext_pkg, view, **args)
|
||||
|
||||
extensions_layout = view.extensions_layout
|
||||
exts = extensions_layout.extension_map(self.spec)
|
||||
exts[ext_pkg.name] = ext_pkg.spec
|
||||
|
||||
self.write_easy_install_pth(exts, prefix=view.get_projection_for_spec(self.spec))
|
||||
|
||||
def deactivate(self, ext_pkg, view, **args):
|
||||
args.update(ignore=self.python_ignore(ext_pkg, args))
|
||||
|
||||
super(Python, self).deactivate(ext_pkg, view, **args)
|
||||
|
||||
extensions_layout = view.extensions_layout
|
||||
exts = extensions_layout.extension_map(self.spec)
|
||||
# Make deactivate idempotent
|
||||
if ext_pkg.name in exts:
|
||||
del exts[ext_pkg.name]
|
||||
self.write_easy_install_pth(exts, prefix=view.get_projection_for_spec(self.spec))
|
||||
|
||||
def add_files_to_view(self, view, merge_map, skip_if_exists=True):
|
||||
bin_dir = self.spec.prefix.bin if sys.platform != "win32" else self.spec.prefix
|
||||
for src, dst in merge_map.items():
|
||||
|
|
|
|||
Loading…
Reference in a new issue