hlrs-software-stack/spack
7
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Deprecate spack bootstrap trust/untrust (#33600)

Browse source 
* Deprecate spack bootstrap trust/untrust
* Update CI
* Update tests
This commit is contained in:
Massimiliano Culpo 2022-10-29 21:24:26 +02:00 committed by GitHub
parent 29f1e8395c
commit 7e645f54c5
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
8 changed files with 113 additions and 70 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
.github/workflows/bootstrap-test.sh vendored
View file

@ -1,7 +1,7 @@
#!/bin/bash
set -ex
source share/spack/setup-env.sh
$PYTHON bin/spack bootstrap untrust spack-install
$PYTHON bin/spack bootstrap disable spack-install
$PYTHON bin/spack -d solve zlib
tree $BOOTSTRAP/store
exit 0

16
.github/workflows/bootstrap.yml vendored
View file

@ -42,7 +42,7 @@ jobs:
shell: runuser -u spack-test -- bash {0}
run: |
source share/spack/setup-env.sh
spack bootstrap untrust github-actions-v0.3
spack bootstrap disable github-actions-v0.3
spack external find cmake bison
spack -d solve zlib
tree ~/.spack/bootstrap/store/
@ -79,7 +79,7 @@ jobs:
shell: runuser -u spack-test -- bash {0}
run: |
source share/spack/setup-env.sh
spack bootstrap untrust github-actions-v0.3
spack bootstrap disable github-actions-v0.3
spack external find cmake bison
spack -d solve zlib
tree ~/.spack/bootstrap/store/
@ -143,7 +143,7 @@ jobs:
- name: Bootstrap clingo
run: |
source share/spack/setup-env.sh
spack bootstrap untrust github-actions-v0.3
spack bootstrap disable github-actions-v0.3
spack external find cmake bison
spack -d solve zlib
tree ~/.spack/bootstrap/store/
@ -160,7 +160,7 @@ jobs:
run: |
source share/spack/setup-env.sh
export PATH=/usr/local/opt/bison@2.7/bin:$PATH
spack bootstrap untrust github-actions-v0.3
spack bootstrap disable github-actions-v0.3
spack external find --not-buildable cmake bison
spack -d solve zlib
tree ~/.spack/bootstrap/store/
@ -261,7 +261,7 @@ jobs:
shell: runuser -u spack-test -- bash {0}
run: |
source share/spack/setup-env.sh
spack bootstrap untrust spack-install
spack bootstrap disable spack-install
spack -d gpg list
tree ~/.spack/bootstrap/store/
@ -298,7 +298,7 @@ jobs:
run: |
source share/spack/setup-env.sh
spack solve zlib
spack bootstrap untrust github-actions-v0.3
spack bootstrap disable github-actions-v0.3
spack -d gpg list
tree ~/.spack/bootstrap/store/
@ -315,7 +315,7 @@ jobs:
- name: Bootstrap GnuPG
run: |
source share/spack/setup-env.sh
spack bootstrap untrust spack-install
spack bootstrap disable spack-install
spack -d gpg list
tree ~/.spack/bootstrap/store/
@ -333,7 +333,7 @@ jobs:
run: |
source share/spack/setup-env.sh
spack solve zlib
spack bootstrap untrust github-actions-v0.3
spack bootstrap disable github-actions-v0.3
spack -d gpg list
tree ~/.spack/bootstrap/store/

4
.github/workflows/unit_tests.yaml vendored
View file

@ -78,7 +78,7 @@ jobs:
SPACK_PYTHON: python
run: |
. share/spack/setup-env.sh
spack bootstrap untrust spack-install
spack bootstrap disable spack-install
spack -v solve zlib
- name: Run unit tests
env:
@ -215,7 +215,7 @@ jobs:
git --version
. .github/workflows/setup_git.sh
. share/spack/setup-env.sh
$(which spack) bootstrap untrust spack-install
$(which spack) bootstrap disable spack-install
$(which spack) solve zlib
common_args=(--dist loadfile --tx '4*popen//python=./bin/spack-tmpconfig python -u ./bin/spack python' -x)
$(which spack) unit-test --cov --cov-config=pyproject.toml "${common_args[@]}"

33
lib/spack/docs/bootstrapping.rst
View file

@ -15,15 +15,13 @@ is an entire command dedicated to the management of every aspect of bootstrappin
.. command-output:: spack bootstrap --help
The first thing to know to understand bootstrapping in Spack is that each of
Spack's dependencies is bootstrapped lazily; i.e. the first time it is needed and
can't be found. You can readily check if any prerequisite for using Spack
is missing by running:
Spack is configured to bootstrap its dependencies lazily by default; i.e. the first time they are needed and
can't be found. You can readily check if any prerequisite for using Spack is missing by running:
.. code-block:: console
% spack bootstrap status
Spack v0.17.1 - python@3.8
Spack v0.19.0 - python@3.8
[FAIL] Core Functionalities
[B] MISSING "clingo": required to concretize specs
@ -48,6 +46,21 @@ they can be bootstrapped. Running a command that concretize a spec, like:
triggers the bootstrapping of clingo from pre-built binaries as expected.
Users can also bootstrap all the dependencies needed by Spack in a single command, which
might be useful to setup containers or other similar environments:
.. code-block:: console
$ spack bootstrap now
==> Bootstrapping clingo from pre-built binaries
==> Fetching https://mirror.spack.io/bootstrap/github-actions/v0.3/build_cache/linux-centos7-x86_64-gcc-10.2.1-clingo-bootstrap-spack-shqedxgvjnhiwdcdrvjhbd73jaevv7wt.spec.json
==> Fetching https://mirror.spack.io/bootstrap/github-actions/v0.3/build_cache/linux-centos7-x86_64/gcc-10.2.1/clingo-bootstrap-spack/linux-centos7-x86_64-gcc-10.2.1-clingo-bootstrap-spack-shqedxgvjnhiwdcdrvjhbd73jaevv7wt.spack
==> Installing "clingo-bootstrap@spack%gcc@10.2.1~docs~ipo+python+static_libstdcpp build_type=Release arch=linux-centos7-x86_64" from a buildcache
==> Bootstrapping patchelf from pre-built binaries
==> Fetching https://mirror.spack.io/bootstrap/github-actions/v0.3/build_cache/linux-centos7-x86_64-gcc-10.2.1-patchelf-0.15.0-htk62k7efo2z22kh6kmhaselru7bfkuc.spec.json
==> Fetching https://mirror.spack.io/bootstrap/github-actions/v0.3/build_cache/linux-centos7-x86_64/gcc-10.2.1/patchelf-0.15.0/linux-centos7-x86_64-gcc-10.2.1-patchelf-0.15.0-htk62k7efo2z22kh6kmhaselru7bfkuc.spack
==> Installing "patchelf@0.15.0%gcc@10.2.1 ldflags="-static-libstdc++ -static-libgcc" arch=linux-centos7-x86_64" from a buildcache
-----------------------
The Bootstrapping store
-----------------------
@ -107,19 +120,19 @@ If need be, you can disable bootstrapping altogether by running:
in which case it's your responsibility to ensure Spack runs in an
environment where all its prerequisites are installed. You can
also configure Spack to skip certain bootstrapping methods by *untrusting*
them. For instance:
also configure Spack to skip certain bootstrapping methods by disabling
them specifically:
.. code-block:: console
% spack bootstrap untrust github-actions
==> "github-actions" is now untrusted and will not be used for bootstrapping
% spack bootstrap disable github-actions
==> "github-actions" is now disabled and will not be used for bootstrapping
tells Spack to skip trying to bootstrap from binaries. To add the "github-actions" method back you can:
.. code-block:: console
% spack bootstrap trust github-actions
% spack bootstrap enable github-actions
There is also an option to reset the bootstrapping configuration to Spack's defaults:

28
lib/spack/docs/getting_started.rst
View file

@ -177,35 +177,13 @@ Spack fall back to bootstrapping from sources:
.. code-block:: console
$ spack bootstrap untrust github-actions-v0.2
==> "github-actions-v0.2" is now untrusted and will not be used for bootstrapping
$ spack bootstrap disable github-actions-v0.3
==> "github-actions-v0.2" is now disabled and will not be used for bootstrapping
You can verify that the new settings are effective with:
.. code-block:: console
.. command-output:: spack bootstrap list
$ spack bootstrap list
Name: github-actions-v0.2 UNTRUSTED
Type: buildcache
Info:
url: https://mirror.spack.io/bootstrap/github-actions/v0.2
homepage: https://github.com/spack/spack-bootstrap-mirrors
releases: https://github.com/spack/spack-bootstrap-mirrors/releases
Description:
Buildcache generated from a public workflow using Github Actions.
The sha256 checksum of binaries is checked before installation.
[ ... ]
Name: spack-install TRUSTED
Type: install
Description:
Specs built from sources by Spack. May take a long time.
.. note::

56
lib/spack/spack/cmd/bootstrap.py
View file

@ -8,6 +8,7 @@
import platform
import shutil
import tempfile
import warnings
import llnl.util.filesystem
import llnl.util.tty
@ -92,9 +93,11 @@ def setup_parser(subparser):
enable = sp.add_parser("enable", help="enable bootstrapping")
_add_scope_option(enable)
enable.add_argument("name", help="name of the source to be enabled", nargs="?", default=None)
disable = sp.add_parser("disable", help="disable bootstrapping")
_add_scope_option(disable)
disable.add_argument("name", help="name of the source to be disabled", nargs="?", default=None)
reset = sp.add_parser("reset", help="reset bootstrapping configuration to Spack defaults")
spack.cmd.common.arguments.add_common_arguments(reset, ["yes_to_all"])
@ -108,11 +111,11 @@ def setup_parser(subparser):
list = sp.add_parser("list", help="list all the sources of software to bootstrap Spack")
_add_scope_option(list)
trust = sp.add_parser("trust", help="trust a bootstrapping source")
trust = sp.add_parser("trust", help="(DEPRECATED) trust a bootstrapping source")
_add_scope_option(trust)
trust.add_argument("name", help="name of the source to be trusted")
untrust = sp.add_parser("untrust", help="untrust a bootstrapping source")
untrust = sp.add_parser("untrust", help="(DEPRECATED) untrust a bootstrapping source")
_add_scope_option(untrust)
untrust.add_argument("name", help="name of the source to be untrusted")
@ -140,9 +143,21 @@ def setup_parser(subparser):
def _enable_or_disable(args):
# Set to True if we called "enable", otherwise set to false
value = args.subcommand == "enable"
if args.name is None:
# Set to True if we called "enable", otherwise set to false
old_value = spack.config.get("bootstrap:enable", scope=args.scope)
if old_value == value:
llnl.util.tty.msg("Bootstrapping is already {}d".format(args.subcommand))
else:
spack.config.set("bootstrap:enable", value, scope=args.scope)
llnl.util.tty.msg("Bootstrapping has been {}d".format(args.subcommand))
return
if value is True:
_trust(args)
else:
_untrust(args)
def _reset(args):
@ -173,6 +188,8 @@ def _reset(args):
if os.path.exists(bootstrap_yaml):
shutil.move(bootstrap_yaml, backup_file)
spack.config.config.clear_caches()
def _root(args):
if args.path:
@ -197,14 +214,15 @@ def fmt(header, content):
header_fmt = "@*b{{{0}:}} {1}"
color.cprint(header_fmt.format(header, content))
trust_str = "@*y{UNKNOWN}"
trust_str = "@*y{DISABLED}"
if trusted is True:
trust_str = "@*g{TRUSTED}"
trust_str = "@*g{ENABLED}"
elif trusted is False:
trust_str = "@*r{UNTRUSTED}"
trust_str = "@*r{DISABLED}"
fmt("Name", source["name"] + " " + trust_str)
print()
if trusted is True or args.verbose:
fmt(" Type", source["type"])
print()
@ -221,6 +239,16 @@ def fmt(header, content):
fmt(" Description", "".join(description_lines))
trusted = spack.config.get("bootstrap:trusted", {})
def sort_fn(x):
x_trust = trusted.get(x["name"], None)
if x_trust is True:
return 0
elif x_trust is None:
return 1
return 2
sources = sorted(sources, key=sort_fn)
for s in sources:
_print_method(s, trusted.get(s["name"], None))
@ -252,15 +280,27 @@ def _write_trust_state(args, value):
spack.config.add("bootstrap:trusted:{0}:{1}".format(name, str(value)), scope=scope)
def _deprecate_command(deprecated_cmd, suggested_cmd):
msg = (
"the 'spack bootstrap {} ...' command is deprecated and will be "
"removed in v0.20, use 'spack bootstrap {} ...' instead"
)
warnings.warn(msg.format(deprecated_cmd, suggested_cmd))
def _trust(args):
if args.subcommand == "trust":
_deprecate_command("trust", "enable")
_write_trust_state(args, value=True)
msg = '"{0}" is now trusted for bootstrapping'
msg = '"{0}" is now enabled for bootstrapping'
llnl.util.tty.msg(msg.format(args.name))
def _untrust(args):
if args.subcommand == "untrust":
_deprecate_command("untrust", "disable")
_write_trust_state(args, value=False)
msg = '"{0}" is now untrusted and will not be used for bootstrapping'
msg = '"{0}" is now disabled and will not be used for bootstrapping'
llnl.util.tty.msg(msg.format(args.name))

6
lib/spack/spack/test/cmd/bootstrap.py
View file

@ -109,7 +109,9 @@ def test_list_sources(capsys):
assert "No method available" in output
@pytest.mark.parametrize("command,value", [("trust", True), ("untrust", False)])
@pytest.mark.parametrize(
"command,value", [("enable", True), ("disable", False), ("trust", True), ("untrust", False)]
)
def test_trust_or_untrust_sources(mutable_config, command, value):
key = "bootstrap:trusted:github-actions"
trusted = spack.config.get(key, default=None)
@ -135,7 +137,7 @@ def test_trust_or_untrust_fails_with_more_than_one_method(mutable_config):
}
with spack.config.override("bootstrap", wrong_config):
with pytest.raises(RuntimeError, match="more than one"):
_bootstrap("trust", "github-actions")
_bootstrap("enable", "github-actions")
@pytest.mark.parametrize("use_existing_dir", [True, False])

10
share/spack/spack-completion.bash
View file

@ -425,11 +425,21 @@ _spack_bootstrap_status() {
}
_spack_bootstrap_enable() {
if $list_options
then
SPACK_COMPREPLY="-h --help --scope"
else
SPACK_COMPREPLY=""
fi
}
_spack_bootstrap_disable() {
if $list_options
then
SPACK_COMPREPLY="-h --help --scope"
else
SPACK_COMPREPLY=""
fi
}
_spack_bootstrap_reset() {