bb42470211
Spack creates a separate process to do package installation. Different operating systems and Python versions use different methods to create it but up until Python 3.8 both Linux and Mac OS used "fork" (which duplicates process memory, file descriptor table, etc.). Python >= 3.8 on Mac OS prefers creating an entirely new process (referred to as the "spawn" start method) because "fork" was found to cause issues (in other words "spawn" is the default start method used by multiprocessing.Process). Spack was dependent on the particular behavior of fork to replicate process memory and transmit file descriptors. This PR refactors the Spack internals to support starting a child process with the "spawn" method. To achieve this, it makes the following changes: - ensure that the package repository and other global state are transmitted to the child process - ensure that file descriptors are transmitted to the child process in a way that works with multiprocessing and spawn - make all the state needed for the build process and tests picklable (package, stage, etc.) - move a number of locally-defined functions into global scope so that they can be pickled - rework tests where needed to avoid using local functions This PR also reworks sbang tests to work on macOS, where temporary directories are deeper than the Linux sbang limit. We make the limit platform-dependent (macOS supports 512-character shebangs) See: #14102
66 lines
2.1 KiB
Python
Executable file
66 lines
2.1 KiB
Python
Executable file
#!/bin/sh
|
|
# -*- python -*-
|
|
#
|
|
# Copyright 2013-2020 Lawrence Livermore National Security, LLC and other
|
|
# Spack Project Developers. See the top-level COPYRIGHT file for details.
|
|
#
|
|
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
|
|
|
|
# This file is bilingual. The following shell code finds our preferred python.
|
|
# Following line is a shell no-op, and starts a multi-line Python comment.
|
|
# See https://stackoverflow.com/a/47886254
|
|
""":"
|
|
# prefer python3, then python, then python2
|
|
for cmd in python3 python python2; do
|
|
command -v > /dev/null $cmd && exec $cmd $0 "$@"
|
|
done
|
|
|
|
echo "==> Error: spack could not find a python interpreter!" >&2
|
|
exit 1
|
|
":"""
|
|
# Line above is a shell no-op, and ends a python multi-line comment.
|
|
# The code above runs this file with our preferred python interpreter.
|
|
|
|
from __future__ import print_function
|
|
|
|
import os
|
|
import sys
|
|
|
|
if sys.version_info[:2] < (2, 6):
|
|
v_info = sys.version_info[:3]
|
|
sys.exit("Spack requires Python 2.6 or higher."
|
|
"This is Python %d.%d.%d." % v_info)
|
|
|
|
# Find spack's location and its prefix.
|
|
spack_file = os.path.realpath(os.path.expanduser(__file__))
|
|
spack_prefix = os.path.dirname(os.path.dirname(spack_file))
|
|
|
|
# Allow spack libs to be imported in our scripts
|
|
spack_lib_path = os.path.join(spack_prefix, "lib", "spack")
|
|
sys.path.insert(0, spack_lib_path)
|
|
|
|
# Add external libs
|
|
spack_external_libs = os.path.join(spack_lib_path, "external")
|
|
|
|
if sys.version_info[:2] == (2, 6):
|
|
sys.path.insert(0, os.path.join(spack_external_libs, 'py26'))
|
|
|
|
sys.path.insert(0, spack_external_libs)
|
|
|
|
# Here we delete ruamel.yaml in case it has been already imported from site
|
|
# (see #9206 for a broader description of the issue).
|
|
#
|
|
# Briefly: ruamel.yaml produces a .pth file when installed with pip that
|
|
# makes the site installed package the preferred one, even though sys.path
|
|
# is modified to point to another version of ruamel.yaml.
|
|
if 'ruamel.yaml' in sys.modules:
|
|
del sys.modules['ruamel.yaml']
|
|
|
|
if 'ruamel' in sys.modules:
|
|
del sys.modules['ruamel']
|
|
|
|
import spack.main # noqa
|
|
|
|
# Once we've set up the system path, run the spack main method
|
|
if __name__ == "__main__":
|
|
sys.exit(spack.main.main())
|