* Speed-up environment concretization with a process pool
We can exploit the fact that the environment is concretized
separately and use a pool of processes to concretize it.
* Add module spack.util.parallel
Module includes `pool` and `parallel_map` abstractions,
along with implementation details for both.
* Add a new hash type to pass specs across processes
* Add tty msg with concretization time
We use POSIX `patch` to apply patches to files when building, but
`patch` by default prompts the user when it looks like a patch
has already been applied. This means that:
1. If a patch lands in upstream and we don't disable it
in a package, the build will start failing.
2. `spack develop` builds (which keep the stage around) will
fail the second time you try to use them.
To avoid that, we can run `patch` with `-N` (also called
`--forward`, but the long option is not in POSIX). `-N` causes
`patch` to just ignore patches that have already been applied.
This *almost* makes `patch` idempotent, except that it returns 1
when it detects already applied patches with `-N`, so we have to
look at the output of the command to see if it's safe to ignore
the error.
- [x] Remove non-POSIX `-s` option from `patch` call
- [x] Add `-N` option to `patch`
- [x] Ignore error status when `patch` returns 1 due to `-N`
- [x] Add tests for applying a patch twice and applying a bad patch
- [x] Tweak `spack.util.executable` so that it saves the error that
*would have been* raised with `fail_on_error=True`. This lets
us easily re-raise it.
Co-authored-by: Greg Becker <becker33@llnl.gov>
Signed-off-by: Mickaël Schoentgen <contact@tiger-222.fr>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-magic: delete redundant package
This package is actually named py-python-magic (since the project itself
is "python-magic").
* New package: libmagic
* Py-python-magic: add required runtime dependency on libmagic and new version
* Py-filemagic: add required runtime dependency
* py-magic: restore and mark as redundant
This reverts commit 4cab7fb69e2c2b8098895bee92aabe5df8b7aaaa.
* file: add implicit dependencies and static variant
Replaces redundant libmagic that I added. Compression headers were previously
being picked up from the system.
* Fix py-python-magic dependency
* Update python version requirements
* relocate: call install_name_tool less
* zstd: fix race condition
Multiple times on my mac, trying to install in parallel led to failures
from multiple tasks trying to simultaneously create `$PREFIX/lib`.
* PackageMeta: simplify callback flush
* Relocate: use spack.platforms instead of platform
* Relocate: code improvements
* fix zstd
* Automatically fix rpaths for packages on macOS
* Only change library IDs when the path is already in the rpath
This restores the hardcoded library path for GCC.
* Delete nonexistent rpaths and add more testing
* Relocate: Allow @executable_path and @loader_path
* downgrade_docutils_version
* invalid version
* Update requirements.txt
* Improve spelling and shorten the reference link
* Update spack.yaml
* update version requirement
* update version to maximum of 0.16
Co-authored-by: bernhardkaindl <43588962+bernhardkaindl@users.noreply.github.com>
* py-jupytext: add new package
* Apply suggestions from code review
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* update jupytext dependencies
* Apply suggestions from code review
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-jupytext: remove py-jupyerlab dependency
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-gevent: add version 1.5
* py-gevent: update dependencies for v1.5.0
* Apply suggestions from code review
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/py-gevent/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* phist: Prefer 1.9.5 (1.9.6 uses mpi_f08, but not available in CI)
* phist: remove dupe of 1.9.5, missing preferred=True
Also, for 1.9.6, patch the (most, one does not work) tests to use
Currently Spack keeps track of the origin in the code of any
modification to the environment variables. This is very slow
and enabled unconditionally even in code paths where the
origin of the modification is never queried.
The only place where we inspect the origins of environment
modifications is before we start a build, If there's an override
of the type `e.set(...)` after incremental changes like
`e.append_path(..)`, which is a "suspicious" change.
This is very rare though.
If an override like this ever happens, it might mean a package is
broken. If that leads to build errors, we can just ask the user to run
`spack -d install ...` and check the warnings issued by Spack to find
the origins of the problem.
