If using mpirun, the R sessions can be started with a wrapper script
that helps set up the R session cluster. Put this wrapper in the PATH so
it is easily accessible.
Add a `spack external find` command that tries to populate
`packages.yaml` with external packages from the user's `$PATH`. This
focuses on finding build dependencies. Currently, support has only been
added for `cmake`.
For a package to be discoverable with `spack external find`, it must define:
* an `executables` class attribute containing a list of
regular expressions that match executable names.
* a `determine_spec_details(prefix, specs_in_prefix)` method
Spack will call `determine_spec_details()` once for each prefix where
executables are found, passing in the path to the prefix and the path to
all found executables. The package is responsible for invoking the
executables and figuring out what type of installation(s) are in the
prefix, and returning one or more specs (each with version, variants or
whatever else the user decides to include in the spec).
The found specs and prefixes will be added to the user's `packages.yaml`
file. Providing the `--not-buildable` option will mark all generated
entries in `packages.yaml` as `buildable: False`
Cray has two machine types. "XC" machines are the larger
machines more common in HPC, but "Cluster" machines are
also cropping up at some HPC sites. Cluster machines run
a slightly different form of the CrayPE programming environment,
and often come without default modules loaded. Cluster
machines also run different versions of some software, and run
a linux distro on the backend nodes instead of running Compute
Node Linux (CNL).
Below are the changes made to support "Cluster" machines in
Spack. Some of these changes are semi-related general upkeep
of the cray platform.
* cray platform: detect properly after module purge
* cray platform: support machines running OSs other than CNL
Make Cray backend OS delegate to LinuxDistro when no cle_release file
favor backend over frontend OS when name clashes
* cray platform: target detection uses multiple strategies
This commit improves the robustness of target
detection on Cray by trying multiple strategies.
The first one that produces results wins. If
nothing is found only the generic family of the
frontend host is used as a target.
* cray-libsci: add package from NERSC
* build_env: unload cray-libsci module when not explicitly needed
cray-libsci is a package in Spack. The cray PrgEnv
modules load it implicitly when we set up the compiler.
We now unload it after setting up the compiler and
only reload it when requested via external package.
* util/module_cmd: more robust module parsing
Cray modules have documentation inside the module
that is visible to the `module show` command.
Spack module parsing is now robust to documentation
inside modules.
* cce compiler: uses clang flags for versions >= 9.0
* build_env: push CRAY_LD_LIBRARY_PATH into everything
Some Cray modules add paths to CRAY_LD_LIBRARY_PATH
instead of LD_LIBRARY_PATH. This has performance benefits
at load time, but leads to Spack builds not finding their
dependencies from external modules.
Spack now prepends CRAY_LD_LIBRARY_PATH to
LD_LIBRARY_PATH before beginning the build.
* mvapich2: setup cray compilers when on cray
previously, mpich was the only mpi implementation to support
cray systems (because it is the MPI on Cray XC systems).
Cray cluster systems use mvapich2, which now supports cray
compiler wrappers.
* build_env: clean pkgconf from environment
Cray modules silently add pkgconf to the user environment
This can break builds that do not user pkgconf.
Now we remove it frmo the environment and add it again if it
is in the spec.
* cray platform: cheat modules for rome/zen2 module on naples/zen node
Cray modules for naples/zen architecture currently specify
rome/zen2. For now, we detect this and return zen for modules
named `craype-x86-rome`.
* compiler: compiler default versions
When detecting compiler default versions for target/compiler
compatibility checks, Spack previously ran the compiler without
setting up its environment. Now we setup a temporary environment
to run the compiler with its modules to detect its version.
* compilers/cce: improve logic to determine C/C++ std flags
* tests: fix existing tests to play nicely with new cray support
* tests: test new functionality
Some new functionality can only be tested on a cray system.
Add tests for what can be tested on a linux system.
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
Since #9481 Python's None is not permitted as a value for
MV variants. The string 'none' is used instead.
Add the same fix for the amgx and lammps packages
* CMake: fix https://github.com/spack/spack/issues/16453 with a patch addressing both libhugetlbfs and icpc warnings on Cray XC40 systems
* Including CMake v3.17.2 in the patched versions
* libspatialite
* flake8
* Added proper version constrain
* Update var/spack/repos/builtin/packages/libspatialite/package.py
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/libspatialite/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Elizabeth Fischer <elizabeth.fischer@alaska.edu>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Otherwise, if I run `xenv` after `spack load py-xenv` it fails with:
```
Traceback (most recent call last):
File "/home/vavolkl/spack/opt/spack/linux-centos7-broadwell/gcc-8.3.0/py-xenv-develop_2018-12-20-lqbxakapsepqo5w3sjhhokj5o7c5jei2/bin/xenv", line 6, in <module>
from pkg_resources import load_entry_point
ModuleNotFoundError: No module named 'pkg_resources'
```
* [gaudi] fixes and patches
* Update var/spack/repos/builtin/packages/gaudi/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/gaudi/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Apply suggestions from code review
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* [gaudi] add older versions and fold +tests into +optional
* [gaudi] set run environment
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
If spack is checked out in a git worktree (see [1]), all git-related
commands fail because the `spack_is_git_repo()`-check is not thorough
enough.
When developing in a feature-branch in a seperate worktree, this is
annoying as all unittests regarding git-related spack commands fail,
cluttering the test results with false-positives.
[1]: https://git-scm.com/docs/git-worktree
Change-Id: I94b573a2c0e058e9ccc169e7ee6561626fbb06fd
* grass
* Update var/spack/repos/builtin/packages/grass/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Elizabeth Fischer <elizabeth.fischer@alaska.edu>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
