dd3762d0f9
Cray has two machine types. "XC" machines are the larger machines more common in HPC, but "Cluster" machines are also cropping up at some HPC sites. Cluster machines run a slightly different form of the CrayPE programming environment, and often come without default modules loaded. Cluster machines also run different versions of some software, and run a linux distro on the backend nodes instead of running Compute Node Linux (CNL). Below are the changes made to support "Cluster" machines in Spack. Some of these changes are semi-related general upkeep of the cray platform. * cray platform: detect properly after module purge * cray platform: support machines running OSs other than CNL Make Cray backend OS delegate to LinuxDistro when no cle_release file favor backend over frontend OS when name clashes * cray platform: target detection uses multiple strategies This commit improves the robustness of target detection on Cray by trying multiple strategies. The first one that produces results wins. If nothing is found only the generic family of the frontend host is used as a target. * cray-libsci: add package from NERSC * build_env: unload cray-libsci module when not explicitly needed cray-libsci is a package in Spack. The cray PrgEnv modules load it implicitly when we set up the compiler. We now unload it after setting up the compiler and only reload it when requested via external package. * util/module_cmd: more robust module parsing Cray modules have documentation inside the module that is visible to the `module show` command. Spack module parsing is now robust to documentation inside modules. * cce compiler: uses clang flags for versions >= 9.0 * build_env: push CRAY_LD_LIBRARY_PATH into everything Some Cray modules add paths to CRAY_LD_LIBRARY_PATH instead of LD_LIBRARY_PATH. This has performance benefits at load time, but leads to Spack builds not finding their dependencies from external modules. Spack now prepends CRAY_LD_LIBRARY_PATH to LD_LIBRARY_PATH before beginning the build. * mvapich2: setup cray compilers when on cray previously, mpich was the only mpi implementation to support cray systems (because it is the MPI on Cray XC systems). Cray cluster systems use mvapich2, which now supports cray compiler wrappers. * build_env: clean pkgconf from environment Cray modules silently add pkgconf to the user environment This can break builds that do not user pkgconf. Now we remove it frmo the environment and add it again if it is in the spec. * cray platform: cheat modules for rome/zen2 module on naples/zen node Cray modules for naples/zen architecture currently specify rome/zen2. For now, we detect this and return zen for modules named `craype-x86-rome`. * compiler: compiler default versions When detecting compiler default versions for target/compiler compatibility checks, Spack previously ran the compiler without setting up its environment. Now we setup a temporary environment to run the compiler with its modules to detect its version. * compilers/cce: improve logic to determine C/C++ std flags * tests: fix existing tests to play nicely with new cray support * tests: test new functionality Some new functionality can only be tested on a cray system. Add tests for what can be tested on a linux system. Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com> |
||
|---|---|---|
| .github | ||
| bin | ||
| etc/spack/defaults | ||
| lib/spack | ||
| share/spack | ||
| var/spack | ||
| .codecov.yml | ||
| .coveragerc | ||
| .dockerignore | ||
| .flake8 | ||
| .flake8_packages | ||
| .gitattributes | ||
| .gitignore | ||
| .mailmap | ||
| .readthedocs.yml | ||
| .travis.yml | ||
| CHANGELOG.md | ||
| COPYRIGHT | ||
| LICENSE-APACHE | ||
| LICENSE-MIT | ||
| NOTICE | ||
| pytest.ini | ||
| README.md | ||
Spack
Spack is a multi-platform package manager that builds and installs multiple versions and configurations of software. It works on Linux, macOS, and many supercomputers. Spack is non-destructive: installing a new version of a package does not break existing installations, so many configurations of the same package can coexist.
Spack offers a simple "spec" syntax that allows users to specify versions and configuration options. Package files are written in pure Python, and specs allow package authors to write a single script for many different builds of the same package. With Spack, you can build your software all the ways you want to.
See the Feature Overview for examples and highlights.
To install spack and your first package, make sure you have Python. Then:
$ git clone https://github.com/spack/spack.git
$ cd spack/bin
$ ./spack install zlib
Documentation
Full documentation is available, or
run spack help or spack help --all.
Tutorial
We maintain a hands-on tutorial. It covers basic to advanced usage, packaging, developer features, and large HPC deployments. You can do all of the exercises on your own laptop using a Docker container.
Feel free to use these materials to teach users at your organization about Spack.
Community
Spack is an open source project. Questions, discussion, and contributions are welcome. Contributions can be anything from new packages to bugfixes, documentation, or even new core features.
Resources:
- Slack workspace: spackpm.slack.com. To get an invitation, click here.
- Mailing list: groups.google.com/d/forum/spack
- Twitter: @spackpm. Be sure to
@mentionus!
Contributing
Contributing to Spack is relatively easy. Just send us a
pull request.
When you send your request, make develop the destination branch on the
Spack repository.
Your PR must pass Spack's unit tests and documentation tests, and must be PEP 8 compliant. We enforce these guidelines with Travis CI. To run these tests locally, and for helpful tips on git, see our Contribution Guide.
Spack uses a rough approximation of the
Git Flow
branching model. The develop branch contains the latest
contributions, and master is always tagged and points to the latest
stable release.
Code of Conduct
Please note that Spack has a Code of Conduct. By participating in the Spack community, you agree to abide by its rules.
Authors
Many thanks go to Spack's contributors.
Spack was created by Todd Gamblin, tgamblin@llnl.gov.
Citing Spack
If you are referencing Spack in a publication, please cite the following paper:
- Todd Gamblin, Matthew P. LeGendre, Michael R. Collette, Gregory L. Lee, Adam Moody, Bronis R. de Supinski, and W. Scott Futral. The Spack Package Manager: Bringing Order to HPC Software Chaos. In Supercomputing 2015 (SC’15), Austin, Texas, November 15-20 2015. LLNL-CONF-669890.
License
Spack is distributed under the terms of both the MIT license and the Apache License (Version 2.0). Users may choose either license, at their option.
All new contributions must be made under both the MIT and Apache-2.0 licenses.
See LICENSE-MIT, LICENSE-APACHE, COPYRIGHT, and NOTICE for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)
LLNL-CODE-647188