3c4322bf1a
Spack can now: - label ppc64, ppc64le, x86_64, etc. builds with specific microarchitecture-specific names, like 'haswell', 'skylake' or 'icelake'. - detect the host architecture of a machine from /proc/cpuinfo or similar tools. - Understand which microarchitectures are compatible with which (for binary reuse) - Understand which compiler flags are needed (for GCC, so far) to build binaries for particular microarchitectures. All of this is managed through a JSON file (microarchitectures.json) that contains detailed auto-detection, compiler flag, and compatibility information for specific microarchitecture targets. The `llnl.util.cpu` module implements a library that allows detection and comparison of microarchitectures based on the data in this file. The `target` part of Spack specs is now essentially a Microarchitecture object, and Specs' targets can be compared for compatibility as well. This allows us to label optimized binary packages at a granularity that enables them to be reused on compatible machines. Previously, we only knew that a package was built for x86_64, NOT which x86_64 machines it was usable on. Currently this feature supports Intel, Power, and AMD chips. Support for ARM is forthcoming. Specifics: - Add microarchitectures.json with descriptions of architectures - Relaxed semantic of compiler's "target" attribute. Before this change the semantic to check if a compiler could be viable for a given target was exact match. This made sense as the finest granularity of targets was architecture families. As now we can target micro-architectures, this commit changes the semantic by interpreting as the architecture family what is stored in the compiler's "target" attribute. A compiler is then a viable choice if the target being concretized belongs to the same family. Similarly when a new compiler is detected the architecture family is stored in the "target" attribute. - Make Spack's `cc` compiler wrapper inject target-specific flags on the command line - Architecture concretization updated to use the same algorithm as compiler concretization - Micro-architecture features, vendor, generation etc. are included in the package hash. Generic architectures, such as x86_64 or ppc64, are still dumped using the name only. - If the compiler for a target is not supported exit with an intelligible error message. If the compiler support is unknown don't try to use optimization flags. - Support and define feature aliases (e.g., sse3 -> ssse3) in microarchitectures.json and on Microarchitecture objects. Feature aliases are defined in targets.json and map a name (the "alias") to a list of rules that must be met for the test to be successful. The rules that are available can be extended later using a decorator. - Implement subset semantics for comparing microarchitectures (treat microarchitectures as a partial order, i.e. (a < b), (a == b) and (b < a) can all be false. - Implement logic to automatically demote the default target if the compiler being used is too old to optimize for it. Updated docs to make this behavior explicit. This avoids surprising the user if the default compiler is older than the host architecture. This commit adds unit tests to verify the semantics of target ranges and target lists in constraints. The implementation to allow target ranges and lists is minimal and doesn't add any new type. A more careful refactor that takes into account the type system might be due later. Co-authored-by: Gregory Becker <becker33.llnl.gov> |
||
|---|---|---|
| .github | ||
| bin | ||
| etc/spack/defaults | ||
| lib/spack | ||
| share/spack | ||
| var/spack | ||
| .codecov.yml | ||
| .coveragerc | ||
| .dockerignore | ||
| .flake8 | ||
| .flake8_packages | ||
| .gitignore | ||
| .gitlab-ci.yml | ||
| .mailmap | ||
| .readthedocs.yml | ||
| .travis.yml | ||
| COPYRIGHT | ||
| LICENSE-APACHE | ||
| LICENSE-MIT | ||
| NOTICE | ||
| README.md | ||
Spack
Spack is a multi-platform package manager that builds and installs multiple versions and configurations of software. It works on Linux, macOS, and many supercomputers. Spack is non-destructive: installing a new version of a package does not break existing installations, so many configurations of the same package can coexist.
Spack offers a simple "spec" syntax that allows users to specify versions and configuration options. Package files are written in pure Python, and specs allow package authors to write a single script for many different builds of the same package. With Spack, you can build your software all the ways you want to.
See the Feature Overview for examples and highlights.
To install spack and your first package, make sure you have Python. Then:
$ git clone https://github.com/spack/spack.git
$ cd spack/bin
$ ./spack install zlib
Documentation
Full documentation is available, or
run spack help or spack help --all.
Tutorial
We maintain a hands-on tutorial. It covers basic to advanced usage, packaging, developer features, and large HPC deployments. You can do all of the exercises on your own laptop using a Docker container.
Feel free to use these materials to teach users at your organization about Spack.
Community
Spack is an open source project. Questions, discussion, and contributions are welcome. Contributions can be anything from new packages to bugfixes, documentation, or even new core features.
Resources:
- Slack workspace: spackpm.slack.com. To get an invitation, click here.
- Mailing list: groups.google.com/d/forum/spack
- Twitter: @spackpm. Be sure to
@mentionus!
Contributing
Contributing to Spack is relatively easy. Just send us a
pull request.
When you send your request, make develop the destination branch on the
Spack repository.
Your PR must pass Spack's unit tests and documentation tests, and must be PEP 8 compliant. We enforce these guidelines with Travis CI. To run these tests locally, and for helpful tips on git, see our Contribution Guide.
Spack uses a rough approximation of the
Git Flow
branching model. The develop branch contains the latest
contributions, and master is always tagged and points to the latest
stable release.
Code of Conduct
Please note that Spack has a Code of Conduct. By participating in the Spack community, you agree to abide by its rules.
Authors
Many thanks go to Spack's contributors.
Spack was created by Todd Gamblin, tgamblin@llnl.gov.
Citing Spack
If you are referencing Spack in a publication, please cite the following paper:
- Todd Gamblin, Matthew P. LeGendre, Michael R. Collette, Gregory L. Lee, Adam Moody, Bronis R. de Supinski, and W. Scott Futral. The Spack Package Manager: Bringing Order to HPC Software Chaos. In Supercomputing 2015 (SC’15), Austin, Texas, November 15-20 2015. LLNL-CONF-669890.
License
Spack is distributed under the terms of both the MIT license and the Apache License (Version 2.0). Users may choose either license, at their option.
All new contributions must be made under both the MIT and Apache-2.0 licenses.
See LICENSE-MIT, LICENSE-APACHE, COPYRIGHT, and NOTICE for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)
LLNL-CODE-647188