Start of basic documentation
This commit is contained in:
parent
c7539fe950
commit
c9e40b725d
23 changed files with 945 additions and 130 deletions
|
@ -35,7 +35,7 @@ parser.add_argument('-m', '--mock', action='store_true', dest='mock',
|
||||||
|
|
||||||
# each command module implements a parser() function, to which we pass its
|
# each command module implements a parser() function, to which we pass its
|
||||||
# subparser for setup.
|
# subparser for setup.
|
||||||
subparsers = parser.add_subparsers(metavar='COMMAND', dest="command")
|
subparsers = parser.add_subparsers(metavar='SUBCOMMAND', dest="command")
|
||||||
|
|
||||||
import spack.cmd
|
import spack.cmd
|
||||||
for cmd in spack.cmd.commands:
|
for cmd in spack.cmd.commands:
|
||||||
|
|
2
lib/spack/docs/.gitignore
vendored
Normal file
2
lib/spack/docs/.gitignore
vendored
Normal file
|
@ -0,0 +1,2 @@
|
||||||
|
spack*.rst
|
||||||
|
_build
|
|
@ -7,6 +7,9 @@ SPHINXBUILD = sphinx-build
|
||||||
PAPER =
|
PAPER =
|
||||||
BUILDDIR = _build
|
BUILDDIR = _build
|
||||||
|
|
||||||
|
export PYTHONPATH = ../../spack
|
||||||
|
APIDOC_FILES = spack*.rst
|
||||||
|
|
||||||
# Internal variables.
|
# Internal variables.
|
||||||
PAPEROPT_a4 = -D latex_paper_size=a4
|
PAPEROPT_a4 = -D latex_paper_size=a4
|
||||||
PAPEROPT_letter = -D latex_paper_size=letter
|
PAPEROPT_letter = -D latex_paper_size=letter
|
||||||
|
@ -14,10 +17,13 @@ ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
|
||||||
# the i18n builder cannot share the environment and doctrees with the others
|
# the i18n builder cannot share the environment and doctrees with the others
|
||||||
I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
|
I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
|
||||||
|
|
||||||
.PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext
|
.PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext apidoc
|
||||||
|
|
||||||
all: html
|
all: html
|
||||||
|
|
||||||
|
apidoc:
|
||||||
|
sphinx-apidoc -T -o . $(PYTHONPATH)/spack
|
||||||
|
|
||||||
help:
|
help:
|
||||||
@echo "Please use \`make <target>' where <target> is one of"
|
@echo "Please use \`make <target>' where <target> is one of"
|
||||||
@echo " html to make standalone HTML files"
|
@echo " html to make standalone HTML files"
|
||||||
|
@ -41,9 +47,9 @@ help:
|
||||||
@echo " doctest to run all doctests embedded in the documentation (if enabled)"
|
@echo " doctest to run all doctests embedded in the documentation (if enabled)"
|
||||||
|
|
||||||
clean:
|
clean:
|
||||||
-rm -rf $(BUILDDIR)/*
|
-rm -rf $(BUILDDIR)/* $(APIDOC_FILES)
|
||||||
|
|
||||||
html:
|
html: apidoc
|
||||||
$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
|
$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
|
||||||
@echo
|
@echo
|
||||||
@echo "Build finished. The HTML pages are in $(BUILDDIR)/html."
|
@echo "Build finished. The HTML pages are in $(BUILDDIR)/html."
|
||||||
|
|
|
@ -24,7 +24,7 @@
|
||||||
{%- if last_updated %}
|
{%- if last_updated %}
|
||||||
{% trans last_updated=last_updated|e %}Last updated on {{ last_updated }}.{% endtrans %}
|
{% trans last_updated=last_updated|e %}Last updated on {{ last_updated }}.{% endtrans %}
|
||||||
{%- endif %}
|
{%- endif %}
|
||||||
</p>
|
|
||||||
|
|
||||||
{% trans %}<a href="https://www.github.com/snide/sphinx_rtd_theme">Sphinx theme</a> provided by <a href="http://readthedocs.org">Read the Docs</a>{% endtrans %}
|
{% trans %}<a href="https://www.github.com/snide/sphinx_rtd_theme">Sphinx theme</a> provided by <a href="http://readthedocs.org">Read the Docs</a>{% endtrans %}
|
||||||
|
</p>
|
||||||
</footer>
|
</footer>
|
||||||
|
|
233
lib/spack/docs/basic_usage.rst
Normal file
233
lib/spack/docs/basic_usage.rst
Normal file
|
@ -0,0 +1,233 @@
|
||||||
|
Basic usage
|
||||||
|
=====================
|
||||||
|
|
||||||
|
Nearly everything you do wtih spack will involve the ``spack``
|
||||||
|
command. Like many well-known tools (``git``, ``cvs``, ``svn``,
|
||||||
|
``yum``, ``port``, ``apt-get``, etc.), ``spack`` is generally called
|
||||||
|
with a *subcommand* indicating the action you want to perform.
|
||||||
|
|
||||||
|
Getting Help
|
||||||
|
-----------------------
|
||||||
|
|
||||||
|
|
||||||
|
``spack help``
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
The first subcommand you should know is ``spack help``. Run with no
|
||||||
|
arguments, it will give a list of all spack options and subcommands:
|
||||||
|
|
||||||
|
.. command-output:: spack help
|
||||||
|
|
||||||
|
If you want help on the usage of a particular subcommand, you can pass
|
||||||
|
it as an argument to ``spack help``:
|
||||||
|
|
||||||
|
.. command-output:: spack help install
|
||||||
|
|
||||||
|
Alternately, you can use ``spack -h`` in place of ``spack help``, or
|
||||||
|
``spack <subcommand> -h`` to get help on a particular subcommand.
|
||||||
|
|
||||||
|
|
||||||
|
Viewing available packages
|
||||||
|
------------------------------
|
||||||
|
|
||||||
|
The first thing you will likely want to do with spack is find out what
|
||||||
|
software is available to install. There are two main commands for
|
||||||
|
this: ``spack list`` and ``spack info``.
|
||||||
|
|
||||||
|
|
||||||
|
``spack list``
|
||||||
|
~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
The ``spack list`` command does what you might expect. it prints out a
|
||||||
|
list of all the available packages you can install. Use it like
|
||||||
|
this:
|
||||||
|
|
||||||
|
.. command-output:: spack list
|
||||||
|
|
||||||
|
The packages are listed by name in alphabetical order. If you just
|
||||||
|
want to see *installed* packages, you should use ``spack list -i``
|
||||||
|
|
||||||
|
|
||||||
|
``spack info``
|
||||||
|
~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
To get information on a particular package from the full list, you can
|
||||||
|
run ``spack info <package name>``. e.g., for ``mpich``:
|
||||||
|
|
||||||
|
.. command-output:: spack info mpich
|
||||||
|
|
||||||
|
This gives basic information about the package, such as where it can
|
||||||
|
be downloaded, what other packages it depends on, virtual package
|
||||||
|
information, and a text description, if one is available. We'll give
|
||||||
|
more details on dependencies and virtual dependencies later in this
|
||||||
|
guide.
|
||||||
|
|
||||||
|
|
||||||
|
Installing and uninstalling
|
||||||
|
------------------------------
|
||||||
|
|
||||||
|
``spack install``
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
You can install any package from ``spack list``, using ``spack
|
||||||
|
install``. In the simplest case, if you just want the latest version
|
||||||
|
and you don't care about any configuration, you can just run ``spack
|
||||||
|
install <package>``:
|
||||||
|
|
||||||
|
.. code-block:: sh
|
||||||
|
|
||||||
|
spack install mpileaks
|
||||||
|
|
||||||
|
This will fetch the tarball for ``mpileaks``, expand it, verify that
|
||||||
|
it was donwloaded without errors, build the package, and install it in
|
||||||
|
its own directory in ``$SPACK_HOME/opt``. If the requested packages
|
||||||
|
depends on other packages in order to build, then they will also be
|
||||||
|
fetched and installed.
|
||||||
|
|
||||||
|
Spack also allows you to ask for *specific* configurations of a
|
||||||
|
package. For example, if you want to install something with a
|
||||||
|
specific version, you can add ``@`` after the package name, followed
|
||||||
|
by the version you want:
|
||||||
|
|
||||||
|
.. code-block:: sh
|
||||||
|
|
||||||
|
spack install mpich@3.0.4
|
||||||
|
|
||||||
|
You can install as many versions of the same pacakge as you want, and
|
||||||
|
they will not interfere with each other. Spack installs each package
|
||||||
|
into its own unique prefix. If you or another user links a library
|
||||||
|
against soething you install using Spack, it will continue to work
|
||||||
|
until you explicitly uninstall it.
|
||||||
|
|
||||||
|
The version isn't all that you can customize on a spack command line.
|
||||||
|
Spack can install many configurations, with different versions,
|
||||||
|
compilers, compiler versions, compile-time options (variants), and
|
||||||
|
even architectures (e.g., on a machine that requires cross-compiling).
|
||||||
|
Spack is also unique in that it lets you customize the *dependencies*
|
||||||
|
you build a package with. That is, you could have two configurations
|
||||||
|
of the same version of a package: one built with boost 1.39.0, and the
|
||||||
|
other version built with version 1.43.0.
|
||||||
|
|
||||||
|
Spack calls the descriptor used to refer to a particular package
|
||||||
|
configuration a **spec**. In the command lines above, both
|
||||||
|
``mpileaks`` and ``mpileaks@3.0.4`` are specs. Specs and their syntax
|
||||||
|
are covered in more detail in :ref:`sec-specs`.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
``spack uninstall``
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
To uninstall a package, just type ``spack uninstall <package>``. This
|
||||||
|
will completely remove the directory in which the package was installed.
|
||||||
|
|
||||||
|
.. code-block:: sh
|
||||||
|
|
||||||
|
spack uninstall mpich
|
||||||
|
|
||||||
|
If there are other installed packages depend on the package you're
|
||||||
|
uninstalling, spack will issue a warning to this effect. In general,
|
||||||
|
you should remove the other packages *before* removing the package
|
||||||
|
they depend on, or you risk breaking packages on your system. If you
|
||||||
|
still want to remove the package without regard for its dependencies,
|
||||||
|
you can run ``spack uninstall -f <package>`` to override Spack's
|
||||||
|
warning.
|
||||||
|
|
||||||
|
If you have more than one version of the same package installed, spack
|
||||||
|
may not be able to figure out which on eyou want uninstalled. For
|
||||||
|
example, if you have both ``mpich@3.0.2`` and ``mpich@3.1`` installed,
|
||||||
|
and you type ``spack uninstall mpich``, then Spack will not know which
|
||||||
|
one you're referring to, and it will ask you to be more specific by
|
||||||
|
providing a version to differentiate, For example, ``spack uninstall
|
||||||
|
mpich@3.1`` is unambiguous.
|
||||||
|
|
||||||
|
|
||||||
|
.. _sec-specs:
|
||||||
|
|
||||||
|
Specs
|
||||||
|
-------------------------
|
||||||
|
|
||||||
|
Dependencies
|
||||||
|
-------------------------
|
||||||
|
|
||||||
|
Virtual dependencies
|
||||||
|
-------------------------
|
||||||
|
|
||||||
|
Versions, compilers, and architectures
|
||||||
|
----------------------------------------
|
||||||
|
|
||||||
|
``spack versions``
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
``spack compilers``
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Architectures
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Spack's specs allow insatllations for multiple architectures to coexist
|
||||||
|
within the same prefix. It is also intended to support multiple
|
||||||
|
architecutres for cross-compilation.
|
||||||
|
|
||||||
|
|
||||||
|
Package lifecycle
|
||||||
|
------------------------------
|
||||||
|
|
||||||
|
The ``spack install`` command performs a number of tasks before it
|
||||||
|
finally installs each package. It downloads an archive, expands it in
|
||||||
|
a temporary directory, and only then performs the installation. Spack
|
||||||
|
has several commands that allow finer-grained control over each of
|
||||||
|
these stages of the build process.
|
||||||
|
|
||||||
|
|
||||||
|
``spack fetch``
|
||||||
|
~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
This is the first step of ``spack install``. It takes a spec and
|
||||||
|
determines the correct download URL to use for the requested package
|
||||||
|
version. It then downloads the archive, checks it against an MD5
|
||||||
|
checksum, and stores it in a staging directory if the check was
|
||||||
|
successful. The staging directory will be located under
|
||||||
|
``$SPACK_HOME/var/spack``.
|
||||||
|
|
||||||
|
If run after the archive has already been downloaded, ``spack fetch``
|
||||||
|
is idempotent and will not download the archive again.
|
||||||
|
|
||||||
|
``spack stage``
|
||||||
|
~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
This is the second step in installation after ``spack fetch``. It
|
||||||
|
expands the downloaded archive in its temporary directory, where it
|
||||||
|
will be built by ``spack install``. If the archive has already been
|
||||||
|
expanded, then this command does nothing.
|
||||||
|
|
||||||
|
``spack clean``
|
||||||
|
~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
This command has several variations, each undoing one of the
|
||||||
|
installation tasks. They are:
|
||||||
|
|
||||||
|
``spack clean``
|
||||||
|
Runs ``make clean`` in the expanded archive directory. This is useful
|
||||||
|
if an attempted build failed, and something needs to be changed to get
|
||||||
|
a package to build. If a particular package does not have a ``make clean``
|
||||||
|
target, this will do nothing.
|
||||||
|
|
||||||
|
``spack clean -w`` or ``spack clean --work``
|
||||||
|
This deletes the entire build directory and re-expands it from the downloaded
|
||||||
|
archive. This is useful if a package does not support a proper ``make clean``
|
||||||
|
target.
|
||||||
|
|
||||||
|
``spack clean -d`` or ``spack clean --dist``
|
||||||
|
This deletes the build directory *and* the downloaded archive. If
|
||||||
|
``fetch``, ``stage``, or ``install`` are run again after this, the
|
||||||
|
process will start from scratch, and the archive archive will be
|
||||||
|
downloaded again. Useful if somehow a bad archive is downloaded
|
||||||
|
accidentally and needs to be cleaned out of the staging area.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
``spack purge``
|
||||||
|
~~~~~~~~~~~~~~~~~
|
|
@ -16,7 +16,10 @@
|
||||||
# If extensions (or modules to document with autodoc) are in another directory,
|
# If extensions (or modules to document with autodoc) are in another directory,
|
||||||
# add these directories to sys.path here. If the directory is relative to the
|
# add these directories to sys.path here. If the directory is relative to the
|
||||||
# documentation root, use os.path.abspath to make it absolute, like shown here.
|
# documentation root, use os.path.abspath to make it absolute, like shown here.
|
||||||
#sys.path.insert(0, os.path.abspath('.'))
|
sys.path.insert(0, os.path.abspath('exts'))
|
||||||
|
os.environ['PATH'] += os.pathsep + '../../../bin'
|
||||||
|
|
||||||
|
todo_include_todos = True
|
||||||
|
|
||||||
# -- General configuration -----------------------------------------------------
|
# -- General configuration -----------------------------------------------------
|
||||||
|
|
||||||
|
@ -25,7 +28,10 @@
|
||||||
|
|
||||||
# Add any Sphinx extension module names here, as strings. They can be extensions
|
# Add any Sphinx extension module names here, as strings. They can be extensions
|
||||||
# coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
|
# coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
|
||||||
extensions = ['sphinx.ext.autodoc']
|
extensions = ['sphinx.ext.autodoc',
|
||||||
|
'sphinx.ext.graphviz',
|
||||||
|
'sphinx.ext.todo',
|
||||||
|
'sphinxcontrib.programoutput']
|
||||||
|
|
||||||
# Add any paths that contain templates here, relative to this directory.
|
# Add any paths that contain templates here, relative to this directory.
|
||||||
templates_path = ['_templates']
|
templates_path = ['_templates']
|
||||||
|
|
31
lib/spack/docs/developer_guide.rst
Normal file
31
lib/spack/docs/developer_guide.rst
Normal file
|
@ -0,0 +1,31 @@
|
||||||
|
Developer Guide
|
||||||
|
=====================
|
||||||
|
|
||||||
|
|
||||||
|
Spec objects
|
||||||
|
-------------------------
|
||||||
|
|
||||||
|
Package objects
|
||||||
|
-------------------------
|
||||||
|
|
||||||
|
Stage objects
|
||||||
|
-------------------------
|
||||||
|
|
||||||
|
Writing commands
|
||||||
|
-------------------------
|
||||||
|
|
||||||
|
Unit tests
|
||||||
|
-------------------------
|
||||||
|
|
||||||
|
Unit testing
|
||||||
|
-------------------------
|
||||||
|
|
||||||
|
|
||||||
|
Developer commands
|
||||||
|
-------------------------
|
||||||
|
|
||||||
|
``spack doc``
|
||||||
|
~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
``spack test``
|
||||||
|
~~~~~~~~~~~~~~~~~
|
25
lib/spack/docs/exts/sphinxcontrib/LICENSE
Normal file
25
lib/spack/docs/exts/sphinxcontrib/LICENSE
Normal file
|
@ -0,0 +1,25 @@
|
||||||
|
Copyright (c) 2010, 2011, 2012 Sebastian Wiesner <lunaryorn@googlemail.com>
|
||||||
|
All rights reserved.
|
||||||
|
|
||||||
|
Redistribution and use in source and binary forms, with or without
|
||||||
|
modification, are permitted provided that the following conditions are
|
||||||
|
met:
|
||||||
|
|
||||||
|
* Redistributions of source code must retain the above copyright
|
||||||
|
notice, this list of conditions and the following disclaimer.
|
||||||
|
|
||||||
|
* Redistributions in binary form must reproduce the above copyright
|
||||||
|
notice, this list of conditions and the following disclaimer in the
|
||||||
|
documentation and/or other materials provided with the distribution.
|
||||||
|
|
||||||
|
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||||
|
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||||
|
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||||
|
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||||
|
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||||
|
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||||
|
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||||
|
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||||
|
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||||
|
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||||
|
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
9
lib/spack/docs/exts/sphinxcontrib/__init__.py
Normal file
9
lib/spack/docs/exts/sphinxcontrib/__init__.py
Normal file
|
@ -0,0 +1,9 @@
|
||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
"""
|
||||||
|
sphinxcontrib
|
||||||
|
~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Contains 3rd party Sphinx extensions.
|
||||||
|
"""
|
||||||
|
|
||||||
|
__import__('pkg_resources').declare_namespace(__name__)
|
263
lib/spack/docs/exts/sphinxcontrib/programoutput.py
Normal file
263
lib/spack/docs/exts/sphinxcontrib/programoutput.py
Normal file
|
@ -0,0 +1,263 @@
|
||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
# Copyright (c) 2010, 2011, 2012, Sebastian Wiesner <lunaryorn@gmail.com>
|
||||||
|
# All rights reserved.
|
||||||
|
|
||||||
|
# Redistribution and use in source and binary forms, with or without
|
||||||
|
# modification, are permitted provided that the following conditions are met:
|
||||||
|
|
||||||
|
# 1. Redistributions of source code must retain the above copyright notice,
|
||||||
|
# this list of conditions and the following disclaimer.
|
||||||
|
# 2. Redistributions in binary form must reproduce the above copyright
|
||||||
|
# notice, this list of conditions and the following disclaimer in the
|
||||||
|
# documentation and/or other materials provided with the distribution.
|
||||||
|
|
||||||
|
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
|
||||||
|
# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
||||||
|
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
|
||||||
|
# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
|
||||||
|
# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
|
||||||
|
# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
|
||||||
|
# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
|
||||||
|
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
|
||||||
|
# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
|
||||||
|
# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
|
||||||
|
# POSSIBILITY OF SUCH DAMAGE.
|
||||||
|
|
||||||
|
"""
|
||||||
|
sphinxcontrib.programoutput
|
||||||
|
===========================
|
||||||
|
|
||||||
|
This extension provides a directive to include the output of commands as
|
||||||
|
literal block while building the docs.
|
||||||
|
|
||||||
|
.. moduleauthor:: Sebastian Wiesner <lunaryorn@gmail.com>
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import (print_function, division, unicode_literals,
|
||||||
|
absolute_import)
|
||||||
|
|
||||||
|
import sys
|
||||||
|
import os
|
||||||
|
import shlex
|
||||||
|
from subprocess import Popen, PIPE, STDOUT
|
||||||
|
from collections import defaultdict, namedtuple
|
||||||
|
|
||||||
|
from docutils import nodes
|
||||||
|
from docutils.parsers import rst
|
||||||
|
from docutils.parsers.rst.directives import flag, unchanged, nonnegative_int
|
||||||
|
|
||||||
|
|
||||||
|
__version__ = '0.9'
|
||||||
|
|
||||||
|
|
||||||
|
class program_output(nodes.Element):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def _slice(value):
|
||||||
|
parts = [int(v.strip()) for v in value.split(',')]
|
||||||
|
if len(parts) > 2:
|
||||||
|
raise ValueError('too many slice parts')
|
||||||
|
return tuple((parts + [None] * 2)[:2])
|
||||||
|
|
||||||
|
|
||||||
|
class ProgramOutputDirective(rst.Directive):
|
||||||
|
has_content = False
|
||||||
|
final_argument_whitespace = True
|
||||||
|
required_arguments = 1
|
||||||
|
|
||||||
|
option_spec = dict(shell=flag, prompt=flag, nostderr=flag,
|
||||||
|
ellipsis=_slice, extraargs=unchanged,
|
||||||
|
returncode=nonnegative_int, cwd=unchanged)
|
||||||
|
|
||||||
|
def run(self):
|
||||||
|
env = self.state.document.settings.env
|
||||||
|
|
||||||
|
node = program_output()
|
||||||
|
node.line = self.lineno
|
||||||
|
node['command'] = self.arguments[0]
|
||||||
|
|
||||||
|
if self.name == 'command-output':
|
||||||
|
node['show_prompt'] = True
|
||||||
|
else:
|
||||||
|
node['show_prompt'] = 'prompt' in self.options
|
||||||
|
|
||||||
|
node['hide_standard_error'] = 'nostderr' in self.options
|
||||||
|
node['extraargs'] = self.options.get('extraargs', '')
|
||||||
|
_, cwd = env.relfn2path(self.options.get('cwd', '/'))
|
||||||
|
node['working_directory'] = cwd
|
||||||
|
node['use_shell'] = 'shell' in self.options
|
||||||
|
node['returncode'] = self.options.get('returncode', 0)
|
||||||
|
if 'ellipsis' in self.options:
|
||||||
|
node['strip_lines'] = self.options['ellipsis']
|
||||||
|
return [node]
|
||||||
|
|
||||||
|
|
||||||
|
_Command = namedtuple(
|
||||||
|
'Command', 'command shell hide_standard_error working_directory')
|
||||||
|
|
||||||
|
|
||||||
|
class Command(_Command):
|
||||||
|
"""
|
||||||
|
A command to be executed.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __new__(cls, command, shell=False, hide_standard_error=False,
|
||||||
|
working_directory='/'):
|
||||||
|
if isinstance(command, list):
|
||||||
|
command = tuple(command)
|
||||||
|
# `chdir()` resolves symlinks, so we need to resolve them too for
|
||||||
|
# caching to make sure that different symlinks to the same directory
|
||||||
|
# don't result in different cache keys. Also normalize paths to make
|
||||||
|
# sure that identical paths are also equal as strings.
|
||||||
|
working_directory = os.path.normpath(os.path.realpath(
|
||||||
|
working_directory))
|
||||||
|
return _Command.__new__(cls, command, shell, hide_standard_error,
|
||||||
|
working_directory)
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_program_output_node(cls, node):
|
||||||
|
"""
|
||||||
|
Create a command from a :class:`program_output` node.
|
||||||
|
"""
|
||||||
|
extraargs = node.get('extraargs', '')
|
||||||
|
command = (node['command'] + ' ' + extraargs).strip()
|
||||||
|
return cls(command, node['use_shell'],
|
||||||
|
node['hide_standard_error'], node['working_directory'])
|
||||||
|
|
||||||
|
def execute(self):
|
||||||
|
"""
|
||||||
|
Execute this command.
|
||||||
|
|
||||||
|
Return the :class:`~subprocess.Popen` object representing the running
|
||||||
|
command.
|
||||||
|
"""
|
||||||
|
if self.shell:
|
||||||
|
if sys.version_info[0] < 3 and isinstance(self.command, unicode):
|
||||||
|
command = self.command.encode(sys.getfilesystemencoding())
|
||||||
|
else:
|
||||||
|
command = self.command
|
||||||
|
else:
|
||||||
|
if sys.version_info[0] < 3 and isinstance(self.command, unicode):
|
||||||
|
command = shlex.split(self.command.encode(
|
||||||
|
sys.getfilesystemencoding()))
|
||||||
|
elif isinstance(self.command, str):
|
||||||
|
command = shlex.split(self.command)
|
||||||
|
else:
|
||||||
|
command = self.command
|
||||||
|
return Popen(command, shell=self.shell, stdout=PIPE,
|
||||||
|
stderr=PIPE if self.hide_standard_error else STDOUT,
|
||||||
|
cwd=self.working_directory)
|
||||||
|
|
||||||
|
def get_output(self):
|
||||||
|
"""
|
||||||
|
Get the output of this command.
|
||||||
|
|
||||||
|
Return a tuple ``(returncode, output)``. ``returncode`` is the
|
||||||
|
integral return code of the process, ``output`` is the output as
|
||||||
|
unicode string, with final trailing spaces and new lines stripped.
|
||||||
|
"""
|
||||||
|
process = self.execute()
|
||||||
|
output = process.communicate()[0].decode(
|
||||||
|
sys.getfilesystemencoding(), 'replace').rstrip()
|
||||||
|
return process.returncode, output
|
||||||
|
|
||||||
|
def __str__(self):
|
||||||
|
if isinstance(self.command, tuple):
|
||||||
|
return repr(list(self.command))
|
||||||
|
return repr(self.command)
|
||||||
|
|
||||||
|
|
||||||
|
class ProgramOutputCache(defaultdict):
|
||||||
|
"""
|
||||||
|
Execute command and cache their output.
|
||||||
|
|
||||||
|
This class is a mapping. Its keys are :class:`Command` objects represeting
|
||||||
|
command invocations. Its values are tuples of the form ``(returncode,
|
||||||
|
output)``, where ``returncode`` is the integral return code of the command,
|
||||||
|
and ``output`` is the output as unicode string.
|
||||||
|
|
||||||
|
The first time, a key is retrieved from this object, the command is
|
||||||
|
invoked, and its result is cached. Subsequent access to the same key
|
||||||
|
returns the cached value.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __missing__(self, command):
|
||||||
|
"""
|
||||||
|
Called, if a command was not found in the cache.
|
||||||
|
|
||||||
|
``command`` is an instance of :class:`Command`.
|
||||||
|
"""
|
||||||
|
result = command.get_output()
|
||||||
|
self[command] = result
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
def run_programs(app, doctree):
|
||||||
|
"""
|
||||||
|
Execute all programs represented by ``program_output`` nodes in
|
||||||
|
``doctree``. Each ``program_output`` node in ``doctree`` is then
|
||||||
|
replaced with a node, that represents the output of this program.
|
||||||
|
|
||||||
|
The program output is retrieved from the cache in
|
||||||
|
``app.env.programoutput_cache``.
|
||||||
|
"""
|
||||||
|
if app.config.programoutput_use_ansi:
|
||||||
|
# enable ANSI support, if requested by config
|
||||||
|
from sphinxcontrib.ansi import ansi_literal_block
|
||||||
|
node_class = ansi_literal_block
|
||||||
|
else:
|
||||||
|
node_class = nodes.literal_block
|
||||||
|
|
||||||
|
cache = app.env.programoutput_cache
|
||||||
|
|
||||||
|
for node in doctree.traverse(program_output):
|
||||||
|
command = Command.from_program_output_node(node)
|
||||||
|
try:
|
||||||
|
returncode, output = cache[command]
|
||||||
|
except EnvironmentError as error:
|
||||||
|
error_message = 'Command {0} failed: {1}'.format(command, error)
|
||||||
|
error_node = doctree.reporter.error(error_message, base_node=node)
|
||||||
|
node.replace_self(error_node)
|
||||||
|
else:
|
||||||
|
if returncode != node['returncode']:
|
||||||
|
app.warn('Unexpected return code {0} from command {1}'.format(
|
||||||
|
returncode, command))
|
||||||
|
|
||||||
|
# replace lines with ..., if ellipsis is specified
|
||||||
|
if 'strip_lines' in node:
|
||||||
|
lines = output.splitlines()
|
||||||
|
start, stop = node['strip_lines']
|
||||||
|
lines[start:stop] = ['...']
|
||||||
|
output = '\n'.join(lines)
|
||||||
|
|
||||||
|
if node['show_prompt']:
|
||||||
|
tmpl = app.config.programoutput_prompt_template
|
||||||
|
output = tmpl.format(command=node['command'], output=output,
|
||||||
|
returncode=returncode)
|
||||||
|
|
||||||
|
new_node = node_class(output, output)
|
||||||
|
new_node['language'] = 'text'
|
||||||
|
node.replace_self(new_node)
|
||||||
|
|
||||||
|
|
||||||
|
def init_cache(app):
|
||||||
|
"""
|
||||||
|
Initialize the cache for program output at
|
||||||
|
``app.env.programoutput_cache``, if not already present (e.g. being
|
||||||
|
loaded from a pickled environment).
|
||||||
|
|
||||||
|
The cache is of type :class:`ProgramOutputCache`.
|
||||||
|
"""
|
||||||
|
if not hasattr(app.env, 'programoutput_cache'):
|
||||||
|
app.env.programoutput_cache = ProgramOutputCache()
|
||||||
|
|
||||||
|
|
||||||
|
def setup(app):
|
||||||
|
app.add_config_value('programoutput_use_ansi', False, 'env')
|
||||||
|
app.add_config_value('programoutput_prompt_template',
|
||||||
|
'$ {command}\n{output}', 'env')
|
||||||
|
app.add_directive('program-output', ProgramOutputDirective)
|
||||||
|
app.add_directive('command-output', ProgramOutputDirective)
|
||||||
|
app.connect(str('builder-inited'), init_cache)
|
||||||
|
app.connect(str('doctree-read'), run_programs)
|
96
lib/spack/docs/features.rst
Normal file
96
lib/spack/docs/features.rst
Normal file
|
@ -0,0 +1,96 @@
|
||||||
|
Feature Overview
|
||||||
|
==================
|
||||||
|
|
||||||
|
This is an overview of features that make Spack different from other
|
||||||
|
`package managers <http://en.wikipedia.org/wiki/Package_management_system>`_
|
||||||
|
and `port systems <http://en.wikipedia.org/wiki/Ports_collection>`_.
|
||||||
|
|
||||||
|
Simple package installation
|
||||||
|
----------------------------
|
||||||
|
|
||||||
|
Installing packages is easy with Spack when you just want the default
|
||||||
|
version. This installs the latest version of mpileaks and all of its
|
||||||
|
dependencies:
|
||||||
|
|
||||||
|
.. code-block:: sh
|
||||||
|
|
||||||
|
$ spack install mpileaks
|
||||||
|
|
||||||
|
Custom versions & configurations
|
||||||
|
-------------------------------------------
|
||||||
|
|
||||||
|
If there's some aspect of your package that you want to customize, you
|
||||||
|
can do that too.
|
||||||
|
|
||||||
|
.. code-block:: sh
|
||||||
|
|
||||||
|
# Install a particular version by appending @
|
||||||
|
$ spack install mpileaks@1.1.2
|
||||||
|
|
||||||
|
# Or your favorite compiler (and its version), with %
|
||||||
|
$ spack install mpileaks@1.1.2 %gcc@4.7.3
|
||||||
|
|
||||||
|
# Add some special compile-time options with +
|
||||||
|
$ spack install mpileaks@1.1.2 %gcc@4.7.3 +debug
|
||||||
|
|
||||||
|
# Cross-compile for a different architecture with =
|
||||||
|
$ spack install mpileaks@1.1.2 =bgqos_0
|
||||||
|
|
||||||
|
Customize dependencies
|
||||||
|
-------------------------------------
|
||||||
|
|
||||||
|
You can customize package dependencies with ``^``. Suppose that
|
||||||
|
``mpileaks`` depends indirectly on ``libelf`` and ``libdwarf``. Using
|
||||||
|
``^``, you can add custom configurations for the dependencies, too.
|
||||||
|
|
||||||
|
.. code-block:: sh
|
||||||
|
|
||||||
|
# Install mpileaks and link it with specific versions of libelf and libdwarf
|
||||||
|
$ spack install mpileaks@1.1.2 %gcc@4.7.3 +debug ^libelf@0.8.12 ^libdwarf@20130729+debug
|
||||||
|
|
||||||
|
|
||||||
|
Non-destructive installs
|
||||||
|
-------------------------------------
|
||||||
|
|
||||||
|
Spack installs every unique package configuration in its own prefix,
|
||||||
|
so you can install as many different versions and configurations as
|
||||||
|
you want. New installs will not break existing ones.
|
||||||
|
|
||||||
|
|
||||||
|
Packages can peacefully coexist
|
||||||
|
-------------------------------------
|
||||||
|
|
||||||
|
Spack uses ``RPATH`` everywhere, so users do not need to customize
|
||||||
|
``LD_LIBRARY_PATH``. If you use a library or run a program, it will
|
||||||
|
run the way you built it.
|
||||||
|
|
||||||
|
|
||||||
|
Creating packages is easy
|
||||||
|
-------------------------------------
|
||||||
|
|
||||||
|
To create your own packages, give spack the tarball URL. Spack
|
||||||
|
creates all the boilerplate for you.
|
||||||
|
|
||||||
|
.. code-block:: sh
|
||||||
|
|
||||||
|
$ spack create http://scalability.llnl.gov/mpileaks/downloads/mpileaks-1.0.tar.gz
|
||||||
|
|
||||||
|
Creates ``mpileaks.py``:
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
from spack import *
|
||||||
|
|
||||||
|
class Mpileaks(Package):
|
||||||
|
homepage = "http://www.example.com/"
|
||||||
|
url = "http://scalability.llnl.gov/mpileaks/downloads/mpileaks-1.0.tar.gz"
|
||||||
|
md5 = "4136d7b4c04df68b686570afa26988ac"
|
||||||
|
|
||||||
|
def install(self, prefix):
|
||||||
|
configure("--prefix=%s" % prefix)
|
||||||
|
make()
|
||||||
|
make("install")
|
||||||
|
|
||||||
|
Packages are pure python, so you have complete freedom when writing
|
||||||
|
build code. Spack also provides a number of feature that make it
|
||||||
|
easier to write packages.
|
44
lib/spack/docs/getting_started.rst
Normal file
44
lib/spack/docs/getting_started.rst
Normal file
|
@ -0,0 +1,44 @@
|
||||||
|
Getting Started
|
||||||
|
====================
|
||||||
|
|
||||||
|
Download
|
||||||
|
--------------------
|
||||||
|
|
||||||
|
Getting spack is easy. Clone it using `git <http://git-scm.com->`_
|
||||||
|
with the following command:
|
||||||
|
|
||||||
|
.. code-block:: sh
|
||||||
|
|
||||||
|
$ git clone ssh://git@cz-stash.llnl.gov:7999/scale/spack.git
|
||||||
|
|
||||||
|
This will create a directory called ``spack``. We'll assume that the
|
||||||
|
full path to this directory is in some environment called
|
||||||
|
``SPACK_HOME``. Add ``$SPACK_HOME/bin`` to your path and you're ready
|
||||||
|
to go:
|
||||||
|
|
||||||
|
.. code-block:: sh
|
||||||
|
|
||||||
|
$ export PATH=spack/bin:$SPACK_HOME
|
||||||
|
$ spack install mpich
|
||||||
|
|
||||||
|
In general, most of your interactions with Spack will be through the
|
||||||
|
``spack`` command.
|
||||||
|
|
||||||
|
|
||||||
|
Install
|
||||||
|
--------------------
|
||||||
|
|
||||||
|
You don't need to install Spack; it's ready to run as soon as you
|
||||||
|
clone it from git.
|
||||||
|
|
||||||
|
You may want to run it out of a prefix other than the git repository
|
||||||
|
you cloned. The ``spack bootstrap`` command provides this
|
||||||
|
functionality. To install spack in a new directory, simply type:
|
||||||
|
|
||||||
|
.. code-block:: sh
|
||||||
|
|
||||||
|
$ spack bootstrap /my/favorite/prefix
|
||||||
|
|
||||||
|
This will install a new spack script in /my/favorite/prefix/bin, which
|
||||||
|
you can use just like you would the regular spack script. Each copy
|
||||||
|
of spack installs packages into its own ``$PREFIX/opt`` directory.
|
|
@ -3,15 +3,39 @@
|
||||||
You can adapt this file completely to your liking, but it should at least
|
You can adapt this file completely to your liking, but it should at least
|
||||||
contain the root `toctree` directive.
|
contain the root `toctree` directive.
|
||||||
|
|
||||||
Welcome to Spack's documentation!
|
Spack Documentation
|
||||||
=================================
|
=================================
|
||||||
|
|
||||||
Contents:
|
Spack builds and installs software the way you want it. Other tools
|
||||||
|
let you install the latest version once. Spack lets you install the
|
||||||
|
versions you want, built with the compilers, libraries, and options
|
||||||
|
you want. Spack is non-destructive; installing a new version does not
|
||||||
|
break your old installs. See the :doc:`features` for more highlights.
|
||||||
|
|
||||||
|
Get spack and install your first package:
|
||||||
|
|
||||||
|
.. code-block:: sh
|
||||||
|
|
||||||
|
$ git clone ssh://git@cz-stash.llnl.gov:7999/scale/spack.git
|
||||||
|
$ cd spack/bin
|
||||||
|
$ ./spack install mpich
|
||||||
|
|
||||||
|
If you're new to spack and want to start using it, see :doc:`getting_started`,
|
||||||
|
or refer to the full manual below.
|
||||||
|
|
||||||
|
Table of Contents
|
||||||
|
---------------------
|
||||||
|
|
||||||
.. toctree::
|
.. toctree::
|
||||||
:maxdepth: 2
|
:maxdepth: 2
|
||||||
|
|
||||||
|
features
|
||||||
|
getting_started
|
||||||
|
basic_usage
|
||||||
|
packaging_guide
|
||||||
|
site_configuration
|
||||||
|
developer_guide
|
||||||
|
API Docs <spack>
|
||||||
|
|
||||||
Indices and tables
|
Indices and tables
|
||||||
==================
|
==================
|
||||||
|
@ -19,4 +43,3 @@ Indices and tables
|
||||||
* :ref:`genindex`
|
* :ref:`genindex`
|
||||||
* :ref:`modindex`
|
* :ref:`modindex`
|
||||||
* :ref:`search`
|
* :ref:`search`
|
||||||
|
|
||||||
|
|
35
lib/spack/docs/packaging_guide.rst
Normal file
35
lib/spack/docs/packaging_guide.rst
Normal file
|
@ -0,0 +1,35 @@
|
||||||
|
Packaging Guide
|
||||||
|
=====================
|
||||||
|
|
||||||
|
|
||||||
|
Package files
|
||||||
|
-------------------------
|
||||||
|
|
||||||
|
|
||||||
|
Dependencies
|
||||||
|
-------------------------
|
||||||
|
|
||||||
|
|
||||||
|
Virtual dependencies
|
||||||
|
-------------------------
|
||||||
|
|
||||||
|
|
||||||
|
Virtual dependencies
|
||||||
|
-------------------------
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
Packaging commands
|
||||||
|
-------------------------
|
||||||
|
|
||||||
|
``spack edit``
|
||||||
|
~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
``spack create``
|
||||||
|
~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
``spack checksum``
|
||||||
|
~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
``spack graph``
|
||||||
|
~~~~~~~~~~~~~~~~~~~~
|
9
lib/spack/docs/site_configuration.rst
Normal file
9
lib/spack/docs/site_configuration.rst
Normal file
|
@ -0,0 +1,9 @@
|
||||||
|
Site-specific configuration
|
||||||
|
===================================
|
||||||
|
|
||||||
|
Temporary space
|
||||||
|
----------------------------
|
||||||
|
|
||||||
|
|
||||||
|
Concretization policies
|
||||||
|
----------------------------
|
|
@ -17,7 +17,7 @@ def get_origin_url():
|
||||||
return origin_url.strip()
|
return origin_url.strip()
|
||||||
|
|
||||||
|
|
||||||
def install_spack(parser, args):
|
def bootstrap(parser, args):
|
||||||
origin_url = get_origin_url()
|
origin_url = get_origin_url()
|
||||||
prefix = args.prefix
|
prefix = args.prefix
|
||||||
|
|
||||||
|
@ -28,7 +28,7 @@ def install_spack(parser, args):
|
||||||
|
|
||||||
files_in_the_way = os.listdir(prefix)
|
files_in_the_way = os.listdir(prefix)
|
||||||
if files_in_the_way:
|
if files_in_the_way:
|
||||||
tty.die("There are already files there! Delete these files before installing spack.",
|
tty.die("There are already files there! Delete these files before boostrapping spack.",
|
||||||
*files_in_the_way)
|
*files_in_the_way)
|
||||||
|
|
||||||
tty.msg("Installing:",
|
tty.msg("Installing:",
|
||||||
|
@ -41,5 +41,5 @@ def install_spack(parser, args):
|
||||||
check_call(['git', 'fetch', 'origin', 'master:refs/remotes/origin/master', '-n', '-q'])
|
check_call(['git', 'fetch', 'origin', 'master:refs/remotes/origin/master', '-n', '-q'])
|
||||||
check_call(['git', 'reset', '--hard', 'origin/master', '-q'])
|
check_call(['git', 'reset', '--hard', 'origin/master', '-q'])
|
||||||
|
|
||||||
tty.msg("Successfully installed spack in %s" % prefix,
|
tty.msg("Successfully created a new spack in %s" % prefix,
|
||||||
"Run %s/bin/spack to use this installation." % prefix)
|
"Run %s/bin/spack to use this installation." % prefix)
|
|
@ -9,9 +9,9 @@
|
||||||
|
|
||||||
def setup_parser(subparser):
|
def setup_parser(subparser):
|
||||||
subparser.add_argument('-c', "--clean", action="store_true", dest='clean',
|
subparser.add_argument('-c', "--clean", action="store_true", dest='clean',
|
||||||
help="run make clean in the stage directory (default)")
|
help="run make clean in the build directory (default)")
|
||||||
subparser.add_argument('-w', "--work", action="store_true", dest='work',
|
subparser.add_argument('-w', "--work", action="store_true", dest='work',
|
||||||
help="delete and re-expand the entire stage directory")
|
help="delete the build directory and re-expand it from its archive.")
|
||||||
subparser.add_argument('-d', "--dist", action="store_true", dest='dist',
|
subparser.add_argument('-d', "--dist", action="store_true", dest='dist',
|
||||||
help="delete the downloaded archive.")
|
help="delete the downloaded archive.")
|
||||||
subparser.add_argument('packages', nargs=argparse.REMAINDER,
|
subparser.add_argument('packages', nargs=argparse.REMAINDER,
|
||||||
|
|
|
@ -35,8 +35,8 @@
|
||||||
class Package(object):
|
class Package(object):
|
||||||
"""This is the superclass for all spack packages.
|
"""This is the superclass for all spack packages.
|
||||||
|
|
||||||
The Package class
|
***The Package class***
|
||||||
==================
|
|
||||||
Package is where the bulk of the work of installing packages is done.
|
Package is where the bulk of the work of installing packages is done.
|
||||||
|
|
||||||
A package defines how to fetch, verfiy (via, e.g., md5), build, and
|
A package defines how to fetch, verfiy (via, e.g., md5), build, and
|
||||||
|
@ -53,11 +53,13 @@ class Package(object):
|
||||||
in this directory. Spack automatically scans the python files there
|
in this directory. Spack automatically scans the python files there
|
||||||
and figures out which one to import when you invoke it.
|
and figures out which one to import when you invoke it.
|
||||||
|
|
||||||
An example package
|
**An example package**
|
||||||
====================
|
|
||||||
Let's look at the cmake package to start with. This package lives in
|
Let's look at the cmake package to start with. This package lives in
|
||||||
$prefix/lib/spack/spack/packages/cmake.py:
|
$prefix/lib/spack/spack/packages/cmake.py:
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
from spack import *
|
from spack import *
|
||||||
class Cmake(Package):
|
class Cmake(Package):
|
||||||
homepage = 'https://www.cmake.org'
|
homepage = 'https://www.cmake.org'
|
||||||
|
@ -70,18 +72,20 @@ def install(self, prefix):
|
||||||
make()
|
make()
|
||||||
make('install')
|
make('install')
|
||||||
|
|
||||||
Naming conventions
|
**Naming conventions**
|
||||||
---------------------
|
|
||||||
There are two names you should care about:
|
There are two names you should care about:
|
||||||
|
|
||||||
1. The module name, 'cmake'.
|
1. The module name, ``cmake``.
|
||||||
- User will refers to this name, e.g. 'spack install cmake'.
|
|
||||||
- Corresponds to the name of the file, 'cmake.py', and it can
|
|
||||||
include _, -, and numbers (it can even start with a number).
|
|
||||||
|
|
||||||
2. The class name, "Cmake". This is formed by converting -'s or _'s
|
* User will refers to this name, e.g. 'spack install cmake'.
|
||||||
in the module name to camel case. If the name starts with a number,
|
* Corresponds to the name of the file, 'cmake.py', and it can
|
||||||
we prefix the class name with 'Num_'. Examples:
|
include ``_``, ``-``, and numbers (it can even start with a
|
||||||
|
number).
|
||||||
|
|
||||||
|
2. The class name, "Cmake". This is formed by converting `-` or
|
||||||
|
``_`` in the module name to camel case. If the name starts with
|
||||||
|
a number, we prefix the class name with ``Num_``. Examples:
|
||||||
|
|
||||||
Module Name Class Name
|
Module Name Class Name
|
||||||
foo_bar FooBar
|
foo_bar FooBar
|
||||||
|
@ -91,23 +95,28 @@ def install(self, prefix):
|
||||||
|
|
||||||
The class name is what spack looks for when it loads a package module.
|
The class name is what spack looks for when it loads a package module.
|
||||||
|
|
||||||
Required Attributes
|
**Required Attributes**
|
||||||
---------------------
|
|
||||||
Aside from proper naming, here is the bare minimum set of things you
|
Aside from proper naming, here is the bare minimum set of things you
|
||||||
need when you make a package:
|
need when you make a package:
|
||||||
homepage informational URL, so that users know what they're
|
|
||||||
|
homepage
|
||||||
|
informational URL, so that users know what they're
|
||||||
installing.
|
installing.
|
||||||
|
|
||||||
url URL of the source archive that spack will fetch.
|
url
|
||||||
|
URL of the source archive that spack will fetch.
|
||||||
|
|
||||||
md5 md5 hash of the source archive, so that we can
|
md5
|
||||||
|
md5 hash of the source archive, so that we can
|
||||||
verify that it was downloaded securely and correctly.
|
verify that it was downloaded securely and correctly.
|
||||||
|
|
||||||
install() This function tells spack how to build and install the
|
install()
|
||||||
|
This function tells spack how to build and install the
|
||||||
software it downloaded.
|
software it downloaded.
|
||||||
|
|
||||||
Optional Attributes
|
**Optional Attributes**
|
||||||
---------------------
|
|
||||||
You can also optionally add these attributes, if needed:
|
You can also optionally add these attributes, if needed:
|
||||||
list_url
|
list_url
|
||||||
Webpage to scrape for available version strings. Default is the
|
Webpage to scrape for available version strings. Default is the
|
||||||
|
@ -121,8 +130,8 @@ def install(self, prefix):
|
||||||
your package needs special version formatting in its URL. boost
|
your package needs special version formatting in its URL. boost
|
||||||
is an example of a package that needs this.
|
is an example of a package that needs this.
|
||||||
|
|
||||||
Creating Packages
|
***Creating Packages***
|
||||||
===================
|
|
||||||
As a package creator, you can probably ignore most of the preceding
|
As a package creator, you can probably ignore most of the preceding
|
||||||
information, because you can use the 'spack create' command to do it
|
information, because you can use the 'spack create' command to do it
|
||||||
all automatically.
|
all automatically.
|
||||||
|
@ -130,8 +139,8 @@ def install(self, prefix):
|
||||||
You as the package creator generally only have to worry about writing
|
You as the package creator generally only have to worry about writing
|
||||||
your install function and specifying dependencies.
|
your install function and specifying dependencies.
|
||||||
|
|
||||||
spack create
|
**spack create**
|
||||||
----------------
|
|
||||||
Most software comes in nicely packaged tarballs, like this one:
|
Most software comes in nicely packaged tarballs, like this one:
|
||||||
http://www.cmake.org/files/v2.8/cmake-2.8.10.2.tar.gz
|
http://www.cmake.org/files/v2.8/cmake-2.8.10.2.tar.gz
|
||||||
|
|
||||||
|
@ -147,11 +156,13 @@ def install(self, prefix):
|
||||||
Once this skeleton code is generated, spack pops up the new package in
|
Once this skeleton code is generated, spack pops up the new package in
|
||||||
your $EDITOR so that you can modify the parts that need changes.
|
your $EDITOR so that you can modify the parts that need changes.
|
||||||
|
|
||||||
Dependencies
|
**Dependencies**
|
||||||
---------------
|
|
||||||
If your package requires another in order to build, you can specify that
|
If your package requires another in order to build, you can specify that
|
||||||
like this:
|
like this:
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
class Stackwalker(Package):
|
class Stackwalker(Package):
|
||||||
...
|
...
|
||||||
depends_on("libdwarf")
|
depends_on("libdwarf")
|
||||||
|
@ -170,8 +181,8 @@ class Stackwalker(Package):
|
||||||
it will find the dependencies automatically.
|
it will find the dependencies automatically.
|
||||||
|
|
||||||
|
|
||||||
The Install Function
|
**The Install Function**
|
||||||
----------------------
|
|
||||||
The install function is designed so that someone not too terribly familiar
|
The install function is designed so that someone not too terribly familiar
|
||||||
with Python could write a package installer. For example, we put a number
|
with Python could write a package installer. For example, we put a number
|
||||||
of commands in install scope that you can use almost like shell commands.
|
of commands in install scope that you can use almost like shell commands.
|
||||||
|
@ -195,13 +206,15 @@ class Stackwalker(Package):
|
||||||
them are created and set on the module.
|
them are created and set on the module.
|
||||||
|
|
||||||
|
|
||||||
Parallel Builds
|
**Parallel Builds**
|
||||||
-------------------
|
|
||||||
By default, Spack will run make in parallel when you run make() in your
|
By default, Spack will run make in parallel when you run make() in your
|
||||||
install function. Spack figures out how many cores are available on
|
install function. Spack figures out how many cores are available on
|
||||||
your system and runs make with -j<cores>. If you do not want this behavior,
|
your system and runs make with -j<cores>. If you do not want this behavior,
|
||||||
you can explicitly mark a package not to use parallel make:
|
you can explicitly mark a package not to use parallel make:
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
class SomePackage(Package):
|
class SomePackage(Package):
|
||||||
...
|
...
|
||||||
parallel = False
|
parallel = False
|
||||||
|
@ -210,20 +223,26 @@ class SomePackage(Package):
|
||||||
This changes thd default behavior so that make is sequential. If you still
|
This changes thd default behavior so that make is sequential. If you still
|
||||||
want to build some parts in parallel, you can do this in your install function:
|
want to build some parts in parallel, you can do this in your install function:
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
make(parallel=True)
|
make(parallel=True)
|
||||||
|
|
||||||
Likewise, if you do not supply parallel = True in your Package, you can keep
|
Likewise, if you do not supply parallel = True in your Package, you can keep
|
||||||
the default parallel behavior and run make like this when you want a
|
the default parallel behavior and run make like this when you want a
|
||||||
sequential build:
|
sequential build:
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
make(parallel=False)
|
make(parallel=False)
|
||||||
|
|
||||||
Package Lifecycle
|
**Package Lifecycle**
|
||||||
==================
|
|
||||||
This section is really only for developers of new spack commands.
|
This section is really only for developers of new spack commands.
|
||||||
|
|
||||||
A package's lifecycle over a run of Spack looks something like this:
|
A package's lifecycle over a run of Spack looks something like this:
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
p = Package() # Done for you by spack
|
p = Package() # Done for you by spack
|
||||||
|
|
||||||
p.do_fetch() # called by spack commands in spack/cmd.
|
p.do_fetch() # called by spack commands in spack/cmd.
|
||||||
|
@ -232,6 +251,9 @@ class SomePackage(Package):
|
||||||
p.do_uninstall()
|
p.do_uninstall()
|
||||||
|
|
||||||
There are also some other commands that clean the build area:
|
There are also some other commands that clean the build area:
|
||||||
|
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
p.do_clean() # runs make clean
|
p.do_clean() # runs make clean
|
||||||
p.do_clean_work() # removes the build directory and
|
p.do_clean_work() # removes the build directory and
|
||||||
# re-expands the archive.
|
# re-expands the archive.
|
||||||
|
@ -244,6 +266,7 @@ class SomePackage(Package):
|
||||||
|
|
||||||
Package creators override functions like install() (all of them do this),
|
Package creators override functions like install() (all of them do this),
|
||||||
clean() (some of them do this), and others to provide custom behavior.
|
clean() (some of them do this), and others to provide custom behavior.
|
||||||
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
#
|
#
|
||||||
|
|
|
@ -68,9 +68,11 @@ def _caller_locals():
|
||||||
del stack
|
del stack
|
||||||
|
|
||||||
|
|
||||||
def _ensure_caller_is_spack_package():
|
def _get_calling_package_name():
|
||||||
"""Make sure that the caller is a spack package. If it's not,
|
"""Make sure that the caller is a class definition, and return
|
||||||
raise ScopeError. if it is, return its name."""
|
the module's name. This is useful for getting the name of
|
||||||
|
spack packages from inside a relation function.
|
||||||
|
"""
|
||||||
stack = inspect.stack()
|
stack = inspect.stack()
|
||||||
try:
|
try:
|
||||||
# get calling function name (the relation)
|
# get calling function name (the relation)
|
||||||
|
@ -85,9 +87,6 @@ def _ensure_caller_is_spack_package():
|
||||||
raise ScopeError(relation)
|
raise ScopeError(relation)
|
||||||
|
|
||||||
module_name = caller_locals['__module__']
|
module_name = caller_locals['__module__']
|
||||||
if not module_name.startswith(packages_module()):
|
|
||||||
raise ScopeError(relation)
|
|
||||||
|
|
||||||
base_name = module_name.split('.')[-1]
|
base_name = module_name.split('.')[-1]
|
||||||
return base_name
|
return base_name
|
||||||
|
|
||||||
|
@ -121,7 +120,7 @@ def _parse_local_spec(spec_like, pkg_name):
|
||||||
"""Adds a dependencies local variable in the locals of
|
"""Adds a dependencies local variable in the locals of
|
||||||
the calling class, based on args. """
|
the calling class, based on args. """
|
||||||
def depends_on(*specs):
|
def depends_on(*specs):
|
||||||
pkg = _ensure_caller_is_spack_package()
|
pkg = _get_calling_package_name()
|
||||||
|
|
||||||
dependencies = _caller_locals().setdefault('dependencies', {})
|
dependencies = _caller_locals().setdefault('dependencies', {})
|
||||||
for string in specs:
|
for string in specs:
|
||||||
|
@ -136,7 +135,7 @@ def provides(*specs, **kwargs):
|
||||||
'mpi', other packages can declare that they depend on "mpi", and spack
|
'mpi', other packages can declare that they depend on "mpi", and spack
|
||||||
can use the providing package to satisfy the dependency.
|
can use the providing package to satisfy the dependency.
|
||||||
"""
|
"""
|
||||||
pkg = _ensure_caller_is_spack_package()
|
pkg = _get_calling_package_name()
|
||||||
spec_string = kwargs.get('when', pkg)
|
spec_string = kwargs.get('when', pkg)
|
||||||
provider_spec = _parse_local_spec(spec_string, pkg)
|
provider_spec = _parse_local_spec(spec_string, pkg)
|
||||||
|
|
||||||
|
@ -170,7 +169,7 @@ class ScopeError(RelationError):
|
||||||
def __init__(self, relation):
|
def __init__(self, relation):
|
||||||
super(ScopeError, self).__init__(
|
super(ScopeError, self).__init__(
|
||||||
relation,
|
relation,
|
||||||
"Cannot inovke '%s' from outside of a Spack package!" % relation)
|
"Must invoke '%s' from inside a class definition!" % relation)
|
||||||
|
|
||||||
|
|
||||||
class CircularReferenceError(RelationError):
|
class CircularReferenceError(RelationError):
|
||||||
|
|
|
@ -6,8 +6,10 @@
|
||||||
|
|
||||||
The syntax looks like this:
|
The syntax looks like this:
|
||||||
|
|
||||||
spack install mpileaks ^openmpi @1.2:1.4 +debug %intel @12.1
|
.. code-block:: sh
|
||||||
0 1 2 3 4 5
|
|
||||||
|
$ spack install mpileaks ^openmpi @1.2:1.4 +debug %intel @12.1 =bgqos_0
|
||||||
|
0 1 2 3 4 5 6
|
||||||
|
|
||||||
The first part of this is the command, 'spack install'. The rest of the
|
The first part of this is the command, 'spack install'. The rest of the
|
||||||
line is a spec for a particular installation of the mpileaks package.
|
line is a spec for a particular installation of the mpileaks package.
|
||||||
|
@ -37,7 +39,10 @@
|
||||||
if it comes immediately after the compiler name. Otherwise it will be
|
if it comes immediately after the compiler name. Otherwise it will be
|
||||||
associated with the current package spec.
|
associated with the current package spec.
|
||||||
|
|
||||||
Here is the EBNF grammar for a spec:
|
6. The architecture to build with. This is needed on machines where
|
||||||
|
cross-compilation is required
|
||||||
|
|
||||||
|
Here is the EBNF grammar for a spec::
|
||||||
|
|
||||||
spec-list = { spec [ dep-list ] }
|
spec-list = { spec [ dep-list ] }
|
||||||
dep_list = { ^ spec }
|
dep_list = { ^ spec }
|
||||||
|
@ -80,8 +85,7 @@
|
||||||
|
|
||||||
"""This map determines the coloring of specs when using color output.
|
"""This map determines the coloring of specs when using color output.
|
||||||
We make the fields different colors to enhance readability.
|
We make the fields different colors to enhance readability.
|
||||||
See spack.color for descriptions of the color codes.
|
See spack.color for descriptions of the color codes. """
|
||||||
"""
|
|
||||||
color_formats = {'%' : '@g', # compiler
|
color_formats = {'%' : '@g', # compiler
|
||||||
'@' : '@c', # version
|
'@' : '@c', # version
|
||||||
'=' : '@m', # architecture
|
'=' : '@m', # architecture
|
||||||
|
@ -129,8 +133,7 @@ def __call__(self, match):
|
||||||
class Compiler(object):
|
class Compiler(object):
|
||||||
"""The Compiler field represents the compiler or range of compiler
|
"""The Compiler field represents the compiler or range of compiler
|
||||||
versions that a package should be built with. Compilers have a
|
versions that a package should be built with. Compilers have a
|
||||||
name and a version list.
|
name and a version list. """
|
||||||
"""
|
|
||||||
def __init__(self, name, version=None):
|
def __init__(self, name, version=None):
|
||||||
if name not in spack.compilers.supported_compilers():
|
if name not in spack.compilers.supported_compilers():
|
||||||
raise UnknownCompilerError(name)
|
raise UnknownCompilerError(name)
|
||||||
|
@ -348,6 +351,7 @@ def package(self):
|
||||||
@property
|
@property
|
||||||
def virtual(self):
|
def virtual(self):
|
||||||
"""Right now, a spec is virtual if no package exists with its name.
|
"""Right now, a spec is virtual if no package exists with its name.
|
||||||
|
|
||||||
TODO: revisit this -- might need to use a separate namespace and
|
TODO: revisit this -- might need to use a separate namespace and
|
||||||
be more explicit about this.
|
be more explicit about this.
|
||||||
Possible idea: just use conventin and make virtual deps all
|
Possible idea: just use conventin and make virtual deps all
|
||||||
|
@ -463,7 +467,9 @@ def _expand_virtual_packages(self):
|
||||||
and normalize again to include the provider's (potentially virtual)
|
and normalize again to include the provider's (potentially virtual)
|
||||||
dependencies. Repeat until there are no virtual deps.
|
dependencies. Repeat until there are no virtual deps.
|
||||||
|
|
||||||
TODO: If a provider depends on something that conflicts with
|
.. todo::
|
||||||
|
|
||||||
|
If a provider depends on something that conflicts with
|
||||||
other dependencies in the spec being expanded, this can
|
other dependencies in the spec being expanded, this can
|
||||||
produce a conflicting spec. For example, if mpich depends
|
produce a conflicting spec. For example, if mpich depends
|
||||||
on hwloc@:1.3 but something in the spec needs hwloc1.4:,
|
on hwloc@:1.3 but something in the spec needs hwloc1.4:,
|
||||||
|
@ -625,13 +631,16 @@ def normalize(self):
|
||||||
"""When specs are parsed, any dependencies specified are hanging off
|
"""When specs are parsed, any dependencies specified are hanging off
|
||||||
the root, and ONLY the ones that were explicitly provided are there.
|
the root, and ONLY the ones that were explicitly provided are there.
|
||||||
Normalization turns a partial flat spec into a DAG, where:
|
Normalization turns a partial flat spec into a DAG, where:
|
||||||
1) ALL dependencies of the root package are in the DAG.
|
|
||||||
2) Each node's dependencies dict only contains its direct deps.
|
1. ALL dependencies of the root package are in the DAG.
|
||||||
3) There is only ONE unique spec for each package in the DAG.
|
2. Each node's dependencies dict only contains its direct deps.
|
||||||
- This includes virtual packages. If there a non-virtual
|
3. There is only ONE unique spec for each package in the DAG.
|
||||||
|
|
||||||
|
* This includes virtual packages. If there a non-virtual
|
||||||
package that provides a virtual package that is in the spec,
|
package that provides a virtual package that is in the spec,
|
||||||
then we replace the virtual package with the non-virtual one.
|
then we replace the virtual package with the non-virtual one.
|
||||||
4) The spec DAG matches package DAG.
|
|
||||||
|
4. The spec DAG matches package DAG.
|
||||||
|
|
||||||
TODO: normalize should probably implement some form of cycle detection,
|
TODO: normalize should probably implement some form of cycle detection,
|
||||||
to ensure that the spec is actually a DAG.
|
to ensure that the spec is actually a DAG.
|
||||||
|
|
|
@ -18,12 +18,16 @@ class Stage(object):
|
||||||
expanded, and built before being installed. It also handles downloading
|
expanded, and built before being installed. It also handles downloading
|
||||||
the archive. A stage's lifecycle looks like this:
|
the archive. A stage's lifecycle looks like this:
|
||||||
|
|
||||||
setup() Create the stage directory.
|
setup()
|
||||||
fetch() Fetch a source archive into the stage.
|
Create the stage directory.
|
||||||
expand_archive() Expand the source archive.
|
fetch()
|
||||||
<install> Build and install the archive. This is handled
|
Fetch a source archive into the stage.
|
||||||
by the Package class.
|
expand_archive()
|
||||||
destroy() Remove the stage once the package has been installed.
|
Expand the source archive.
|
||||||
|
<install>
|
||||||
|
Build and install the archive. This is handled by the Package class.
|
||||||
|
destroy()
|
||||||
|
Remove the stage once the package has been installed.
|
||||||
|
|
||||||
If spack.use_tmp_stage is True, spack will attempt to create stages
|
If spack.use_tmp_stage is True, spack will attempt to create stages
|
||||||
in a tmp directory. Otherwise, stages are created directly in
|
in a tmp directory. Otherwise, stages are created directly in
|
||||||
|
|
|
@ -9,7 +9,7 @@
|
||||||
A list of Versions and VersionRanges.
|
A list of Versions and VersionRanges.
|
||||||
|
|
||||||
All of these types support the following operations, which can
|
All of these types support the following operations, which can
|
||||||
be called on any of the types:
|
be called on any of the types::
|
||||||
|
|
||||||
__eq__, __ne__, __lt__, __gt__, __ge__, __le__, __hash__
|
__eq__, __ne__, __lt__, __gt__, __ge__, __le__, __hash__
|
||||||
__contains__
|
__contains__
|
||||||
|
@ -18,8 +18,6 @@
|
||||||
union
|
union
|
||||||
intersection
|
intersection
|
||||||
concrete
|
concrete
|
||||||
True if the Version, VersionRange or VersionList represents
|
|
||||||
a single version.
|
|
||||||
"""
|
"""
|
||||||
import os
|
import os
|
||||||
import sys
|
import sys
|
||||||
|
|
Loading…
Reference in a new issue