hlrs-software-stack/spack
7
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Updated docs.

Browse source
This commit is contained in:
Todd Gamblin 2014-06-24 17:01:29 -07:00
parent bf498991d3
commit 3b1e6d4bbc
3 changed files with 238 additions and 93 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

112
lib/spack/docs/basic_usage.rst
View file

@ -78,18 +78,110 @@ Packages like ``mpich`` use traditional version numbers like
like ``20130729``. Versions can contain numbers, letters, dashes,
underscores, and periods.
``spack compilers``
~~~~~~~~~~~~~~~~~~~~~~~~
You can see supported compilers by running ``spack compilers``. The
output will depend on the platform you run it on.
.. command-output:: spack compilers
Seeing installed packages
Compiler Configuration
-----------------------------------
Spack has the ability to build packages with multiple compilers and
compiler versions. Spack searches for compilers on your machine
automatically the first time it is run. It does this by inspecting
your path.
``spack compilers``
~~~~~~~~~~~~~~~~~~~~~~~
You can see which compilers spack has found by running ``spack
compilers`` or ``spack compiler list``::
$ spack compilers
==> Available compilers
-- gcc ---------------------------------------------------------
gcc@4.9.0 gcc@4.8.0 gcc@4.7.0 gcc@4.6.2 gcc@4.4.7
gcc@4.8.2 gcc@4.7.1 gcc@4.6.3 gcc@4.6.1 gcc@4.1.2
-- intel -------------------------------------------------------
intel@15.0.0 intel@14.0.0 intel@13.0.0 intel@12.1.0 intel@10.0
intel@14.0.3 intel@13.1.1 intel@12.1.5 intel@12.0.4 intel@9.1
intel@14.0.2 intel@13.1.0 intel@12.1.3 intel@11.1
intel@14.0.1 intel@13.0.1 intel@12.1.2 intel@10.1
-- clang -------------------------------------------------------
clang@3.4 clang@3.3 clang@3.2 clang@3.1
-- pgi ---------------------------------------------------------
pgi@14.3-0 pgi@13.2-0 pgi@12.1-0 pgi@10.9-0 pgi@8.0-1
pgi@13.10-0 pgi@13.1-1 pgi@11.10-0 pgi@10.2-0 pgi@7.1-3
pgi@13.6-0 pgi@12.8-0 pgi@11.1-0 pgi@9.0-4 pgi@7.0-6
Any of these compilers can be used to build Spack packages. More on
how this is done is in :ref:`sec-specs`.
``spack compiler add``
~~~~~~~~~~~~~~~~~~~~~~~
If you do not see a compiler in this list, but you want to use it with
Spack, you can simply run ``spack compiler add`` with the path to
where the compiler is installed. For example::
$ spack compiler add /usr/local/tools/ic-13.0.079
==> Added 1 new compiler to /Users/gamblin2/.spackconfig
intel@13.0.079
Or you can run ``spack compiler add`` with no arguments to force
autodetection. This is useful if you do not know where compilers
live, but new compilers have been added to your ``PATH``. For
example, using dotkit, you might do this::
$ use gcc-4.9.0
$ spack compiler add
==> Added 1 new compiler to /Users/gamblin2/.spackconfig
gcc@4.9.0
``spack compiler info``
~~~~~~~~~~~~~~~~~~~~~~~
If you want to see specifics on a particular compiler, you can run
``spack compiler info`` on it::
$ spack compiler info intel@12.1.3
intel@12.1.3:
cc = /usr/local/bin/icc-12.1.293
cxx = /usr/local/bin/icpc-12.1.293
f77 = /usr/local/bin/ifort-12.1.293
fc = /usr/local/bin/ifort-12.1.293
This shows which C, C++, and Fortran compilers were detected by Spack.
Manual configuration
~~~~~~~~~~~~~~~~~~~~~~~
If autodetection fails, you can manually conigure a compiler by
editing your ``~/.spackconfig`` file. You can do this by running
``spack config edit``, which will open the file in your ``$EDITOR``.
Each compiler configuration in the file looks like this::
...
[compiler "intel@15.0.0"]
cc = /usr/local/bin/icc-15.0.024-beta
cxx = /usr/local/bin/icpc-15.0.024-beta
f77 = /usr/local/bin/ifort-15.0.024-beta
fc = /usr/local/bin/ifort-15.0.024-beta
...
For compilers, like ``clang``, that do not support Fortran, you can simply
put ``None`` for ``f77`` and ``fc``::
[compiler "clang@3.3svn"]
cc = /usr/bin/clang
cxx = /usr/bin/clang++
f77 = None
fc = None
Once you save the file, the configured compilers will show up in the
list displayed when you run ``spack compilers``.
Seeing installed packages -----------------------------------
``spack find``
~~~~~~~~~~~~~~~~~~~~~~

8
lib/spack/docs/packaging_guide.rst
View file

@ -753,8 +753,6 @@ This is due to Spack's compiler interceptors.
Environment variables
~~~~~~~~~~~~~~~~~~~~~~~~~~
.. warning:: Environment variable setting is not fully implemented.
Spack sets a number of standard environment variables so that build
systems use its compiler wrappers for their builds. The standard
enviroment variables are:
@ -764,10 +762,8 @@ enviroment variables are:
======================= =============================
``CC`` C compiler
``CXX`` C++ compiler
``CPP`` C Preprocessor
``F77`` Fortran 77 compiler
``F90`` Fortran 90 compiler
``F95`` Fortran 95 compiler
``FC`` Fortran 90 and above compiler
``CMAKE_PREFIX_PATH`` Path to dependency prefixes for CMake
======================= =============================
@ -787,8 +783,6 @@ correct compilers into the project's build system.
Forked process
~~~~~~~~~~~~~~~~~~~~~
.. warning:: This is not implemented yet.
To give packages free reign over how they install things, how they
modify the environemnt, and how they use Spack's internal APIs, we
fork a new process each time we invoke ``install()``. This allows

211
lib/spack/docs/site_configuration.rst
View file

@ -8,6 +8,10 @@ Site-specific configuration
Temporary space
----------------------------
.. deprecated::
This will be moved to configuration files.
By default, Spack will try to do all of its building in temporary
space. There are two main reasons for this. First, Spack is designed
to run out of a user's home directory, and on may systems the home
@ -18,7 +22,7 @@ this. Second, many systems impose quotas on home directories, and
helps conserve space for installations in users' home directories.
You can customize temporary directories by editing
``lib/spack/spack/globals.py``. Specifically, find this part of the file:
``lib/spack/spack/__init__.py``. Specifically, find this part of the file:
.. code-block:: python
@ -32,7 +36,7 @@ You can customize temporary directories by editing
# is a shared filesystem. Spack will use the first of these paths
# that it can create.
tmp_dirs = ['/nfs/tmp2/%u/spack-stage',
'/var/tmp/%u/spcak-stage',
'/var/tmp/%u/spack-stage',
'/tmp/%u/spack-stage']
The ``use_tmp_stage`` variable controls whether Spack builds
@ -91,100 +95,155 @@ package's fetch URL.
``spack mirror``
~~~~~~~~~~~~~~~~~~~~~~~
You can create a mirror using the ``spack mirror`` command, assuming
you're on a machine where you can access the internet. ``spack
mirror`` will iterate through all of Spack's packages and download the
safe ones into a directory structure like the one above. Here is what
it looks like:
Mirrors are managed with the ``spack mirror`` command. The help for
``spack mirror`` looks like this::
$ spack mirror -h
usage: spack mirror [-h] SUBCOMMAND ...
positional arguments:
SUBCOMMAND
create Create a directory to be used as a spack mirror, and fill
it with package archives.
add Add a mirror to Spack.
remove Remove a mirror by name.
list Print out available mirrors to the console.
optional arguments:
-h, --help show this help message and exit
The ``create`` command actually builds a mirror by fetching all of its
packages from the internet and checksumming them.
The other three commands are for managing mirror configuration. They
control the URL(s) from which Spack downloads its packages.
``spack mirror create``
~~~~~~~~~~~~~~~~~~~~~~~
You can create a mirror using the ``spack mirror create`` command, assuming
you're on a machine where you can access the internet.
The command will iterate through all of Spack's packages and download
the safe ones into a directory structure like the one above. Here is
what it looks like:
.. code-block:: bash
$ spack mirror mirror-dir
==> No safe (checksummed) versions for package callpath. Skipping.
==> Trying to fetch from http://www.cmake.org/files/v2.8/cmake-2.8.10.2.tar.gz
################################################################ 90.2%
==> Added mirror-dir/cmake/cmake-2.8.10.2.tar.gz to mirror
==> Trying to fetch from http://www.dyninst.org/sites/default/files/downloads/dyninst/8.1.2/DyninstAPI-8.1.2.tgz
########################################################### 82.0%
==> Added mirror-dir/dyninst/DyninstAPI-8.1.2.tgz to mirror
==> Trying to fetch from http://www.dyninst.org/sites/default/files/downloads/dyninst/8.1.2/DyninstAPI-8.1.1.tgz
######################################################################## 100.0%
==> Added mirror-dir/dyninst/DyninstAPI-8.1.1.tgz to mirror
...
$ spack mirror create libelf libdwarf
==> Created new mirror in spack-mirror-2014-06-24
==> Trying to fetch from http://www.mr511.de/software/libelf-0.8.13.tar.gz
########################################################## 81.6%
==> Checksum passed for libelf@0.8.13
==> Added spack-mirror-2014-06-24/libelf/libelf-0.8.13.tar.gz to mirror
==> Trying to fetch from http://www.mr511.de/software/libelf-0.8.12.tar.gz
############################################################## 86.5%
==> Added mirror-dir/libelf/libelf-0.8.12.tar.gz to mirror
==> Trying to fetch from http://download.savannah.gnu.org/releases/libunwind/libunwind-1.1.tar.gz
################################################################ 89.3%
==> Added mirror-dir/libunwind/libunwind-1.1.tar.gz to mirror
==> Trying to fetch from http://www.mpich.org/static/downloads/3.0.4/mpich-3.0.4.tar.gz
##################################################################### 96.4%
==> Added mirror-dir/mpich/mpich-3.0.4.tar.gz to mirror
==> No safe (checksummed) versions for package mpileaks. Skipping.
==> Trying to fetch from http://mvapich.cse.ohio-state.edu/download/mvapich2/mv2/mvapich2-1.9.tgz
####################################################################### 99.2%
==> Added mirror-dir/mvapich2/mvapich2-1.9.tgz to mirror
==> Created Spack mirror in mirror-dir
###################################################################### 98.6%
==> Checksum passed for libelf@0.8.12
==> Added spack-mirror-2014-06-24/libelf/libelf-0.8.12.tar.gz to mirror
==> Trying to fetch from http://www.prevanders.net/libdwarf-20130207.tar.gz
###################################################################### 97.3%
==> Checksum passed for libdwarf@20130207
==> Added spack-mirror-2014-06-24/libdwarf/libdwarf-20130207.tar.gz to mirror
==> Trying to fetch from http://www.prevanders.net/libdwarf-20130126.tar.gz
######################################################## 78.9%
==> Checksum passed for libdwarf@20130126
==> Added spack-mirror-2014-06-24/libdwarf/libdwarf-20130126.tar.gz to mirror
==> Trying to fetch from http://www.prevanders.net/libdwarf-20130729.tar.gz
############################################################# 84.7%
==> Checksum passed for libdwarf@20130729
==> Added spack-mirror-2014-06-24/libdwarf/libdwarf-20130729.tar.gz to mirror
Once this is done, you can tar up the ``mirror-dir`` directory and
Once this is done, you can tar up the ``spack-mirror-2014-06-24`` directory and
copy it over to the machine you want it hosted on.
Normally, ``spack mirror`` downloads all the archives it has checksums
for. If you want to only create a mirror for a subset of packages,
you can do that by supplying a list of package names on the command
line after ``spack mirror``.
Custom package sets
^^^^^^^^^^^^^^^^^^^^^^^^
Normally, ``spack mirror create`` downloads all the archives it has
checksums for. If you want to only create a mirror for a subset of
packages, you can do that by supplying a list of package specs on the
command line after ``spack mirror create``. For example, this
command::
$ spack mirror create libelf@0.8.12: boost@1.44:
Will create a mirror for libelf versions greater than or equal to
0.8.12 and boost versions greater than or equal to 1.44.
Mirror files
^^^^^^^^^^^^^^^^^^^^^^^^
If you have a *very* large number of packages you want to mirror, you
can supply a file with specs in it, one per line::
$ cat specs.txt
libdwarf
libelf@0.8.12:
boost@1.44:
boost@1.39.0
...
$ spack mirror create -f specs.txt
...
This is useful if there is a specific suite of software managed by
your site.
Setting up a mirror
``spack mirror add``
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Once you have a mirrror, you need to let spack know about it. Find
this section in ``globals.py``:
Once you have a mirrror, you need to let spack know about it. This is
relatively simple. First, figure out the URL for the mirror. If it's
a file, you can use a file URL like this one::
.. code-block:: python
file:///Users/gamblin2/spack-mirror-2014-06-24
#
# Places to download tarballs from. Examples:
#
# For a local directory:
# mirrors = ['file:///Users/gamblin2/spack-mirror']
#
# For a website:
# mirrors = ['http://spackports.org/spack-mirror/']
#
# For no mirrors:
# mirrors = []
#
mirrors = []
That points to the directory on the local filesystem. If it were on a
web server, you could use a URL like this one:
Change the list of mirrors to include the location where you copied
your directory created by ``spack mirror``. If it's on a local
filesystem, you want to use a ``file://`` URL. If it's on a private
web server, you will need to use a ``http://`` or ``https://`` URL.
https://example.com/some/web-hosted/directory/spack-mirror-2014-06-24
Spack will use the URL as the root for all of the packages it fetches.
You can tell your Spack installation to use that mirror like this:
.. code-block:: bash
$ spack mirror add local_filesystem file:///Users/gamblin2/spack-mirror-2014-06-24
Each mirror has a name so that you can refer to it again later.
``spack mirror list``
~~~~~~~~~~~~~~~~~~~~~~~~~~~
If you want to see all the mirrors Spack knows about you can run ``spack mirror list``::
$ spack mirror list
local_filesystem file:///Users/gamblin2/spack-mirror-2014-06-24
``spack mirror remove``
~~~~~~~~~~~~~~~~~~~~~~~~~~~
And, if you want to remove a mirror, just remove it by name::
$ spack mirror remove local_filesystem
$ spack mirror list
==> No mirrors configured.
Mirror precedence
~~~~~~~~~~~~~~~~~~~~~~~~~
If you have specified mirrors in ``globals.py``, then Spack will try
to find an archive in each mirror in the list, in order, before it
downloads from the URL in a package file.
Adding a mirror really just adds a section in ``~/.spackconfig``::
You can test whether a mirror is working properly by first setting it
in ``globals.py``, then running ``spack fetch`` to test fetching the
archive. Example:
.. code-block:: bash
$ spack fetch dyninst
==> Trying to fetch from file:///Users/gamblin2/mirror-dir/dyninst/DyninstAPI-8.1.2.tgz
==> Checksum passed for dyninst
If the mirror setup worked, you should see the mirror URL in the fetch
output, like the ``file://`` URL above.
[mirror "local_filesystem"]
url = file:///Users/gamblin2/spack-mirror-2014-06-24
[mirror "remote_server"]
url = https://example.com/some/web-hosted/directory/spack-mirror-2014-06-24
If you want to change the order in which mirrors are searched for
packages, you can edit this file and reorder the sections. Spack will
search the topmost mirror first and the bottom-most mirror last.
.. _concretization-policies: