Update documentation for use on Cray
Updating how to use compilers.yaml on Cray as well as setting up external packages. Also includes what needs to be set up for usage on Cray and explains what needs to be done to get Spack properly working. Also explain the architecture spec and what it does.
This commit is contained in:
Mario Melara
parent
3ea9d1e665
commit
7fb45e4bfd
2 changed files with 152 additions and 70 deletions
|
|
@ -114,13 +114,13 @@ that the packages is installed:
|
||||||
|
|
||||||
$ spack install mpileaks
|
$ spack install mpileaks
|
||||||
==> Installing mpileaks
|
==> Installing mpileaks
|
||||||
==> mpich is already installed in /home/gamblin2/spack/opt/chaos_5_x86_64_ib/gcc@4.4.7/mpich@3.0.4.
|
==> mpich is already installed in /home/gamblin2/spack/opt/linux-x86_64-debian7/gcc@4.4.7/mpich@3.0.4.
|
||||||
==> callpath is already installed in /home/gamblin2/spack/opt/chaos_5_x86_64_ib/gcc@4.4.7/callpath@1.0.2-5dce4318.
|
==> callpath is already installed in /home/gamblin2/spack/opt/linux-x86_64-debian7/gcc@4.4.7/callpath@1.0.2-5dce4318.
|
||||||
==> adept-utils is already installed in /home/gamblin2/spack/opt/chaos_5_x86_64_ib/gcc@4.4.7/adept-utils@1.0-5adef8da.
|
==> adept-utils is already installed in /home/gamblin2/spack/opt/linux-x86_64-debian7/gcc@4.4.7/adept-utils@1.0-5adef8da.
|
||||||
==> Trying to fetch from https://github.com/hpc/mpileaks/releases/download/v1.0/mpileaks-1.0.tar.gz
|
==> Trying to fetch from https://github.com/hpc/mpileaks/releases/download/v1.0/mpileaks-1.0.tar.gz
|
||||||
######################################################################## 100.0%
|
######################################################################## 100.0%
|
||||||
==> Staging archive: /home/gamblin2/spack/var/spack/stage/mpileaks@1.0%gcc@4.4.7 arch=chaos_5_x86_64_ib-59f6ad23/mpileaks-1.0.tar.gz
|
==> Staging archive: /home/gamblin2/spack/var/spack/stage/mpileaks@1.0%gcc@4.4.7 arch=linux-x86_64-debian7-59f6ad23/mpileaks-1.0.tar.gz
|
||||||
==> Created stage in /home/gamblin2/spack/var/spack/stage/mpileaks@1.0%gcc@4.4.7 arch=chaos_5_x86_64_ib-59f6ad23.
|
==> Created stage in /home/gamblin2/spack/var/spack/stage/mpileaks@1.0%gcc@4.4.7 arch=linux-x86_64-debian7-59f6ad23.
|
||||||
==> No patches needed for mpileaks.
|
==> No patches needed for mpileaks.
|
||||||
==> Building mpileaks.
|
==> Building mpileaks.
|
||||||
|
|
||||||
|
|
@ -128,7 +128,7 @@ that the packages is installed:
|
||||||
|
|
||||||
==> Successfully installed mpileaks.
|
==> Successfully installed mpileaks.
|
||||||
Fetch: 2.16s. Build: 9.82s. Total: 11.98s.
|
Fetch: 2.16s. Build: 9.82s. Total: 11.98s.
|
||||||
[+] /home/gamblin2/spack/opt/chaos_5_x86_64_ib/gcc@4.4.7/mpileaks@1.0-59f6ad23
|
[+] /home/gamblin2/spack/opt/linux-x86_64-debian7/gcc@4.4.7/mpileaks@1.0-59f6ad23
|
||||||
|
|
||||||
The last line, with the ``[+]``, indicates where the package is
|
The last line, with the ``[+]``, indicates where the package is
|
||||||
installed.
|
installed.
|
||||||
|
|
@ -230,7 +230,7 @@ Running ``spack find`` with no arguments lists installed packages:
|
||||||
|
|
||||||
$ spack find
|
$ spack find
|
||||||
==> 74 installed packages.
|
==> 74 installed packages.
|
||||||
-- chaos_5_x86_64_ib / gcc@4.4.7 --------------------------------
|
-- linux-x86_64-debian7 / gcc@4.4.7 --------------------------------
|
||||||
ImageMagick@6.8.9-10 libdwarf@20130729 py-dateutil@2.4.0
|
ImageMagick@6.8.9-10 libdwarf@20130729 py-dateutil@2.4.0
|
||||||
adept-utils@1.0 libdwarf@20130729 py-ipython@2.3.1
|
adept-utils@1.0 libdwarf@20130729 py-ipython@2.3.1
|
||||||
atk@2.14.0 libelf@0.8.12 py-matplotlib@1.4.2
|
atk@2.14.0 libelf@0.8.12 py-matplotlib@1.4.2
|
||||||
|
|
@ -256,7 +256,7 @@ Running ``spack find`` with no arguments lists installed packages:
|
||||||
lcms@2.6 pixman@0.32.6 xz@5.2.0
|
lcms@2.6 pixman@0.32.6 xz@5.2.0
|
||||||
libdrm@2.4.33 py-dateutil@2.4.0 zlib@1.2.8
|
libdrm@2.4.33 py-dateutil@2.4.0 zlib@1.2.8
|
||||||
|
|
||||||
-- chaos_5_x86_64_ib / gcc@4.9.2 --------------------------------
|
-- linux-x86_64-debian7 / gcc@4.9.2 --------------------------------
|
||||||
libelf@0.8.10 mpich@3.0.4
|
libelf@0.8.10 mpich@3.0.4
|
||||||
|
|
||||||
Packages are divided into groups according to their architecture and
|
Packages are divided into groups according to their architecture and
|
||||||
|
|
@ -279,7 +279,7 @@ in more detail using ``spack find -d``, and by asking only to show
|
||||||
|
|
||||||
$ spack find --deps libdwarf
|
$ spack find --deps libdwarf
|
||||||
==> 2 installed packages.
|
==> 2 installed packages.
|
||||||
-- chaos_5_x86_64_ib / gcc@4.4.7 --------------------------------
|
-- linux-x86_64-debian7 / gcc@4.4.7 --------------------------------
|
||||||
libdwarf@20130729-d9b90962
|
libdwarf@20130729-d9b90962
|
||||||
^libelf@0.8.12
|
^libelf@0.8.12
|
||||||
libdwarf@20130729-b52fac98
|
libdwarf@20130729-b52fac98
|
||||||
|
|
@ -295,7 +295,7 @@ want to know whether two packages' dependencies differ, you can use
|
||||||
|
|
||||||
$ spack find -l libdwarf
|
$ spack find -l libdwarf
|
||||||
==> 2 installed packages.
|
==> 2 installed packages.
|
||||||
-- chaos_5_x86_64_ib / gcc@4.4.7 --------------------------------
|
-- linux-x86_64-debian7 / gcc@4.4.7 --------------------------------
|
||||||
libdwarf@20130729-d9b90962 libdwarf@20130729-b52fac98
|
libdwarf@20130729-d9b90962 libdwarf@20130729-b52fac98
|
||||||
|
|
||||||
Now the ``libwarf`` installs have hashes after their names. These are
|
Now the ``libwarf`` installs have hashes after their names. These are
|
||||||
|
|
@ -309,14 +309,14 @@ use ``spack find -p``:
|
||||||
|
|
||||||
$ spack find -p
|
$ spack find -p
|
||||||
==> 74 installed packages.
|
==> 74 installed packages.
|
||||||
-- chaos_5_x86_64_ib / gcc@4.4.7 --------------------------------
|
-- linux-x86_64-debian7 / gcc@4.4.7 --------------------------------
|
||||||
ImageMagick@6.8.9-10 /home/gamblin2/spack/opt/chaos_5_x86_64_ib/gcc@4.4.7/ImageMagick@6.8.9-10-4df950dd
|
ImageMagick@6.8.9-10 /home/gamblin2/spack/opt/linux-x86_64-debian7/gcc@4.4.7/ImageMagick@6.8.9-10-4df950dd
|
||||||
adept-utils@1.0 /home/gamblin2/spack/opt/chaos_5_x86_64_ib/gcc@4.4.7/adept-utils@1.0-5adef8da
|
adept-utils@1.0 /home/gamblin2/spack/opt/linux-x86_64-debian7/gcc@4.4.7/adept-utils@1.0-5adef8da
|
||||||
atk@2.14.0 /home/gamblin2/spack/opt/chaos_5_x86_64_ib/gcc@4.4.7/atk@2.14.0-3d09ac09
|
atk@2.14.0 /home/gamblin2/spack/opt/linux-x86_64-debian7/gcc@4.4.7/atk@2.14.0-3d09ac09
|
||||||
boost@1.55.0 /home/gamblin2/spack/opt/chaos_5_x86_64_ib/gcc@4.4.7/boost@1.55.0
|
boost@1.55.0 /home/gamblin2/spack/opt/linux-x86_64-debian7/gcc@4.4.7/boost@1.55.0
|
||||||
bzip2@1.0.6 /home/gamblin2/spack/opt/chaos_5_x86_64_ib/gcc@4.4.7/bzip2@1.0.6
|
bzip2@1.0.6 /home/gamblin2/spack/opt/linux-x86_64-debian7/gcc@4.4.7/bzip2@1.0.6
|
||||||
cairo@1.14.0 /home/gamblin2/spack/opt/chaos_5_x86_64_ib/gcc@4.4.7/cairo@1.14.0-fcc2ab44
|
cairo@1.14.0 /home/gamblin2/spack/opt/linux-x86_64-debian7/gcc@4.4.7/cairo@1.14.0-fcc2ab44
|
||||||
callpath@1.0.2 /home/gamblin2/spack/opt/chaos_5_x86_64_ib/gcc@4.4.7/callpath@1.0.2-5dce4318
|
callpath@1.0.2 /home/gamblin2/spack/opt/linux-x86_64-debian7/gcc@4.4.7/callpath@1.0.2-5dce4318
|
||||||
...
|
...
|
||||||
|
|
||||||
And, finally, you can restrict your search to a particular package
|
And, finally, you can restrict your search to a particular package
|
||||||
|
|
@ -325,10 +325,10 @@ by supplying its name:
|
||||||
.. code-block:: sh
|
.. code-block:: sh
|
||||||
|
|
||||||
$ spack find -p libelf
|
$ spack find -p libelf
|
||||||
-- chaos_5_x86_64_ib / gcc@4.4.7 --------------------------------
|
-- linux-x86_64-debian7 / gcc@4.4.7 --------------------------------
|
||||||
libelf@0.8.11 /home/gamblin2/spack/opt/chaos_5_x86_64_ib/gcc@4.4.7/libelf@0.8.11
|
libelf@0.8.11 /home/gamblin2/spack/opt/linux-x86_64-debian7/gcc@4.4.7/libelf@0.8.11
|
||||||
libelf@0.8.12 /home/gamblin2/spack/opt/chaos_5_x86_64_ib/gcc@4.4.7/libelf@0.8.12
|
libelf@0.8.12 /home/gamblin2/spack/opt/linux-x86_64-debian7/gcc@4.4.7/libelf@0.8.12
|
||||||
libelf@0.8.13 /home/gamblin2/spack/opt/chaos_5_x86_64_ib/gcc@4.4.7/libelf@0.8.13
|
libelf@0.8.13 /home/gamblin2/spack/opt/linux-x86_64-debian7/gcc@4.4.7/libelf@0.8.13
|
||||||
|
|
||||||
``spack find`` actually does a lot more than this. You can use
|
``spack find`` actually does a lot more than this. You can use
|
||||||
*specs* to query for specific configurations and builds of each
|
*specs* to query for specific configurations and builds of each
|
||||||
|
|
@ -338,7 +338,7 @@ package. If you want to find only libelf versions greater than version
|
||||||
.. code-block:: sh
|
.. code-block:: sh
|
||||||
|
|
||||||
$ spack find libelf@0.8.12:
|
$ spack find libelf@0.8.12:
|
||||||
-- chaos_5_x86_64_ib / gcc@4.4.7 --------------------------------
|
-- linux-x86_64-debian7 / gcc@4.4.7 --------------------------------
|
||||||
libelf@0.8.12 libelf@0.8.13
|
libelf@0.8.12 libelf@0.8.13
|
||||||
|
|
||||||
Finding just the versions of libdwarf built with a particular version
|
Finding just the versions of libdwarf built with a particular version
|
||||||
|
|
@ -348,7 +348,7 @@ of libelf would look like this:
|
||||||
|
|
||||||
$ spack find -l libdwarf ^libelf@0.8.12
|
$ spack find -l libdwarf ^libelf@0.8.12
|
||||||
==> 1 installed packages.
|
==> 1 installed packages.
|
||||||
-- chaos_5_x86_64_ib / gcc@4.4.7 --------------------------------
|
-- linux-x86_64-debian7 / gcc@4.4.7 --------------------------------
|
||||||
libdwarf@20130729-d9b90962
|
libdwarf@20130729-d9b90962
|
||||||
|
|
||||||
We can also search for packages that have a certain attribute. For example,
|
We can also search for packages that have a certain attribute. For example,
|
||||||
|
|
@ -460,19 +460,38 @@ editing your ``~/.spack/compilers.yaml`` file. You can do this by running
|
||||||
Each compiler configuration in the file looks like this::
|
Each compiler configuration in the file looks like this::
|
||||||
|
|
||||||
...
|
...
|
||||||
chaos_5_x86_64_ib:
|
compilers:
|
||||||
...
|
- compiler:
|
||||||
intel@15.0.0:
|
modules = []
|
||||||
|
operating_system: OS
|
||||||
|
paths:
|
||||||
cc: /usr/local/bin/icc-15.0.024-beta
|
cc: /usr/local/bin/icc-15.0.024-beta
|
||||||
cxx: /usr/local/bin/icpc-15.0.024-beta
|
cxx: /usr/local/bin/icpc-15.0.024-beta
|
||||||
f77: /usr/local/bin/ifort-15.0.024-beta
|
f77: /usr/local/bin/ifort-15.0.024-beta
|
||||||
fc: /usr/local/bin/ifort-15.0.024-beta
|
fc: /usr/local/bin/ifort-15.0.024-beta
|
||||||
|
|
||||||
|
spec: intel@15.0.0:
|
||||||
|
|
||||||
|
If you're on a Cray system, the modules array will hold the names of the
|
||||||
|
compiler module as well as the corresponding PrgEnv. For example, on Edison
|
||||||
|
at NERSC the intel compiler looks just like this::
|
||||||
|
...
|
||||||
|
- compiler:
|
||||||
|
modules:
|
||||||
|
- PrEnv-intel
|
||||||
|
- intel/15.0.109
|
||||||
...
|
...
|
||||||
|
|
||||||
The chaos_5_x86_64_ib string is an architecture string, and multiple
|
The compiler paths will also look different on a Cray system. Since most
|
||||||
compilers can be listed underneath an architecture. The architecture
|
compilers are invoked using cc, CC and ftn, the paths for each compiler are
|
||||||
string may be replaced with the string 'all' to signify compilers that
|
replaced with their respective Cray compiler wrapper names::
|
||||||
work on all architectures.
|
...
|
||||||
|
paths:
|
||||||
|
cc: cc
|
||||||
|
cxx: CC
|
||||||
|
f77: ftn
|
||||||
|
fc: ftn
|
||||||
|
...
|
||||||
|
|
||||||
For compilers, like ``clang``, that do not support Fortran, put
|
For compilers, like ``clang``, that do not support Fortran, put
|
||||||
``None`` for ``f77`` and ``fc``::
|
``None`` for ``f77`` and ``fc``::
|
||||||
|
|
@ -488,10 +507,11 @@ list displayed by ``spack compilers``.
|
||||||
|
|
||||||
You can also add compiler flags to manually configured compilers. The
|
You can also add compiler flags to manually configured compilers. The
|
||||||
valid flags are ``cflags``, ``cxxflags``, ``fflags``, ``cppflags``,
|
valid flags are ``cflags``, ``cxxflags``, ``fflags``, ``cppflags``,
|
||||||
``ldflags``, and ``ldlibs``. For example,::
|
``ldflags``, and ``ldlibs``. For example::
|
||||||
|
|
||||||
...
|
...
|
||||||
chaos_5_x86_64_ib:
|
compilers:
|
||||||
|
- compiler:
|
||||||
...
|
...
|
||||||
intel@15.0.0:
|
intel@15.0.0:
|
||||||
cc: /usr/local/bin/icc-15.0.024-beta
|
cc: /usr/local/bin/icc-15.0.024-beta
|
||||||
|
|
@ -546,8 +566,8 @@ More formally, a spec consists of the following pieces:
|
||||||
boolean variants
|
boolean variants
|
||||||
* ``name=<value>`` Optional compiler flag specifiers. Valid flag names are
|
* ``name=<value>`` Optional compiler flag specifiers. Valid flag names are
|
||||||
``cflags``, ``cxxflags``, ``fflags``, ``cppflags``, ``ldflags``, and ``ldlibs``.
|
``cflags``, ``cxxflags``, ``fflags``, ``cppflags``, ``ldflags``, and ``ldlibs``.
|
||||||
* ``arch=<value>`` Optional architecture specifier (``arch=bgq_os``)
|
* ``target=<value> os=<value>`` Optional architecture specifier
|
||||||
* ``^`` Dependency specs (``^callpath@1.1``)
|
(``target=haswell os=CNL10``) * ``^`` Dependency specs (``^callpath@1.1``)
|
||||||
|
|
||||||
There are two things to notice here. The first is that specs are
|
There are two things to notice here. The first is that specs are
|
||||||
recursively defined. That is, each dependency after ``^`` is a spec
|
recursively defined. That is, each dependency after ``^`` is a spec
|
||||||
|
|
@ -626,7 +646,7 @@ compilers, variants, and architectures just like any other spec.
|
||||||
Specifiers are associated with the nearest package name to their left.
|
Specifiers are associated with the nearest package name to their left.
|
||||||
For example, above, ``@1.1`` and ``%gcc@4.7.2`` associates with the
|
For example, above, ``@1.1`` and ``%gcc@4.7.2`` associates with the
|
||||||
``callpath`` package, while ``@1.2:1.4``, ``%gcc@4.7.5``, ``+debug``,
|
``callpath`` package, while ``@1.2:1.4``, ``%gcc@4.7.5``, ``+debug``,
|
||||||
``-qt``, and ``arch=bgq_os`` all associate with the ``mpileaks`` package.
|
``-qt``, and ``target=haswell os=CNL10`` all associate with the ``mpileaks`` package.
|
||||||
|
|
||||||
In the diagram above, ``mpileaks`` depends on ``mpich`` with an
|
In the diagram above, ``mpileaks`` depends on ``mpich`` with an
|
||||||
unspecified version, but packages can depend on other packages with
|
unspecified version, but packages can depend on other packages with
|
||||||
|
|
@ -758,14 +778,20 @@ in gnu autotools. If all flags are set, the order is
|
||||||
Architecture specifiers
|
Architecture specifiers
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
.. Note::
|
|
||||||
|
|
||||||
Architecture specifiers are part of specs but are not yet
|
|
||||||
functional. They will be in Spack version 1.0, due in Q3 2015.
|
|
||||||
|
|
||||||
The architecture specifier looks identical to a variant specifier for a
|
The architecture specifier looks identical to a variant specifier for a
|
||||||
non-boolean variant. The architecture can be specified only using the
|
non-boolean variant. The architecture can be specified by using the reserved
|
||||||
reserved name ``arch`` (``arch=bgq_os``).
|
words ``target`` and/or ``os`` (``target=x86-64 os=debian7``).
|
||||||
|
|
||||||
|
If you are on a Cray system, you can specify which target processor to
|
||||||
|
build with. For example, if you want to build against a compute node processor
|
||||||
|
with the compute node operating system, you would specify
|
||||||
|
``target=haswell os=CNL10``. Spack will then load the appropriate module for
|
||||||
|
the target. Additionally, Spack can also intepret the following values:
|
||||||
|
``be, backend, fe, frontend``. Backend is used for specifying the compute-node
|
||||||
|
processor and operating sytem, and frontend is used for login nodes.
|
||||||
|
If you decide to leave this field empty, Spack will use the
|
||||||
|
default architecture (compute nodes). The architecture spec is displayed as a
|
||||||
|
triplet of platform-target-operating_system. (``arch=linux-x86_64-debian7``)
|
||||||
|
|
||||||
|
|
||||||
.. _sec-virtual-dependencies:
|
.. _sec-virtual-dependencies:
|
||||||
|
|
@ -985,7 +1011,7 @@ of installed packages.
|
||||||
|
|
||||||
$ module avail
|
$ module avail
|
||||||
|
|
||||||
------- /home/gamblin2/spack/share/spack/modules/chaos_5_x86_64_ib --------
|
------- /home/gamblin2/spack/share/spack/modules/linux-x86_64-debian7 --------
|
||||||
adept-utils@1.0%gcc@4.4.7-5adef8da libelf@0.8.13%gcc@4.4.7
|
adept-utils@1.0%gcc@4.4.7-5adef8da libelf@0.8.13%gcc@4.4.7
|
||||||
automaded@1.0%gcc@4.4.7-d9691bb0 libelf@0.8.13%intel@15.0.0
|
automaded@1.0%gcc@4.4.7-d9691bb0 libelf@0.8.13%intel@15.0.0
|
||||||
boost@1.55.0%gcc@4.4.7 mpc@1.0.2%gcc@4.4.7-559607f5
|
boost@1.55.0%gcc@4.4.7 mpc@1.0.2%gcc@4.4.7-559607f5
|
||||||
|
|
@ -1056,7 +1082,7 @@ Spack. For example, this will add the ``mpich`` package built with
|
||||||
$ spack use mpich %gcc@4.4.7
|
$ spack use mpich %gcc@4.4.7
|
||||||
Prepending: mpich@3.0.4%gcc@4.4.7 (ok)
|
Prepending: mpich@3.0.4%gcc@4.4.7 (ok)
|
||||||
$ which mpicc
|
$ which mpicc
|
||||||
~/src/spack/opt/chaos_5_x86_64_ib/gcc@4.4.7/mpich@3.0.4/bin/mpicc
|
~/src/spack/opt/linux-x86_64-debian7/gcc@4.4.7/mpich@3.0.4/bin/mpicc
|
||||||
|
|
||||||
Or, similarly with modules, you could type:
|
Or, similarly with modules, you could type:
|
||||||
|
|
||||||
|
|
@ -1089,8 +1115,8 @@ than one installed package matches it), then Spack will warn you:
|
||||||
|
|
||||||
$ spack load libelf
|
$ spack load libelf
|
||||||
==> Error: Multiple matches for spec libelf. Choose one:
|
==> Error: Multiple matches for spec libelf. Choose one:
|
||||||
libelf@0.8.13%gcc@4.4.7 arch=chaos_5_x86_64_ib
|
libelf@0.8.13%gcc@4.4.7 arch=linux-x86_64-debian7
|
||||||
libelf@0.8.13%intel@15.0.0 arch=chaos_5_x86_64_ib
|
libelf@0.8.13%intel@15.0.0 arch=linux-x86_64-debian7
|
||||||
|
|
||||||
You can either type the ``spack load`` command again with a fully
|
You can either type the ``spack load`` command again with a fully
|
||||||
qualified argument, or you can add just enough extra constraints to
|
qualified argument, or you can add just enough extra constraints to
|
||||||
|
|
@ -1470,7 +1496,7 @@ an *extension*. Suppose you have Python installed like so:
|
||||||
|
|
||||||
$ spack find python
|
$ spack find python
|
||||||
==> 1 installed packages.
|
==> 1 installed packages.
|
||||||
-- chaos_5_x86_64_ib / gcc@4.4.7 --------------------------------
|
-- linux-x86_64-debian7 / gcc@4.4.7 --------------------------------
|
||||||
python@2.7.8
|
python@2.7.8
|
||||||
|
|
||||||
.. _spack-extensions:
|
.. _spack-extensions:
|
||||||
|
|
@ -1483,7 +1509,7 @@ You can find extensions for your Python installation like this:
|
||||||
.. code-block:: sh
|
.. code-block:: sh
|
||||||
|
|
||||||
$ spack extensions python
|
$ spack extensions python
|
||||||
==> python@2.7.8%gcc@4.4.7 arch=chaos_5_x86_64_ib-703c7a96
|
==> python@2.7.8%gcc@4.4.7 arch=linux-x86_64-debian7-703c7a96
|
||||||
==> 36 extensions:
|
==> 36 extensions:
|
||||||
geos py-ipython py-pexpect py-pyside py-sip
|
geos py-ipython py-pexpect py-pyside py-sip
|
||||||
py-basemap py-libxml2 py-pil py-pytz py-six
|
py-basemap py-libxml2 py-pil py-pytz py-six
|
||||||
|
|
@ -1495,7 +1521,7 @@ You can find extensions for your Python installation like this:
|
||||||
py-h5py py-numpy py-pyqt py-shiboken
|
py-h5py py-numpy py-pyqt py-shiboken
|
||||||
|
|
||||||
==> 12 installed:
|
==> 12 installed:
|
||||||
-- chaos_5_x86_64_ib / gcc@4.4.7 --------------------------------
|
-- linux-x86_64-debian7 / gcc@4.4.7 --------------------------------
|
||||||
py-dateutil@2.4.0 py-nose@1.3.4 py-pyside@1.2.2
|
py-dateutil@2.4.0 py-nose@1.3.4 py-pyside@1.2.2
|
||||||
py-dateutil@2.4.0 py-numpy@1.9.1 py-pytz@2014.10
|
py-dateutil@2.4.0 py-numpy@1.9.1 py-pytz@2014.10
|
||||||
py-ipython@2.3.1 py-pygments@2.0.1 py-setuptools@11.3.1
|
py-ipython@2.3.1 py-pygments@2.0.1 py-setuptools@11.3.1
|
||||||
|
|
@ -1511,8 +1537,8 @@ prefixes, and you can see this with ``spack find -p``:
|
||||||
|
|
||||||
$ spack find -p py-numpy
|
$ spack find -p py-numpy
|
||||||
==> 1 installed packages.
|
==> 1 installed packages.
|
||||||
-- chaos_5_x86_64_ib / gcc@4.4.7 --------------------------------
|
-- linux-x86_64-debian7 / gcc@4.4.7 --------------------------------
|
||||||
py-numpy@1.9.1 /g/g21/gamblin2/src/spack/opt/chaos_5_x86_64_ib/gcc@4.4.7/py-numpy@1.9.1-66733244
|
py-numpy@1.9.1 /g/g21/gamblin2/src/spack/opt/linux-x86_64-debian7/gcc@4.4.7/py-numpy@1.9.1-66733244
|
||||||
|
|
||||||
However, even though this package is installed, you cannot use it
|
However, even though this package is installed, you cannot use it
|
||||||
directly when you run ``python``:
|
directly when you run ``python``:
|
||||||
|
|
@ -1573,9 +1599,9 @@ installation:
|
||||||
.. code-block:: sh
|
.. code-block:: sh
|
||||||
|
|
||||||
$ spack activate py-numpy
|
$ spack activate py-numpy
|
||||||
==> Activated extension py-setuptools@11.3.1%gcc@4.4.7 arch=chaos_5_x86_64_ib-3c74eb69 for python@2.7.8%gcc@4.4.7.
|
==> Activated extension py-setuptools@11.3.1%gcc@4.4.7 arch=linux-x86_64-debian7-3c74eb69 for python@2.7.8%gcc@4.4.7.
|
||||||
==> Activated extension py-nose@1.3.4%gcc@4.4.7 arch=chaos_5_x86_64_ib-5f70f816 for python@2.7.8%gcc@4.4.7.
|
==> Activated extension py-nose@1.3.4%gcc@4.4.7 arch=linux-x86_64-debian7-5f70f816 for python@2.7.8%gcc@4.4.7.
|
||||||
==> Activated extension py-numpy@1.9.1%gcc@4.4.7 arch=chaos_5_x86_64_ib-66733244 for python@2.7.8%gcc@4.4.7.
|
==> Activated extension py-numpy@1.9.1%gcc@4.4.7 arch=linux-x86_64-debian7-66733244 for python@2.7.8%gcc@4.4.7.
|
||||||
|
|
||||||
Several things have happened here. The user requested that
|
Several things have happened here. The user requested that
|
||||||
``py-numpy`` be activated in the ``python`` installation it was built
|
``py-numpy`` be activated in the ``python`` installation it was built
|
||||||
|
|
@ -1590,7 +1616,7 @@ packages listed as activated:
|
||||||
.. code-block:: sh
|
.. code-block:: sh
|
||||||
|
|
||||||
$ spack extensions python
|
$ spack extensions python
|
||||||
==> python@2.7.8%gcc@4.4.7 arch=chaos_5_x86_64_ib-703c7a96
|
==> python@2.7.8%gcc@4.4.7 arch=linux-x86_64-debian7-703c7a96
|
||||||
==> 36 extensions:
|
==> 36 extensions:
|
||||||
geos py-ipython py-pexpect py-pyside py-sip
|
geos py-ipython py-pexpect py-pyside py-sip
|
||||||
py-basemap py-libxml2 py-pil py-pytz py-six
|
py-basemap py-libxml2 py-pil py-pytz py-six
|
||||||
|
|
@ -1602,14 +1628,14 @@ packages listed as activated:
|
||||||
py-h5py py-numpy py-pyqt py-shiboken
|
py-h5py py-numpy py-pyqt py-shiboken
|
||||||
|
|
||||||
==> 12 installed:
|
==> 12 installed:
|
||||||
-- chaos_5_x86_64_ib / gcc@4.4.7 --------------------------------
|
-- linux-x86_64-debian7 / gcc@4.4.7 --------------------------------
|
||||||
py-dateutil@2.4.0 py-nose@1.3.4 py-pyside@1.2.2
|
py-dateutil@2.4.0 py-nose@1.3.4 py-pyside@1.2.2
|
||||||
py-dateutil@2.4.0 py-numpy@1.9.1 py-pytz@2014.10
|
py-dateutil@2.4.0 py-numpy@1.9.1 py-pytz@2014.10
|
||||||
py-ipython@2.3.1 py-pygments@2.0.1 py-setuptools@11.3.1
|
py-ipython@2.3.1 py-pygments@2.0.1 py-setuptools@11.3.1
|
||||||
py-matplotlib@1.4.2 py-pyparsing@2.0.3 py-six@1.9.0
|
py-matplotlib@1.4.2 py-pyparsing@2.0.3 py-six@1.9.0
|
||||||
|
|
||||||
==> 3 currently activated:
|
==> 3 currently activated:
|
||||||
-- chaos_5_x86_64_ib / gcc@4.4.7 --------------------------------
|
-- linux-x86_64-debian7 / gcc@4.4.7 --------------------------------
|
||||||
py-nose@1.3.4 py-numpy@1.9.1 py-setuptools@11.3.1
|
py-nose@1.3.4 py-numpy@1.9.1 py-setuptools@11.3.1
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -1691,7 +1717,7 @@ This issue typically manifests with the error below:
|
||||||
Traceback (most recent call last):
|
Traceback (most recent call last):
|
||||||
File "./spack", line 176, in <module>
|
File "./spack", line 176, in <module>
|
||||||
main()
|
main()
|
||||||
File "./spack", line 154, in main
|
File "./spack", line 154,' in main
|
||||||
return_val = command(parser, args)
|
return_val = command(parser, args)
|
||||||
File "./spack/lib/spack/spack/cmd/find.py", line 170, in find
|
File "./spack/lib/spack/spack/cmd/find.py", line 170, in find
|
||||||
specs = set(spack.installed_db.query(**q_args))
|
specs = set(spack.installed_db.query(**q_args))
|
||||||
|
|
@ -1709,6 +1735,20 @@ This issue typically manifests with the error below:
|
||||||
|
|
||||||
A nicer error message is TBD in future versions of Spack.
|
A nicer error message is TBD in future versions of Spack.
|
||||||
|
|
||||||
|
|
||||||
|
Spack on Cray
|
||||||
|
-----------------------------
|
||||||
|
|
||||||
|
Spack is able to detect compilers through the module avail command. Once it
|
||||||
|
detects the compiler it writes the appropriate PrgEnv and compiler module
|
||||||
|
name to compilers.yaml and sets the paths to each compiler with Cray\'s
|
||||||
|
compiler wrapper names ie (cc, CC, ftn). During build time, Spack will
|
||||||
|
load the correct PrgEnv and compiler module and will call either cc, CC
|
||||||
|
or ftn. If you want to use default compilers for each PrgEnv and also be able
|
||||||
|
to link to cray external modules, you will need to set up a packages.yaml
|
||||||
|
:ref:`Exernal Packages`<sec-external packages>
|
||||||
|
|
||||||
|
|
||||||
Getting Help
|
Getting Help
|
||||||
-----------------------
|
-----------------------
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -70,20 +70,31 @@ directory. Here's an example of an external configuration:
|
||||||
packages:
|
packages:
|
||||||
openmpi:
|
openmpi:
|
||||||
paths:
|
paths:
|
||||||
openmpi@1.4.3%gcc@4.4.7 arch=chaos_5_x86_64_ib: /opt/openmpi-1.4.3
|
openmpi@1.4.3%gcc@4.4.7 arch=linux-x86_64-debian7: /opt/openmpi-1.4.3
|
||||||
openmpi@1.4.3%gcc@4.4.7 arch=chaos_5_x86_64_ib+debug: /opt/openmpi-1.4.3-debug
|
openmpi@1.4.3%gcc@4.4.7 arch=linux-x86_64-debian7+debug: /opt/openmpi-1.4.3-debug
|
||||||
openmpi@1.6.5%intel@10.1 arch=chaos_5_x86_64_ib: /opt/openmpi-1.6.5-intel
|
openmpi@1.6.5%intel@10.1 arch=linux-x86_64-debian7: /opt/openmpi-1.6.5-intel
|
||||||
|
|
||||||
This example lists three installations of OpenMPI, one built with gcc,
|
This example lists three installations of OpenMPI, one built with gcc,
|
||||||
one built with gcc and debug information, and another built with Intel.
|
one built with gcc and debug information, and another built with Intel.
|
||||||
If Spack is asked to build a package that uses one of these MPIs as a
|
If Spack is asked to build a package that uses one of these MPIs as a
|
||||||
dependency, it will use the the pre-installed OpenMPI in
|
dependency, it will use the the pre-installed OpenMPI in
|
||||||
the given directory.
|
the given directory. Packages.yaml can also be used to specify modules
|
||||||
|
|
||||||
|
Here's an example of an external configuration for cray modules:
|
||||||
|
|
||||||
|
.. code-block:: yaml
|
||||||
|
packages:
|
||||||
|
mpich:
|
||||||
|
modules:
|
||||||
|
mpich@7.3.1%gcc@5.2.0 arch=cray_xc-haswell-CNL10: cray-mpich
|
||||||
|
mpich@7.3.1%intel@16.0.0.109 arch=cray_xc-haswell-CNL10: cray-mpich
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
Each ``packages.yaml`` begins with a ``packages:`` token, followed
|
Each ``packages.yaml`` begins with a ``packages:`` token, followed
|
||||||
by a list of package names. To specify externals, add a ``paths``
|
by a list of package names. To specify externals, add a ``paths`` or ``modules``
|
||||||
token under the package name, which lists externals in a
|
token under the package name, which lists externals in a
|
||||||
``spec : /path`` format. Each spec should be as
|
``spec: /path`` or ``spec: module-name`` format. Each spec should be as
|
||||||
well-defined as reasonably possible. If a
|
well-defined as reasonably possible. If a
|
||||||
package lacks a spec component, such as missing a compiler or
|
package lacks a spec component, such as missing a compiler or
|
||||||
package version, then Spack will guess the missing component based
|
package version, then Spack will guess the missing component based
|
||||||
|
|
@ -108,9 +119,9 @@ be:
|
||||||
packages:
|
packages:
|
||||||
openmpi:
|
openmpi:
|
||||||
paths:
|
paths:
|
||||||
openmpi@1.4.3%gcc@4.4.7 arch=chaos_5_x86_64_ib: /opt/openmpi-1.4.3
|
openmpi@1.4.3%gcc@4.4.7 arch=linux-x86_64-debian7: /opt/openmpi-1.4.3
|
||||||
openmpi@1.4.3%gcc@4.4.7 arch=chaos_5_x86_64_ib+debug: /opt/openmpi-1.4.3-debug
|
openmpi@1.4.3%gcc@4.4.7 arch=linux-x86_64-debian7+debug: /opt/openmpi-1.4.3-debug
|
||||||
openmpi@1.6.5%intel@10.1 arch=chaos_5_x86_64_ib: /opt/openmpi-1.6.5-intel
|
openmpi@1.6.5%intel@10.1 arch=linux-x86_64-debian7: /opt/openmpi-1.6.5-intel
|
||||||
buildable: False
|
buildable: False
|
||||||
|
|
||||||
The addition of the ``buildable`` flag tells Spack that it should never build
|
The addition of the ``buildable`` flag tells Spack that it should never build
|
||||||
|
|
@ -118,6 +129,9 @@ its own version of OpenMPI, and it will instead always rely on a pre-built
|
||||||
OpenMPI. Similar to ``paths``, ``buildable`` is specified as a property under
|
OpenMPI. Similar to ``paths``, ``buildable`` is specified as a property under
|
||||||
a package name.
|
a package name.
|
||||||
|
|
||||||
|
If an external module is specified as not buildable, then Spack will load the
|
||||||
|
external module into the build environment which can be used for linking.
|
||||||
|
|
||||||
The ``buildable`` does not need to be paired with external packages.
|
The ``buildable`` does not need to be paired with external packages.
|
||||||
It could also be used alone to forbid packages that may be
|
It could also be used alone to forbid packages that may be
|
||||||
buggy or otherwise undesirable.
|
buggy or otherwise undesirable.
|
||||||
|
|
@ -180,7 +194,35 @@ concretization rules. A provider lists a value that packages may
|
||||||
``depend_on`` (e.g, mpi) and a list of rules for fulfilling that
|
``depend_on`` (e.g, mpi) and a list of rules for fulfilling that
|
||||||
dependency.
|
dependency.
|
||||||
|
|
||||||
|
For Cray users, you can specify the default compiler that Spack will
|
||||||
|
concretize too. If you want to use the Cray defaults, then set them
|
||||||
|
under all: in packages.yaml. You can also specify concretization preferences
|
||||||
|
to cray-mpich. Just set the cray-mpich external module as a preferred package,
|
||||||
|
set buildable to False and set the preference of mpi to mpich.
|
||||||
|
|
||||||
|
Here is an example of a full packages.yaml used at NERSC
|
||||||
|
|
||||||
|
.. code-block:: sh
|
||||||
|
packages:
|
||||||
|
mpich:
|
||||||
|
modules:
|
||||||
|
mpich@7.3.1%gcc@5.2.0 arch=cray_xc-CNL10-ivybridge: cray-mpich
|
||||||
|
mpich@7.3.1%intel@16.0.0.109 arch=cray_xc-SuSE11-ivybridge: cray-mpich
|
||||||
|
buildable: False
|
||||||
|
netcdf:
|
||||||
|
modules:
|
||||||
|
netcdf@4.3.3.1%gcc@5.2.0 arch=cray_xc-CNL10-ivybridge: cray-netcdf
|
||||||
|
netcdf@4.3.3.1%intel@16.0.0.109 arch=cray_xc-CNL10-ivybridge: cray-netcdf
|
||||||
|
buildable: False
|
||||||
|
hdf5:
|
||||||
|
paths:
|
||||||
|
hdf5@1.8.14%gcc@5.2.0 arch=cray_xc-CNL10-ivybridge: cray-hdf5
|
||||||
|
hdf5@1.8.14%intel@16.0.0.109 arch=cray_xc-CNL10-ivybridge: cray-hdf5
|
||||||
|
buildable: False
|
||||||
|
all:
|
||||||
|
compiler: [gcc@5.2.0, intel@16.0.0.109]
|
||||||
|
providers:
|
||||||
|
mpi: [mpich, openmpi, intelmpi]ackages:
|
||||||
|
|
||||||
Profiling
|
Profiling
|
||||||
------------------
|
------------------
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue