hlrs-software-stack/spack
7
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

module files tutorial : first complete draft (#2228)

Browse source 
* module files tutorial : first complete draft

- first complete draft for module files tutorial
- minor corrections to module file reference

* module file tutorial : first batch of corrections

- module avail spelled out fully
- typos from @adamjstewart
- rewording of a few sentences

* module file tutorial : first batch of corrections

- emphasized lines in yaml files

* module file tutorial : fixes according to @citibeth and @adamjstewart reviews

- used long format for command options
- reworded unclear sentence on tokes
- reworked table in reference manual to make it clearer

* module file tutorial : implemented corrections collected on site from @schlyfts

* module file tutorial : removed comment (@hartzell suggestion)
This commit is contained in:
Massimiliano Culpo 2016-11-04 20:55:43 +01:00 committed by Todd Gamblin
parent ee6eb508cb
commit 401b4cb137
3 changed files with 1009 additions and 27 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
lib/spack/docs/index.rst
View file

@ -46,6 +46,7 @@ or refer to the full manual below.
getting_started getting_started
basic_usage basic_usage
workflows workflows
module_file_tutorial
.. toctree:: .. toctree::
:maxdepth: 2 :maxdepth: 2

60
lib/spack/docs/module_file_support.rst
View file

@ -1,8 +1,8 @@
.. _modules: .. _modules:
===================================== =======
Modules Modules
===================================== =======
The use of module systems to manage user environment in a controlled way The use of module systems to manage user environment in a controlled way
is a common practice at HPC centers that is often embraced also by individual is a common practice at HPC centers that is often embraced also by individual
@ -24,6 +24,8 @@ directly with automatically generated module files.
we advise you to use either Environment Modules or LMod. See :ref:`InstallEnvironmentModules` we advise you to use either Environment Modules or LMod. See :ref:`InstallEnvironmentModules`
for more details. for more details.
.. _shell_support:
------------- -------------
Shell support Shell support
------------- -------------
@ -110,9 +112,9 @@ you could type either of these commands to load the callpath module:
.. _cmd-spack-load: .. _cmd-spack-load:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^
``spack load / unload`` ``spack load / unload``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^
Neither of these is particularly pretty, easy to remember, or Neither of these is particularly pretty, easy to remember, or
easy to type. Luckily, Spack has its own interface for using modules easy to type. Luckily, Spack has its own interface for using modules
@ -161,9 +163,9 @@ want to use a package, you can type unload or unuse similarly:
only available if you have enabled Spack's shell support *and* you only available if you have enabled Spack's shell support *and* you
have dotkit or modules installed on your machine. have dotkit or modules installed on your machine.
^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^
Ambiguous module names Ambiguous module names
^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^
If a spec used with load/unload or use/unuse is ambiguous (i.e. more If a spec used with load/unload or use/unuse is ambiguous (i.e. more
than one installed package matches it), then Spack will warn you: than one installed package matches it), then Spack will warn you:
@ -294,23 +296,24 @@ For example, consider the following on one system:
# antlr@2.7.7%gcc@5.3.0~csharp+cxx~java~python arch=linux-SuSE11-x86_64 # antlr@2.7.7%gcc@5.3.0~csharp+cxx~java~python arch=linux-SuSE11-x86_64
module load linux-SuSE11-x86_64/antlr-2.7.7-gcc-5.3.0-bdpl46y module load linux-SuSE11-x86_64/antlr-2.7.7-gcc-5.3.0-bdpl46y
----------------------------- ----------------------------
Auto-generating Module Files Auto-generating Module Files
----------------------------- ----------------------------
Module files are generated by post-install hooks after the successful Module files are generated by post-install hooks after the successful
installation of a package. They are placed in the following directories installation of a package. The following table summarizes the essential
under the Spack root: information associated with the different file formats
that can be generated by Spack:
+----------------------------------------+------------------------------------+------------------------+ +-----------------------------+--------------------+-------------------------------+----------------------+
| | **Module Files Root directory** | **Compatible systems** | | | **Hook name** | **Default root directory** | **Compatible tools** |
+========================================+====================================+========================+ +=============================+====================+===============================+======================+
| **Dotkit Module Files** | share/spack/dotkit | DotKit | | **Dotkit** | ``dotkit`` | share/spack/dotkit | DotKit |
+----------------------------------------+------------------------------------+------------------------+ +-----------------------------+--------------------+-------------------------------+----------------------+
| **Non-Hierarchical TCL Module Files** | share/spack/modules | Env. Modules/LMod | | **TCL - Non-Hierarchical** | ``tcl`` | share/spack/modules | Env. Modules/LMod |
+----------------------------------------+------------------------------------+------------------------+ +-----------------------------+--------------------+-------------------------------+----------------------+
| **Lua Hierarchical Module Files** | share/spack/lmod | LMod | | **Lua - Hierarchical** | ``lmod`` | share/spack/lmod | LMod |
+----------------------------------------+------------------------------------+------------------------+ +-----------------------------+--------------------+-------------------------------+----------------------+
Though Spack ships with sensible defaults for the generation of module files, Though Spack ships with sensible defaults for the generation of module files,
@ -324,9 +327,9 @@ The former method fits best cases that are site independent, e.g. injecting vari
from language interpreters into their extensions. The latter instead permits to from language interpreters into their extensions. The latter instead permits to
fine tune the content, naming and creation of module files to meet site specific conventions. fine tune the content, naming and creation of module files to meet site specific conventions.
^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^
``Package`` file API ``Package`` file API
^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^
There are two methods that can be overridden in any ``package.py`` to affect the There are two methods that can be overridden in any ``package.py`` to affect the
content of generated module files. The first one is: content of generated module files. The first one is:
@ -376,9 +379,9 @@ and has similar effects on module file of dependees. Even in this case
.. _modules-yaml: .. _modules-yaml:
---------------------------------- ---------------------------------
Configuration in ``modules.yaml`` Configuration in ``modules.yaml``
---------------------------------- ---------------------------------
The name of the configuration file that controls module generation behavior The name of the configuration file that controls module generation behavior
is ``modules.yaml``. The default configuration: is ``modules.yaml``. The default configuration:
@ -426,6 +429,8 @@ All these module sections allow for both:
For the latter point in particular it is possible to use anonymous specs For the latter point in particular it is possible to use anonymous specs
to select an appropriate set of packages on which the modifications should be applied. to select an appropriate set of packages on which the modifications should be applied.
.. _anonymous_specs:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Selection by anonymous specs Selection by anonymous specs
^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@ -544,7 +549,7 @@ most likely via the ``+blas`` variant specification.
modules: modules:
tcl: tcl:
naming_scheme: '${PACKAGE}/${VERSION}-${COMPILERNAME}-${COMPILERVERSION}' naming_scheme: '${PACKAGE}/${VERSION}-${COMPILERNAME}-${COMPILERVER}'
all: all:
conflict: ['${PACKAGE}', 'intel/14.0.1'] conflict: ['${PACKAGE}', 'intel/14.0.1']
@ -660,9 +665,9 @@ before regeneration if the change in layout is radical.
.. _cmd-spack-module-rm: .. _cmd-spack-module-rm:
^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^
``spack module rm`` ``spack module rm``
^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^
If instead what you need is just to delete a few module files, then the right If instead what you need is just to delete a few module files, then the right
command is ``module rm``: command is ``module rm``:
@ -670,7 +675,8 @@ command is ``module rm``:
.. command-output:: spack module rm --help .. command-output:: spack module rm --help
.. note:: .. note::
We care about your module files! Every modification done on modules We care about your module files!
Every modification done on modules
that are already existing will ask for a confirmation by default. If that are already existing will ask for a confirmation by default. If
the command is used in a script it is possible though to pass the the command is used in a script it is possible though to pass the
``-y`` argument, that will skip this safety measure. ``-y`` argument, that will skip this safety measure.

975
lib/spack/docs/module_file_tutorial.rst Normal file
View file

@ -0,0 +1,975 @@
.. _module_tutorial:
=======
Modules
=======
This tutorial will guide you through the customization of both
content and naming of module files generated by Spack.
Starting from the default Spack settings you will add an increasing
number of directives to the ``modules.yaml`` configuration file to
satisfy a number of constraints that mimic those that you may encounter
in a typical production environment at HPC sites.
Even though the focus will be for the most part on customizing
TCL non-hierarchical module files, everything
you'll see applies also to other kinds of module files generated by Spack.
The generation of Lua hierarchical
module files will be addressed at the end of the tutorial,
and you'll see that with minor modifications
to an existing ``modules.yaml`` written for TCL
non-hierarchical modules you'll get almost
for free the possibility to try a hierarchical layout.
Let's start!
.. _module_file_tutorial_prerequisites:
-------------
Prerequisites
-------------
Before proceeding further ensure:
- you have LMod or Environment Modules available
- have :ref:`shell support <shell_support>` activated in Spack
If you need to install Lmod or Environment module you can refer
to the documentation :ref:`here <InstallEnvironmentModules>`.
^^^^^^^^^^^^^^^^^^
Add a new compiler
^^^^^^^^^^^^^^^^^^
Spack automatically scans the environment to search for available
compilers on first use. On a Ubuntu 14.04 a fresh clone will show
something like this:
.. code-block:: console
$ uname -a
Linux nuvolari 4.4.0-45-generic #66~14.04.1-Ubuntu SMP Wed Oct 19 15:05:38 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
$ spack compilers
==> Available compilers
-- gcc ----------------------------------------------------------
gcc@4.8
For the purpose of building a limited set of packages with some features
that will help showcasing the capabilities of
module customization the first thing we need is to build a new compiler:
.. code-block:: console
$ spack install gcc@6.2.0
# ...
# Wait a long time
# ...
Then we can use shell support for modules to add it to the list of known compilers:
.. code-block:: console
# The name of the generated module may vary
$ module load gcc-6.2.0-gcc-4.8-twd5nqg
$ spack compiler add
==> Added 1 new compiler to /home/mculpo/.spack/linux/compilers.yaml
gcc@6.2.0
$ spack compilers
==> Available compilers
-- gcc ----------------------------------------------------------
gcc@6.2.0 gcc@4.8
Note that the final 7 digits hash at the end of the generated module may vary depending
on architecture or package version.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Build software that will be used in the tutorial
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Next you should install a few modules that will be used in the tutorial:
.. code-block:: console
$ spack install netlib-scalapack ^openmpi ^openblas
# ...
The packages you need to install are:
- ``netlib-scalapack ^openmpi ^openblas``
- ``netlib-scalapack ^mpich ^openblas``
- ``netlib-scalapack ^openmpi ^netlib-lapack``
- ``netlib-scalapack ^mpich ^netlib-lapack``
- ``py-scipy ^openblas``
In the end your environment should look something like:
.. code-block:: console
$ module avail
------------------------------------------------------------------------ /home/mculpo/wdir/spack/share/spack/modules/linux-Ubuntu14-x86_64 ------------------------------------------------------------------------
binutils-2.27-gcc-4.8-dz3xevw libpciaccess-0.13.4-gcc-6.2.0-eo2siet lzo-2.09-gcc-6.2.0-jcngz72 netlib-scalapack-2.0.2-gcc-6.2.0-wnimqhw python-2.7.12-gcc-6.2.0-qu7rc5p
bzip2-1.0.6-gcc-6.2.0-csoc2mq libsigsegv-2.10-gcc-4.8-avb6azw m4-1.4.17-gcc-4.8-iggewke netlib-scalapack-2.0.2-gcc-6.2.0-wojunhq sqlite-3.8.5-gcc-6.2.0-td3zfe7
cmake-3.5.2-gcc-6.2.0-6poypqg libsigsegv-2.10-gcc-6.2.0-g3qpmbi m4-1.4.17-gcc-6.2.0-lhgqa6s nettle-3.2-gcc-6.2.0-djdthlh tcl-8.6.5-gcc-4.8-atddxu7
curl-7.50.3-gcc-6.2.0-2ffacqm libtool-2.4.6-gcc-6.2.0-kiepac6 mpc-1.0.3-gcc-4.8-lylv7lk openblas-0.2.19-gcc-6.2.0-js33umc util-macros-1.19.0-gcc-6.2.0-uoukuqk
expat-2.2.0-gcc-6.2.0-bxqnjar libxml2-2.9.4-gcc-6.2.0-3k4ykbe mpfr-3.1.4-gcc-4.8-bldfx3w openmpi-2.0.1-gcc-6.2.0-s3qbtby xz-5.2.2-gcc-6.2.0-t5lk6in
gcc-6.2.0-gcc-4.8-twd5nqg lmod-6.4.5-gcc-4.8-7v7bh7b mpich-3.2-gcc-6.2.0-5n5xoep openssl-1.0.2j-gcc-6.2.0-hibnfda zlib-1.2.8-gcc-4.8-bds4ies
gmp-6.1.1-gcc-4.8-uq52e2n lua-5.3.2-gcc-4.8-xozf2hx ncurses-6.0-gcc-4.8-u62fit4 pkg-config-0.29.1-gcc-6.2.0-rslsgcs zlib-1.2.8-gcc-6.2.0-asydrba
gmp-6.1.1-gcc-6.2.0-3cfh3hi lua-luafilesystem-1_6_3-gcc-4.8-sbzejlz ncurses-6.0-gcc-6.2.0-7tb426s py-nose-1.3.7-gcc-6.2.0-4gl5c42
hwloc-1.11.4-gcc-6.2.0-3ostwel lua-luaposix-33.4.0-gcc-4.8-xf7y2p5 netlib-lapack-3.6.1-gcc-6.2.0-mirer2l py-numpy-1.11.1-gcc-6.2.0-i3rpk4e
isl-0.14-gcc-4.8-cq73t5m lz4-131-gcc-6.2.0-cagoem4 netlib-scalapack-2.0.2-gcc-6.2.0-6bqlxqy py-scipy-0.18.1-gcc-6.2.0-e6uljfi
libarchive-3.2.1-gcc-6.2.0-2b54aos lzma-4.32.7-gcc-6.2.0-sfmeynw netlib-scalapack-2.0.2-gcc-6.2.0-hpqb3dp py-setuptools-25.2.0-gcc-6.2.0-hkqauaa
------------------------------------------------
Filter unwanted modifications to the environment
------------------------------------------------
The non-hierarchical TCL module files that have been generated so far
follow the default rules for module generation, which are given
:ref:`here <modules-yaml>` in the reference part of the manual. Taking a
look at the ``gcc`` module you'll see something like:
.. code-block:: console
$ module show gcc-6.2.0-gcc-4.8-twd5nqg
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
/home/mculpo/wdir/spack/share/spack/modules/linux-Ubuntu14-x86_64/gcc-6.2.0-gcc-4.8-twd5nqg:
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
whatis("gcc @6.2.0 ")
prepend_path("PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/bin")
prepend_path("CMAKE_PREFIX_PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/")
prepend_path("MANPATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/share/man")
prepend_path("PKG_CONFIG_PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/lib64/pkgconfig")
prepend_path("LIBRARY_PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/lib64")
prepend_path("LD_LIBRARY_PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/lib64")
prepend_path("CPATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/include")
help([[The GNU Compiler Collection includes front ends for C, C++, Objective-C,
Fortran, and Java.
]])
As expected, a few environment variables representing paths will be modified
by the modules according to the default prefix inspection rules.
Consider now the case that your site has decided that e.g. ``CPATH`` and
``LIBRARY_PATH`` modifications should not be present in module files. What you can
do to abide by the rules is to create a configuration file ``~/.spack/modules.yaml``
with the following content:
.. code-block:: yaml
modules:
tcl:
all:
filter:
environment_blacklist: ['CPATH', 'LIBRARY_PATH']
Next you should regenerate all the module files:
.. code-block:: console
$ spack module refresh --module-type tcl
==> You are about to regenerate tcl module files for:
-- linux-Ubuntu14-x86_64 / gcc@4.8 ------------------------------
dz3xevw binutils@2.27 uq52e2n gmp@6.1.1 avb6azw libsigsegv@2.10 xozf2hx lua@5.3.2 xf7y2p5 lua-luaposix@33.4.0 lylv7lk mpc@1.0.3 u62fit4 ncurses@6.0 bds4ies zlib@1.2.8
twd5nqg gcc@6.2.0 cq73t5m isl@0.14 7v7bh7b lmod@6.4.5 sbzejlz lua-luafilesystem@1_6_3 iggewke m4@1.4.17 bldfx3w mpfr@3.1.4 atddxu7 tcl@8.6.5
...
==> Do you want to proceed ? [y/n]
y
==> Regenerating tcl module files
If you take a look now at the module for ``gcc`` you'll see that the unwanted
paths have disappeared:
.. code-block:: console
$ module show gcc-6.2.0-gcc-4.8-twd5nqg
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
/home/mculpo/wdir/spack/share/spack/modules/linux-Ubuntu14-x86_64/gcc-6.2.0-gcc-4.8-twd5nqg:
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
whatis("gcc @6.2.0 ")
prepend_path("PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/bin")
prepend_path("CMAKE_PREFIX_PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/")
prepend_path("MANPATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/share/man")
prepend_path("PKG_CONFIG_PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/lib64/pkgconfig")
prepend_path("LD_LIBRARY_PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/lib64")
help([[The GNU Compiler Collection includes front ends for C, C++, Objective-C,
Fortran, and Java.
]])
----------------------------------------------
Prevent some module files from being generated
----------------------------------------------
Another common request at many sites is to avoid exposing software that
is only needed as an intermediate step when building a newer stack.
Let's try to prevent the generation of
module files for anything that is compiled with ``gcc@4.8`` (the OS provided compiler).
To do this you should add a ``blacklist`` keyword to the configuration file:
.. code-block:: yaml
:emphasize-lines: 3,4
modules:
tcl:
blacklist:
- '%gcc@4.8'
all:
filter:
environment_blacklist: ['CPATH', 'LIBRARY_PATH']
and regenerate the module files:
.. code-block:: console
$ spack module refresh --module-type tcl --delete-tree
==> You are about to regenerate tcl module files for:
-- linux-Ubuntu14-x86_64 / gcc@4.8 ------------------------------
dz3xevw binutils@2.27 uq52e2n gmp@6.1.1 avb6azw libsigsegv@2.10 xozf2hx lua@5.3.2 xf7y2p5 lua-luaposix@33.4.0 lylv7lk mpc@1.0.3 u62fit4 ncurses@6.0 bds4ies zlib@1.2.8
twd5nqg gcc@6.2.0 cq73t5m isl@0.14 7v7bh7b lmod@6.4.5 sbzejlz lua-luafilesystem@1_6_3 iggewke m4@1.4.17 bldfx3w mpfr@3.1.4 atddxu7 tcl@8.6.5
-- linux-Ubuntu14-x86_64 / gcc@6.2.0 ----------------------------
csoc2mq bzip2@1.0.6 2b54aos libarchive@3.2.1 sfmeynw lzma@4.32.7 wnimqhw netlib-scalapack@2.0.2 s3qbtby openmpi@2.0.1 hkqauaa py-setuptools@25.2.0
6poypqg cmake@3.5.2 eo2siet libpciaccess@0.13.4 jcngz72 lzo@2.09 6bqlxqy netlib-scalapack@2.0.2 hibnfda openssl@1.0.2j qu7rc5p python@2.7.12
2ffacqm curl@7.50.3 g3qpmbi libsigsegv@2.10 lhgqa6s m4@1.4.17 wojunhq netlib-scalapack@2.0.2 rslsgcs pkg-config@0.29.1 td3zfe7 sqlite@3.8.5
bxqnjar expat@2.2.0 kiepac6 libtool@2.4.6 5n5xoep mpich@3.2 hpqb3dp netlib-scalapack@2.0.2 4gl5c42 py-nose@1.3.7 uoukuqk util-macros@1.19.0
3cfh3hi gmp@6.1.1 3k4ykbe libxml2@2.9.4 7tb426s ncurses@6.0 djdthlh nettle@3.2 i3rpk4e py-numpy@1.11.1 t5lk6in xz@5.2.2
3ostwel hwloc@1.11.4 cagoem4 lz4@131 mirer2l netlib-lapack@3.6.1 js33umc openblas@0.2.19 e6uljfi py-scipy@0.18.1 asydrba zlib@1.2.8
==> Do you want to proceed ? [y/n]
y
$ module avail
------------------------------------------------------------------------ /home/mculpo/wdir/spack/share/spack/modules/linux-Ubuntu14-x86_64 ------------------------------------------------------------------------
bzip2-1.0.6-gcc-6.2.0-csoc2mq libsigsegv-2.10-gcc-6.2.0-g3qpmbi ncurses-6.0-gcc-6.2.0-7tb426s openmpi-2.0.1-gcc-6.2.0-s3qbtby sqlite-3.8.5-gcc-6.2.0-td3zfe7
cmake-3.5.2-gcc-6.2.0-6poypqg libtool-2.4.6-gcc-6.2.0-kiepac6 netlib-lapack-3.6.1-gcc-6.2.0-mirer2l openssl-1.0.2j-gcc-6.2.0-hibnfda util-macros-1.19.0-gcc-6.2.0-uoukuqk
curl-7.50.3-gcc-6.2.0-2ffacqm libxml2-2.9.4-gcc-6.2.0-3k4ykbe netlib-scalapack-2.0.2-gcc-6.2.0-6bqlxqy pkg-config-0.29.1-gcc-6.2.0-rslsgcs xz-5.2.2-gcc-6.2.0-t5lk6in
expat-2.2.0-gcc-6.2.0-bxqnjar lz4-131-gcc-6.2.0-cagoem4 netlib-scalapack-2.0.2-gcc-6.2.0-hpqb3dp py-nose-1.3.7-gcc-6.2.0-4gl5c42 zlib-1.2.8-gcc-6.2.0-asydrba
gmp-6.1.1-gcc-6.2.0-3cfh3hi lzma-4.32.7-gcc-6.2.0-sfmeynw netlib-scalapack-2.0.2-gcc-6.2.0-wnimqhw py-numpy-1.11.1-gcc-6.2.0-i3rpk4e
hwloc-1.11.4-gcc-6.2.0-3ostwel lzo-2.09-gcc-6.2.0-jcngz72 netlib-scalapack-2.0.2-gcc-6.2.0-wojunhq py-scipy-0.18.1-gcc-6.2.0-e6uljfi
libarchive-3.2.1-gcc-6.2.0-2b54aos m4-1.4.17-gcc-6.2.0-lhgqa6s nettle-3.2-gcc-6.2.0-djdthlh py-setuptools-25.2.0-gcc-6.2.0-hkqauaa
libpciaccess-0.13.4-gcc-6.2.0-eo2siet mpich-3.2-gcc-6.2.0-5n5xoep openblas-0.2.19-gcc-6.2.0-js33umc python-2.7.12-gcc-6.2.0-qu7rc5p
This time it is convenient to pass the option ``--delete-tree`` to the command that
regenerates the module files to instruct it to delete the existing tree and regenerate
a new one instead of overwriting the files in the existing directory.
If you pay careful attention you'll see though that we went too far in blacklisting modules:
the module for ``gcc@6.2.0`` disappeared as it was bootstrapped with ``gcc@4.8``. To specify
exceptions to the blacklist rules you can use ``whitelist``:
.. code-block:: yaml
:emphasize-lines: 3,4
modules:
tcl:
whitelist:
- gcc
blacklist:
- '%gcc@4.8'
all:
filter:
environment_blacklist: ['CPATH', 'LIBRARY_PATH']
``whitelist`` rules always have precedence over ``blacklist`` rules. If you regenerate the modules again:
.. code-block:: console
$ spack module refresh --module-type tcl -y
you'll see that now the module for ``gcc@6.2.0`` has reappeared:
.. code-block:: console
$ module avail gcc-6.2.0-gcc-4.8-twd5nqg
------------------------------------------------------------------------ /home/mculpo/wdir/spack/share/spack/modules/linux-Ubuntu14-x86_64 ------------------------------------------------------------------------
gcc-6.2.0-gcc-4.8-twd5nqg
-------------------------
Change module file naming
-------------------------
The next step in making module files more user-friendly is to
improve their naming scheme.
To reduce the length of the hash or remove it altogether you can
use the ``hash_length`` keyword in the configuration file:
.. TODO: give reasons to remove hashes if they are not evident enough?
.. code-block:: yaml
:emphasize-lines: 3
modules:
tcl:
hash_length: 0
whitelist:
- gcc
blacklist:
- '%gcc@4.8'
all:
filter:
environment_blacklist: ['CPATH', 'LIBRARY_PATH']
If you try to regenerate the module files now you will get an error:
.. code-block:: console
$ spack module refresh --module-type tcl --delete-tree -y
==> Error: Name clashes detected in module files:
file : /home/mculpo/wdir/spack/share/spack/modules/linux-Ubuntu14-x86_64/netlib-scalapack-2.0.2-gcc-6.2.0
spec : netlib-scalapack@2.0.2%gcc@6.2.0~fpic+shared arch=linux-Ubuntu14-x86_64
spec : netlib-scalapack@2.0.2%gcc@6.2.0~fpic+shared arch=linux-Ubuntu14-x86_64
spec : netlib-scalapack@2.0.2%gcc@6.2.0~fpic+shared arch=linux-Ubuntu14-x86_64
spec : netlib-scalapack@2.0.2%gcc@6.2.0~fpic+shared arch=linux-Ubuntu14-x86_64
==> Error: Operation aborted
.. note::
We try to check for errors upfront!
In Spack we check for errors upfront whenever possible, so don't worry about your module files:
as a name clash was detected nothing has been changed on disk.
The problem here is that without
the hashes the four different flavors of ``netlib-scalapack`` map to the same module file
name. We have the possibility to add suffixes to differentiate them:
.. code-block:: yaml
:emphasize-lines: 9-11,14-17
modules:
tcl:
hash_length: 0
whitelist:
- gcc
blacklist:
- '%gcc@4.8'
all:
suffixes:
'^openblas': openblas
'^netlib-lapack': netlib
filter:
environment_blacklist: ['CPATH', 'LIBRARY_PATH']
netlib-scalapack:
suffixes:
'^openmpi': openmpi
'^mpich': mpich
As you can see it is possible to specify rules that applies only to a
restricted set of packages using :ref:`anonymous specs <anonymous_specs>`.
Regenerating module files now we obtain:
.. code-block:: console
$ spack module refresh --module-type tcl --delete-tree -y
==> Regenerating tcl module files
$ module avail
------------------------------------------------------------------------ /home/mculpo/wdir/spack/share/spack/modules/linux-Ubuntu14-x86_64 ------------------------------------------------------------------------
bzip2-1.0.6-gcc-6.2.0 libpciaccess-0.13.4-gcc-6.2.0 mpich-3.2-gcc-6.2.0 openblas-0.2.19-gcc-6.2.0 python-2.7.12-gcc-6.2.0
cmake-3.5.2-gcc-6.2.0 libsigsegv-2.10-gcc-6.2.0 ncurses-6.0-gcc-6.2.0 openmpi-2.0.1-gcc-6.2.0 sqlite-3.8.5-gcc-6.2.0
curl-7.50.3-gcc-6.2.0 libtool-2.4.6-gcc-6.2.0 netlib-lapack-3.6.1-gcc-6.2.0 openssl-1.0.2j-gcc-6.2.0 util-macros-1.19.0-gcc-6.2.0
expat-2.2.0-gcc-6.2.0 libxml2-2.9.4-gcc-6.2.0 netlib-scalapack-2.0.2-gcc-6.2.0-netlib-mpich pkg-config-0.29.1-gcc-6.2.0 xz-5.2.2-gcc-6.2.0
gcc-6.2.0-gcc-4.8 lz4-131-gcc-6.2.0 netlib-scalapack-2.0.2-gcc-6.2.0-netlib-openmpi py-nose-1.3.7-gcc-6.2.0 zlib-1.2.8-gcc-6.2.0
gmp-6.1.1-gcc-6.2.0 lzma-4.32.7-gcc-6.2.0 netlib-scalapack-2.0.2-gcc-6.2.0-openblas-mpich py-numpy-1.11.1-gcc-6.2.0-openblas
hwloc-1.11.4-gcc-6.2.0 lzo-2.09-gcc-6.2.0 netlib-scalapack-2.0.2-gcc-6.2.0-openblas-openmpi py-scipy-0.18.1-gcc-6.2.0-openblas
libarchive-3.2.1-gcc-6.2.0 m4-1.4.17-gcc-6.2.0 nettle-3.2-gcc-6.2.0 py-setuptools-25.2.0-gcc-6.2.0
Finally we can set a ``naming_scheme`` to prevent users from loading
modules that refer to different flavors of the same library/application:
.. code-block:: yaml
:emphasize-lines: 4,10,11
modules:
tcl:
hash_length: 0
naming_scheme: '${PACKAGE}/${VERSION}-${COMPILERNAME}-${COMPILERVER}'
whitelist:
- gcc
blacklist:
- '%gcc@4.8'
all:
conflict:
- '${PACKAGE}'
suffixes:
'^openblas': openblas
'^netlib-lapack': netlib
filter:
environment_blacklist: ['CPATH', 'LIBRARY_PATH']
netlib-scalapack:
suffixes:
'^openmpi': openmpi
'^mpich': mpich
The final result should look like:
.. code-block:: console
$ module avail
------------------------------------------------------------------------ /home/mculpo/wdir/spack/share/spack/modules/linux-Ubuntu14-x86_64 ------------------------------------------------------------------------
bzip2/1.0.6-gcc-6.2.0 libpciaccess/0.13.4-gcc-6.2.0 mpich/3.2-gcc-6.2.0 openblas/0.2.19-gcc-6.2.0 python/2.7.12-gcc-6.2.0
cmake/3.5.2-gcc-6.2.0 libsigsegv/2.10-gcc-6.2.0 ncurses/6.0-gcc-6.2.0 openmpi/2.0.1-gcc-6.2.0 sqlite/3.8.5-gcc-6.2.0
curl/7.50.3-gcc-6.2.0 libtool/2.4.6-gcc-6.2.0 netlib-lapack/3.6.1-gcc-6.2.0 openssl/1.0.2j-gcc-6.2.0 util-macros/1.19.0-gcc-6.2.0
expat/2.2.0-gcc-6.2.0 libxml2/2.9.4-gcc-6.2.0 netlib-scalapack/2.0.2-gcc-6.2.0-netlib-mpich pkg-config/0.29.1-gcc-6.2.0 xz/5.2.2-gcc-6.2.0
gcc/6.2.0-gcc-4.8 lz4/131-gcc-6.2.0 netlib-scalapack/2.0.2-gcc-6.2.0-netlib-openmpi py-nose/1.3.7-gcc-6.2.0 zlib/1.2.8-gcc-6.2.0
gmp/6.1.1-gcc-6.2.0 lzma/4.32.7-gcc-6.2.0 netlib-scalapack/2.0.2-gcc-6.2.0-openblas-mpich py-numpy/1.11.1-gcc-6.2.0-openblas
hwloc/1.11.4-gcc-6.2.0 lzo/2.09-gcc-6.2.0 netlib-scalapack/2.0.2-gcc-6.2.0-openblas-openmpi (D) py-scipy/0.18.1-gcc-6.2.0-openblas
libarchive/3.2.1-gcc-6.2.0 m4/1.4.17-gcc-6.2.0 nettle/3.2-gcc-6.2.0 py-setuptools/25.2.0-gcc-6.2.0
.. note::
TCL specific directive
The directives ``naming_scheme`` and ``conflict`` are TCL specific and do not apply
to the ``dotkit`` or ``lmod`` sections in the configuration file.
------------------------------------
Add custom environment modifications
------------------------------------
At many sites it is customary to set in module file for packages
an environment variable that points to the root folder where the package
is installed. You can achieve this using Spack by adding an
``environment`` directive to the configuration file:
.. code-block:: yaml
:emphasize-lines: 17-19
modules:
tcl:
hash_length: 0
naming_scheme: '${PACKAGE}/${VERSION}-${COMPILERNAME}-${COMPILERVER}'
whitelist:
- gcc
blacklist:
- '%gcc@4.8'
all:
conflict:
- '${PACKAGE}'
suffixes:
'^openblas': openblas
'^netlib-lapack': netlib
filter:
environment_blacklist: ['CPATH', 'LIBRARY_PATH']
environment:
set:
'${PACKAGE}_ROOT': '${PREFIX}'
netlib-scalapack:
suffixes:
'^openmpi': openmpi
'^mpich': mpich
There are many variable tokens available to use in the ``environment``
and ``naming_scheme`` directives, such as ``${PACKAGE}``,
``${VERSION}``, etc. (see :meth:`~spack.spec.Spec.format` API documentation
for the complete list).
Regenerating the module files should result in:
.. code-block:: console
:emphasize-lines: 14
$ spack module refresh -y --module-type tcl
==> Regenerating tcl module files
$ module show gcc
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
/home/mculpo/wdir/spack/share/spack/modules/linux-Ubuntu14-x86_64/gcc/6.2.0-gcc-4.8:
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
whatis("gcc @6.2.0 ")
prepend_path("PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/bin")
prepend_path("CMAKE_PREFIX_PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/")
prepend_path("MANPATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/share/man")
prepend_path("PKG_CONFIG_PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/lib64/pkgconfig")
prepend_path("LD_LIBRARY_PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/lib64")
setenv("GCC_ROOT","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u")
conflict("gcc")
help([[The GNU Compiler Collection includes front ends for C, C++, Objective-C,
Fortran, and Java.
]])
As you see the ``gcc`` module has the environment variable ``GCC_ROOT`` set.
Sometimes it's also useful to apply environment modifications selectively and target
only certain packages. You can for instance set the common variables ``CC``, ``CXX``,
etc. in the ``gcc`` module file and apply other custom modifications to the
``openmpi`` modules as follows:
.. code-block:: yaml
:emphasize-lines: 20-32
modules:
tcl:
hash_length: 0
naming_scheme: '${PACKAGE}/${VERSION}-${COMPILERNAME}-${COMPILERVER}'
whitelist:
- gcc
blacklist:
- '%gcc@4.8'
all:
conflict:
- '${PACKAGE}'
suffixes:
'^openblas': openblas
'^netlib-lapack': netlib
filter:
environment_blacklist: ['CPATH', 'LIBRARY_PATH']
environment:
set:
'${PACKAGE}_ROOT': '${PREFIX}'
gcc:
environment:
set:
CC: gcc
CXX: g++
FC: gfortran
F90: gfortran
F77: gfortran
openmpi:
environment:
set:
SLURM_MPI_TYPE: pmi2
OMPI_MCA_btl_openib_warn_default_gid_prefix: '0'
netlib-scalapack:
suffixes:
'^openmpi': openmpi
'^mpich': mpich
This time we will be more selective and regenerate only the ``gcc`` and
``openmpi`` module files:
.. code-block:: console
$ spack module refresh -y --module-type tcl gcc
==> Regenerating tcl module files
$ spack module refresh -y --module-type tcl openmpi
==> Regenerating tcl module files
$ module show gcc
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
/home/mculpo/wdir/spack/share/spack/modules/linux-Ubuntu14-x86_64/gcc/6.2.0-gcc-4.8:
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
whatis("gcc @6.2.0 ")
prepend_path("PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/bin")
prepend_path("CMAKE_PREFIX_PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/")
prepend_path("MANPATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/share/man")
prepend_path("PKG_CONFIG_PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/lib64/pkgconfig")
prepend_path("LD_LIBRARY_PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/lib64")
setenv("GCC_ROOT","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u")
setenv("CC","gcc")
setenv("CXX","g++")
setenv("F90","gfortran")
setenv("FC","gfortran")
setenv("F77","gfortran")
conflict("gcc")
help([[The GNU Compiler Collection includes front ends for C, C++, Objective-C,
Fortran, and Java.
]])
$ module show openmpi
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
/home/mculpo/wdir/spack/share/spack/modules/linux-Ubuntu14-x86_64/openmpi/2.0.1-gcc-6.2.0:
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
whatis("openmpi @2.0.1 ")
prepend_path("PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-6.2.0/openmpi-2.0.1-s3qbtbyh3y5y4gkchmhcuak7th44l53w/bin")
prepend_path("CMAKE_PREFIX_PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-6.2.0/openmpi-2.0.1-s3qbtbyh3y5y4gkchmhcuak7th44l53w/")
prepend_path("LD_LIBRARY_PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-6.2.0/openmpi-2.0.1-s3qbtbyh3y5y4gkchmhcuak7th44l53w/lib")
prepend_path("PKG_CONFIG_PATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-6.2.0/openmpi-2.0.1-s3qbtbyh3y5y4gkchmhcuak7th44l53w/lib/pkgconfig")
prepend_path("MANPATH","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-6.2.0/openmpi-2.0.1-s3qbtbyh3y5y4gkchmhcuak7th44l53w/share/man")
setenv("SLURM_MPI_TYPE","pmi2")
setenv("OMPI_MCA_BTL_OPENIB_WARN_DEFAULT_GID_PREFIX","0")
setenv("OPENMPI_ROOT","/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-6.2.0/openmpi-2.0.1-s3qbtbyh3y5y4gkchmhcuak7th44l53w")
conflict("openmpi")
help([[The Open MPI Project is an open source Message Passing Interface
implementation that is developed and maintained by a consortium of
academic, research, and industry partners. Open MPI is therefore able to
combine the expertise, technologies, and resources from all across the
High Performance Computing community in order to build the best MPI
library available. Open MPI offers advantages for system and software
vendors, application developers and computer science researchers.
]])
---------------------
Autoload dependencies
---------------------
Spack can also generate module files that contain code to load
the dependencies automatically. You can for instance generate python
modules that load their dependencies by adding the ``autoload`` directive:
.. code-block:: yaml
:emphasize-lines: 37,38
modules:
tcl:
hash_length: 0
naming_scheme: '${PACKAGE}/${VERSION}-${COMPILERNAME}-${COMPILERVER}'
whitelist:
- gcc
blacklist:
- '%gcc@4.8'
all:
conflict:
- '${PACKAGE}'
suffixes:
'^openblas': openblas
'^netlib-lapack': netlib
filter:
environment_blacklist: ['CPATH', 'LIBRARY_PATH']
environment:
set:
'${PACKAGE}_ROOT': '${PREFIX}'
gcc:
environment:
set:
CC: gcc
CXX: g++
FC: gfortran
F90: gfortran
F77: gfortran
openmpi:
environment:
set:
SLURM_MPI_TYPE: pmi2
OMPI_MCA_btl_openib_warn_default_gid_prefix: '0'
netlib-scalapack:
suffixes:
'^openmpi': openmpi
'^mpich': mpich
^python:
autoload: 'direct'
and regenerating the module files for every package that depends on ``python``:
.. code-block:: console
$ spack module refresh -y --module-type tcl ^python
==> Regenerating tcl module files
Now the ``py-scipy`` module will be:
.. code-block:: tcl
#%Module1.0
## Module file created by spack (https://github.com/LLNL/spack) on 2016-11-02 20:53:21.283547
##
## py-scipy@0.18.1%gcc@6.2.0 arch=linux-Ubuntu14-x86_64-e6uljfi
##
module-whatis "py-scipy @0.18.1"
proc ModulesHelp { } {
puts stderr "SciPy (pronounced "Sigh Pie") is a Scientific Library for Python. It"
puts stderr "provides many user-friendly and efficient numerical routines such as"
puts stderr "routines for numerical integration and optimization."
}
if ![ is-loaded python/2.7.12-gcc-6.2.0 ] {
puts stderr "Autoloading python/2.7.12-gcc-6.2.0"
module load python/2.7.12-gcc-6.2.0
}
if ![ is-loaded openblas/0.2.19-gcc-6.2.0 ] {
puts stderr "Autoloading openblas/0.2.19-gcc-6.2.0"
module load openblas/0.2.19-gcc-6.2.0
}
if ![ is-loaded py-numpy/1.11.1-gcc-6.2.0-openblas ] {
puts stderr "Autoloading py-numpy/1.11.1-gcc-6.2.0-openblas"
module load py-numpy/1.11.1-gcc-6.2.0-openblas
}
prepend-path CMAKE_PREFIX_PATH "/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-6.2.0/py-scipy-0.18.1-e6uljfiffgym4xvj6wveevqxfqnfb3gh/"
prepend-path LD_LIBRARY_PATH "/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-6.2.0/py-scipy-0.18.1-e6uljfiffgym4xvj6wveevqxfqnfb3gh/lib"
prepend-path PYTHONPATH "/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-6.2.0/py-scipy-0.18.1-e6uljfiffgym4xvj6wveevqxfqnfb3gh/lib/python2.7/site-packages"
setenv PY_SCIPY_ROOT "/home/mculpo/wdir/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-6.2.0/py-scipy-0.18.1-e6uljfiffgym4xvj6wveevqxfqnfb3gh"
conflict py-scipy
and will contain code to autoload all the dependencies:
.. code-block:: console
$ module load py-scipy
Autoloading python/2.7.12-gcc-6.2.0
Autoloading openblas/0.2.19-gcc-6.2.0
Autoloading py-numpy/1.11.1-gcc-6.2.0-openblas
-----------------------------
Lua hierarchical module files
-----------------------------
In the final part of this tutorial you will modify ``modules.yaml`` to generate
Lua hierarchical module files. You will see that most of the directives used before
are also valid in the ``lmod`` context.
^^^^^^^^^^^^^^^^^
Core/Compiler/MPI
^^^^^^^^^^^^^^^^^
.. warning::
Only LMod supports Lua hierarchical module files
For this part of the tutorial you need to have LMod as a tool to
manage your environment.
The most common hierarchy is the so called ``Core/Compiler/MPI``. To have an idea
how a hierarchy is organized you may refer to the
`Lmod guide <https://www.tacc.utexas.edu/research-development/tacc-projects/lmod/user-guide/module-hierarchy>`_.
Since ``lmod`` is not enabled by default, you need to add it to the list of
enabled module file generators. The other things you need to do are:
- change the ``tcl`` tag to ``lmod``
- remove ``tcl`` specific directives (``naming_scheme`` and ``conflict``)
- set which compilers are considered ``core``
- remove the ``mpi`` related suffixes (as they will be substituted by hierarchies)
After modifications the configuration file will be:
.. code-block:: yaml
:emphasize-lines: 2-6
modules:
enable::
- lmod
lmod:
core_compilers:
- 'gcc@4.8'
hash_length: 0
whitelist:
- gcc
blacklist:
- '%gcc@4.8'
all:
suffixes:
'^openblas': openblas
'^netlib-lapack': netlib
filter:
environment_blacklist: ['CPATH', 'LIBRARY_PATH']
environment:
set:
'${PACKAGE}_ROOT': '${PREFIX}'
gcc:
environment:
set:
CC: gcc
CXX: g++
FC: gfortran
F90: gfortran
F77: gfortran
openmpi:
environment:
set:
SLURM_MPI_TYPE: pmi2
OMPI_MCA_btl_openib_warn_default_gid_prefix: '0'
The double colon after ``enable`` is intentional and it serves the purpose of
overriding the default list of enabled generators so that only ``lmod`` will be active
(see :ref:`the reference manual<config-overrides>` for a more detailed explanation
of config scopes).
The directive ``core_compilers`` accepts a list of compilers : everything built
using these compilers will create a module in the ``Core`` part of the hierarchy. It is
common practice to put OS provided compilers in the list and only build common utilities
and other compilers in ``Core``.
You can now regenerate the module files
.. code-block:: console
$ spack module refresh --module-type lmod --delete-tree -y
and update ``MODULEPATH`` to point to the ``Core`` folder:
.. code-block:: console
$ module unuse /home/mculpo/wdir/spack/share/spack/modules/linux-Ubuntu14-x86_64
$ module use /home/mculpo/wdir/spack/share/spack/lmod/linux-Ubuntu14-x86_64/Core
$ module avail
----------------------------------------------------------------------- /home/mculpo/wdir/spack/share/spack/lmod/linux-Ubuntu14-x86_64/Core -----------------------------------------------------------------------
gcc/6.2.0
As you see the only module visible now is ``gcc``. Loading that you will make
visible the ``Compiler`` part of the software stack that was built with ``gcc/6.2.0``:
.. code-block:: console
$ module load gcc
$ module avail
-------------------------------------------------------------------- /home/mculpo/wdir/spack/share/spack/lmod/linux-Ubuntu14-x86_64/gcc/6.2.0 ---------------------------------------------------------------------
binutils/2.27 curl/7.50.3 hwloc/1.11.4 libtool/2.4.6 lzo/2.09 netlib-lapack/3.6.1 openssl/1.0.2j py-scipy/0.18.1-openblas util-macros/1.19.0
bison/3.0.4 expat/2.2.0 libarchive/3.2.1 libxml2/2.9.4 m4/1.4.17 nettle/3.2 pkg-config/0.29.1 py-setuptools/25.2.0 xz/5.2.2
bzip2/1.0.6 flex/2.6.0 libpciaccess/0.13.4 lz4/131 mpich/3.2 openblas/0.2.19 py-nose/1.3.7 python/2.7.12 zlib/1.2.8
cmake/3.6.1 gmp/6.1.1 libsigsegv/2.10 lzma/4.32.7 ncurses/6.0 openmpi/2.0.1 py-numpy/1.11.1-openblas sqlite/3.8.5
----------------------------------------------------------------------- /home/mculpo/wdir/spack/share/spack/lmod/linux-Ubuntu14-x86_64/Core -----------------------------------------------------------------------
gcc/6.2.0 (L)
The same holds true for the ``MPI`` part of the stack, that you can enable by loading
either ``mpich`` or ``openmpi``. The nice features of LMod will become evident
once you'll try switching among different stacks:
.. code-block:: console
$ module load mpich
$ module avail
----------------------------------------------------------- /home/mculpo/wdir/spack/share/spack/lmod/linux-Ubuntu14-x86_64/mpich/3.2-5n5xoep/gcc/6.2.0 ------------------------------------------------------------
netlib-scalapack/2.0.2-netlib netlib-scalapack/2.0.2-openblas (D)
-------------------------------------------------------------------- /home/mculpo/wdir/spack/share/spack/lmod/linux-Ubuntu14-x86_64/gcc/6.2.0 ---------------------------------------------------------------------
binutils/2.27 curl/7.50.3 hwloc/1.11.4 libtool/2.4.6 lzo/2.09 netlib-lapack/3.6.1 openssl/1.0.2j py-scipy/0.18.1-openblas util-macros/1.19.0
bison/3.0.4 expat/2.2.0 libarchive/3.2.1 libxml2/2.9.4 m4/1.4.17 nettle/3.2 pkg-config/0.29.1 py-setuptools/25.2.0 xz/5.2.2
bzip2/1.0.6 flex/2.6.0 libpciaccess/0.13.4 lz4/131 mpich/3.2 (L) openblas/0.2.19 py-nose/1.3.7 python/2.7.12 zlib/1.2.8
cmake/3.6.1 gmp/6.1.1 libsigsegv/2.10 lzma/4.32.7 ncurses/6.0 openmpi/2.0.1 py-numpy/1.11.1-openblas sqlite/3.8.5
----------------------------------------------------------------------- /home/mculpo/wdir/spack/share/spack/lmod/linux-Ubuntu14-x86_64/Core -----------------------------------------------------------------------
gcc/6.2.0 (L)
$ module load openblas netlib-scalapack/2.0.2-openblas
$ module list
Currently Loaded Modules:
1) gcc/6.2.0 2) mpich/3.2 3) openblas/0.2.19 4) netlib-scalapack/2.0.2-openblas
$ module load openmpi
Lmod is automatically replacing "mpich/3.2" with "openmpi/2.0.1"
Due to MODULEPATH changes the following have been reloaded:
1) netlib-scalapack/2.0.2-openblas
This layout is already a great improvement over the usual non-hierarchical layout,
but it still has an asymmetry: ``LAPACK`` providers are semantically the same as ``MPI``
providers, but they are still not part of the hierarchy. One possible solution is shown next.
.. Activate lmod and turn the previous modifications into lmod:
Add core compilers
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Extend the hierarchy to other virtual providers
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. warning::
This is an experimental feature
Having a hierarchy deeper than ``Core``/``Compiler``/``MPI`` is an experimental
feature, still not fully supported by ``module spider``,
see `here <https://github.com/TACC/Lmod/issues/114>`_. Furthermore its use
with hierarchies more complex than ``Core``/``Compiler``/``MPI``/``LAPACK``
has not been thoroughly tested in production environments.
Spack permits you to generate Lua hierarchical module files where users
can add an arbitrary list of virtual providers to the triplet
``Core``/``Compiler``/``MPI``. A configuration file like:
.. code-block:: yaml
:emphasize-lines: 7,8
modules:
enable::
- lmod
lmod:
core_compilers:
- 'gcc@4.8'
hierarchical_scheme:
- lapack
hash_length: 0
whitelist:
- gcc
blacklist:
- '%gcc@4.8'
- readline
all:
filter:
environment_blacklist: ['CPATH', 'LIBRARY_PATH']
environment:
set:
'${PACKAGE}_ROOT': '${PREFIX}'
gcc:
environment:
set:
CC: gcc
CXX: g++
FC: gfortran
F90: gfortran
F77: gfortran
openmpi:
environment:
set:
SLURM_MPI_TYPE: pmi2
OMPI_MCA_btl_openib_warn_default_gid_prefix: '0'
will add ``lapack`` providers to the mix. After the usual regeneration of module files:
.. code-block:: console
$ module purge
$ spack module refresh --module-type lmod --delete-tree -y
==> Regenerating lmod module files
you will have something like:
.. code-block:: console
$ module load gcc
$ module load openblas
$ module load openmpi
$ module avail
--------------------------------------------- /home/mculpo/wdir/spack/share/spack/lmod/linux-Ubuntu14-x86_64/openblas/0.2.19-js33umc/openmpi/2.0.1-s3qbtby/gcc/6.2.0 ----------------------------------------------
netlib-scalapack/2.0.2
-------------------------------------------------------- /home/mculpo/wdir/spack/share/spack/lmod/linux-Ubuntu14-x86_64/openblas/0.2.19-js33umc/gcc/6.2.0 ---------------------------------------------------------
py-numpy/1.11.1 py-scipy/0.18.1
-------------------------------------------------------------------- /home/mculpo/wdir/spack/share/spack/lmod/linux-Ubuntu14-x86_64/gcc/6.2.0 ---------------------------------------------------------------------
binutils/2.27 curl/7.50.3 hwloc/1.11.4 libtool/2.4.6 lzo/2.09 netlib-lapack/3.6.1 openssl/1.0.2j python/2.7.12 zlib/1.2.8
bison/3.0.4 expat/2.2.0 libarchive/3.2.1 libxml2/2.9.4 m4/1.4.17 nettle/3.2 pkg-config/0.29.1 sqlite/3.8.5
bzip2/1.0.6 flex/2.6.0 libpciaccess/0.13.4 lz4/131 mpich/3.2 openblas/0.2.19 (L) py-nose/1.3.7 util-macros/1.19.0
cmake/3.6.1 gmp/6.1.1 libsigsegv/2.10 lzma/4.32.7 ncurses/6.0 openmpi/2.0.1 (L) py-setuptools/25.2.0 xz/5.2.2
----------------------------------------------------------------------- /home/mculpo/wdir/spack/share/spack/lmod/linux-Ubuntu14-x86_64/Core -----------------------------------------------------------------------
gcc/6.2.0 (L)
Now both the ``MPI`` and the ``LAPACK`` providers are handled by LMod as hierarchies:
.. code-block:: console
$ module load py-numpy netlib-scalapack
$ module load mpich
Lmod is automatically replacing "openmpi/2.0.1" with "mpich/3.2"
Due to MODULEPATH changes the following have been reloaded:
1) netlib-scalapack/2.0.2
$ module load netlib-lapack
Lmod is automatically replacing "openblas/0.2.19" with "netlib-lapack/3.6.1"
Inactive Modules:
1) py-numpy
Due to MODULEPATH changes the following have been reloaded:
1) netlib-scalapack/2.0.2
making the use of tags to differentiate them unnecessary.
Note that as we compiled ``py-numpy`` only with ``openblas`` the module
is made inactive when we switch the ``LAPACK`` provider. The user
environment will now be consistent by design!