hlrs-software-stack/spack
7
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

re-merged mainline develop

Browse source
This commit is contained in:
Gregory Becker 2016-05-27 13:13:19 -07:00
commit 9dad7c2ace
27 changed files with 1241 additions and 802 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
.flake8
View file

@ -8,6 +8,9 @@
# - E221: multiple spaces before operator
# - E241: multiple spaces after ‘,’
#
# Let people use terse Python features:
# - E731 : lambda expressions
#
# Spack allows wildcard imports:
# - F403: disable wildcard import
#
@ -16,5 +19,5 @@
# - F999: name name be undefined or undefined from star imports.
#
[flake8]
ignore = E221,E241,F403,F821,F999
ignore = E221,E241,E731,F403,F821,F999
max-line-length = 79

161
lib/spack/docs/basic_usage.rst
View file

@ -102,8 +102,8 @@ that the packages is installed:
==> adept-utils is already installed in /home/gamblin2/spack/opt/chaos_5_x86_64_ib/gcc@4.4.7/adept-utils@1.0-5adef8da.
==> Trying to fetch from https://github.com/hpc/mpileaks/releases/download/v1.0/mpileaks-1.0.tar.gz
######################################################################## 100.0%
==> Staging archive: /home/gamblin2/spack/var/spack/stage/mpileaks@1.0%gcc@4.4.7=chaos_5_x86_64_ib-59f6ad23/mpileaks-1.0.tar.gz
==> Created stage in /home/gamblin2/spack/var/spack/stage/mpileaks@1.0%gcc@4.4.7=chaos_5_x86_64_ib-59f6ad23.
==> Staging archive: /home/gamblin2/spack/var/spack/stage/mpileaks@1.0%gcc@4.4.7 arch=chaos_5_x86_64_ib-59f6ad23/mpileaks-1.0.tar.gz
==> Created stage in /home/gamblin2/spack/var/spack/stage/mpileaks@1.0%gcc@4.4.7 arch=chaos_5_x86_64_ib-59f6ad23.
==> No patches needed for mpileaks.
==> Building mpileaks.
@ -132,10 +132,10 @@ sites, as installing a version that one user needs will not disrupt
existing installations for other users.
In addition to different versions, Spack can customize the compiler,
compile-time options (variants), and platform (for cross compiles) of
an installation. Spack is unique in that it can also configure the
*dependencies* a package is built with. For example, two
configurations of the same version of a package, one built with boost
compile-time options (variants), compiler flags, and platform (for
cross compiles) of an installation. Spack is unique in that it can
also configure the *dependencies* a package is built with. For example,
two configurations of the same version of a package, one built with boost
1.39.0, and the other version built with version 1.43.0, can coexist.
This can all be done on the command line using the *spec* syntax.
@ -246,6 +246,12 @@ Packages are divided into groups according to their architecture and
compiler. Within each group, Spack tries to keep the view simple, and
only shows the version of installed packages.
``spack find`` can filter the package list based on the package name, spec, or
a number of properties of their installation status. For example, missing
dependencies of a spec can be shown with ``-m``, packages which were
explicitly installed with ``spack install <package>`` can be singled out with
``-e`` and those which have been pulled in only as dependencies with ``-E``.
In some cases, there may be different configurations of the *same*
version of a package installed. For example, there are two
installations of of ``libdwarf@20130729`` above. We can look at them
@ -328,6 +334,11 @@ of libelf would look like this:
-- chaos_5_x86_64_ib / gcc@4.4.7 --------------------------------
libdwarf@20130729-d9b90962
We can also search for packages that have a certain attribute. For example,
``spack find -l libdwarf +debug`` will show only installations of libdwarf
with the 'debug' compile-time option enabled, while ``spack find -l +debug``
will find every installed package with a 'debug' compile-time option enabled.
The full spec syntax is discussed in detail in :ref:`sec-specs`.
@ -457,6 +468,26 @@ For compilers, like ``clang``, that do not support Fortran, put
Once you save the file, the configured compilers will show up in the
list displayed by ``spack compilers``.
You can also add compiler flags to manually configured compilers. The
valid flags are ``cflags``, ``cxxflags``, ``fflags``, ``cppflags``,
``ldflags``, and ``ldlibs``. For example,::
...
chaos_5_x86_64_ib:
...
intel@15.0.0:
cc: /usr/local/bin/icc-15.0.024-beta
cxx: /usr/local/bin/icpc-15.0.024-beta
f77: /usr/local/bin/ifort-15.0.024-beta
fc: /usr/local/bin/ifort-15.0.024-beta
cppflags: -O3 -fPIC
...
These flags will be treated by spack as if they were enterred from
the command line each time this compiler is used. The compiler wrappers
then inject those flags into the compiler command. Compiler flags
enterred from the command line will be discussed in more detail in the
following section.
.. _sec-specs:
@ -474,7 +505,7 @@ the full syntax of specs.
Here is an example of a much longer spec than we've seen thus far::
mpileaks @1.2:1.4 %gcc@4.7.5 +debug -qt =bgqos_0 ^callpath @1.1 %gcc@4.7.2
mpileaks @1.2:1.4 %gcc@4.7.5 +debug -qt arch=bgq_os ^callpath @1.1 %gcc@4.7.2
If provided to ``spack install``, this will install the ``mpileaks``
library at some version between ``1.2`` and ``1.4`` (inclusive),
@ -492,8 +523,12 @@ More formally, a spec consists of the following pieces:
* ``%`` Optional compiler specifier, with an optional compiler version
(``gcc`` or ``gcc@4.7.3``)
* ``+`` or ``-`` or ``~`` Optional variant specifiers (``+debug``,
``-qt``, or ``~qt``)
* ``=`` Optional architecture specifier (``bgqos_0``)
``-qt``, or ``~qt``) for boolean variants
* ``name=<value>`` Optional variant specifiers that are not restricted to
boolean variants
* ``name=<value>`` Optional compiler flag specifiers. Valid flag names are
``cflags``, ``cxxflags``, ``fflags``, ``cppflags``, ``ldflags``, and ``ldlibs``.
* ``arch=<value>`` Optional architecture specifier (``arch=bgq_os``)
* ``^`` Dependency specs (``^callpath@1.1``)
There are two things to notice here. The first is that specs are
@ -573,7 +608,7 @@ compilers, variants, and architectures just like any other spec.
Specifiers are associated with the nearest package name to their left.
For example, above, ``@1.1`` and ``%gcc@4.7.2`` associates with the
``callpath`` package, while ``@1.2:1.4``, ``%gcc@4.7.5``, ``+debug``,
``-qt``, and ``=bgqos_0`` all associate with the ``mpileaks`` package.
``-qt``, and ``arch=bgq_os`` all associate with the ``mpileaks`` package.
In the diagram above, ``mpileaks`` depends on ``mpich`` with an
unspecified version, but packages can depend on other packages with
@ -629,22 +664,25 @@ based on site policies.
Variants
~~~~~~~~~~~~~~~~~~~~~~~
.. Note::
Variants are not yet supported, but will be in the next Spack
release (0.9), due in Q2 2015.
Variants are named options associated with a particular package, and
they can be turned on or off. For example, above, supplying
``+debug`` causes ``mpileaks`` to be built with debug flags. The
names of particular variants available for a package depend on what
was provided by the package author. ``spack info <package>`` will
Variants are named options associated with a particular package. They are
optional, as each package must provide default values for each variant it
makes available. Variants can be specified using
a flexible parameter syntax ``name=<value>``. For example,
``spack install libelf debug=True`` will install libelf build with debug
flags. The names of particular variants available for a package depend on
what was provided by the package author. ``spack into <package>`` will
provide information on what build variants are available.
Depending on the package a variant may be on or off by default. For
``mpileaks`` here, ``debug`` is off by default, and we turned it on
with ``+debug``. If a package is on by default you can turn it off by
either adding ``-name`` or ``~name`` to the spec.
For compatibility with earlier versions, variants which happen to be
boolean in nature can be specified by a syntax that represents turning
options on and off. For example, in the previous spec we could have
supplied ``libelf +debug`` with the same effect of enabling the debug
compile time option for the libelf package.
Depending on the package a variant may have any default value. For
``libelf`` here, ``debug`` is ``False`` by default, and we turned it on
with ``debug=True`` or ``+debug``. If a package is ``True`` by default
you can turn it off by either adding ``-name`` or ``~name`` to the spec.
There are two syntaxes here because, depending on context, ``~`` and
``-`` may mean different things. In most shells, the following will
@ -656,7 +694,7 @@ result in the shell performing home directory substitution:
mpileaks~debug # use this instead
If there is a user called ``debug``, the ``~`` will be incorrectly
expanded. In this situation, you would want to write ``mpileaks
expanded. In this situation, you would want to write ``libelf
-debug``. However, ``-`` can be ambiguous when included after a
package name without spaces:
@ -671,12 +709,35 @@ package, not a request for ``mpileaks`` built without ``debug``
options. In this scenario, you should write ``mpileaks~debug`` to
avoid ambiguity.
When spack normalizes specs, it prints them out with no spaces and
uses only ``~`` for disabled variants. We allow ``-`` and spaces on
the command line is provided for convenience and legibility.
When spack normalizes specs, it prints them out with no spaces boolean
variants using the backwards compatibility syntax and uses only ``~``
for disabled boolean variants. We allow ``-`` and spaces on the command
line is provided for convenience and legibility.
Architecture specifier
Compiler Flags
~~~~~~~~~~~~~~~~~~~~~~~
Compiler flags are specified using the same syntax as non-boolean variants,
but fulfill a different purpose. While the function of a variant is set by
the package, compiler flags are used by the compiler wrappers to inject
flags into the compile line of the build. Additionally, compiler flags are
inherited by dependencies. ``spack install libdwarf cppflags=\"-g\"`` will
install both libdwarf and libelf with the ``-g`` flag injected into their
compile line.
Notice that the value of the compiler flags must be escape quoted on the
command line. From within python files, the same spec would be specified
``libdwarf cppflags="-g"``. This is necessary because of how the shell
handles the quote symbols.
The six compiler flags are injected in the order of implicit make commands
in gnu autotools. If all flags are set, the order is
``$cppflags $cflags|$cxxflags $ldflags command $ldlibs`` for C and C++ and
``$fflags $cppflags $ldflags command $ldlibs`` for fortran.
Architecture specifiers
~~~~~~~~~~~~~~~~~~~~~~~
.. Note::
@ -684,12 +745,9 @@ Architecture specifier
Architecture specifiers are part of specs but are not yet
functional. They will be in Spack version 1.0, due in Q3 2015.
The architecture specifier starts with a ``=`` and also comes after
some package name within a spec. It allows a user to specify a
particular architecture for the package to be built. This is mostly
used for architectures that need cross-compilation, and in most cases,
users will not need to specify the architecture when they install a
package.
The architecture specifier looks identical to a variant specifier for a
non-boolean variant. The architecture can be specified only using the
reserved name ``arch`` (``arch=bgq_os``).
.. _sec-virtual-dependencies:
@ -767,6 +825,23 @@ any MPI implementation will do. If another package depends on
error. Likewise, if you try to plug in some package that doesn't
provide MPI, Spack will raise an error.
Specifying Specs by Hash
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Complicated specs can become cumbersome to enter on the command line,
especially when many of the qualifications are necessary to
distinguish between similar installs, for example when using the
``uninstall`` command. To avoid this, when referencing an existing spec,
Spack allows you to reference specs by their hash. We previously
discussed the spec hash that Spack computes. In place of a spec in any
command, substitute ``/<hash>`` where ``<hash>`` is any amount from
the beginning of a spec hash. If the given spec hash is sufficient
to be unique, Spack will replace the reference with the spec to which
it refers. Otherwise, it will prompt for a more qualified hash.
Note that this will not work to reinstall a depencency uninstalled by
``spack uninstall -f``.
.. _spack-providers:
``spack providers``
@ -996,8 +1071,8 @@ than one installed package matches it), then Spack will warn you:
$ spack load libelf
==> Error: Multiple matches for spec libelf. Choose one:
libelf@0.8.13%gcc@4.4.7=chaos_5_x86_64_ib
libelf@0.8.13%intel@15.0.0=chaos_5_x86_64_ib
libelf@0.8.13%gcc@4.4.7 arch=chaos_5_x86_64_ib
libelf@0.8.13%intel@15.0.0 arch=chaos_5_x86_64_ib
You can either type the ``spack load`` command again with a fully
qualified argument, or you can add just enough extra constraints to
@ -1276,7 +1351,7 @@ You can find extensions for your Python installation like this:
.. code-block:: sh
$ spack extensions python
==> python@2.7.8%gcc@4.4.7=chaos_5_x86_64_ib-703c7a96
==> python@2.7.8%gcc@4.4.7 arch=chaos_5_x86_64_ib-703c7a96
==> 36 extensions:
geos py-ipython py-pexpect py-pyside py-sip
py-basemap py-libxml2 py-pil py-pytz py-six
@ -1366,9 +1441,9 @@ installation:
.. code-block:: sh
$ spack activate py-numpy
==> Activated extension py-setuptools@11.3.1%gcc@4.4.7=chaos_5_x86_64_ib-3c74eb69 for python@2.7.8%gcc@4.4.7.
==> Activated extension py-nose@1.3.4%gcc@4.4.7=chaos_5_x86_64_ib-5f70f816 for python@2.7.8%gcc@4.4.7.
==> Activated extension py-numpy@1.9.1%gcc@4.4.7=chaos_5_x86_64_ib-66733244 for python@2.7.8%gcc@4.4.7.
==> Activated extension py-setuptools@11.3.1%gcc@4.4.7 arch=chaos_5_x86_64_ib-3c74eb69 for python@2.7.8%gcc@4.4.7.
==> Activated extension py-nose@1.3.4%gcc@4.4.7 arch=chaos_5_x86_64_ib-5f70f816 for python@2.7.8%gcc@4.4.7.
==> Activated extension py-numpy@1.9.1%gcc@4.4.7 arch=chaos_5_x86_64_ib-66733244 for python@2.7.8%gcc@4.4.7.
Several things have happened here. The user requested that
``py-numpy`` be activated in the ``python`` installation it was built
@ -1383,7 +1458,7 @@ packages listed as activated:
.. code-block:: sh
$ spack extensions python
==> python@2.7.8%gcc@4.4.7=chaos_5_x86_64_ib-703c7a96
==> python@2.7.8%gcc@4.4.7 arch=chaos_5_x86_64_ib-703c7a96
==> 36 extensions:
geos py-ipython py-pexpect py-pyside py-sip
py-basemap py-libxml2 py-pil py-pytz py-six
@ -1431,7 +1506,7 @@ dependencies, you can use ``spack activate -f``:
.. code-block:: sh
$ spack activate -f py-numpy
==> Activated extension py-numpy@1.9.1%gcc@4.4.7=chaos_5_x86_64_ib-66733244 for python@2.7.8%gcc@4.4.7.
==> Activated extension py-numpy@1.9.1%gcc@4.4.7 arch=chaos_5_x86_64_ib-66733244 for python@2.7.8%gcc@4.4.7.
.. _spack-deactivate:

88
lib/spack/docs/site_configuration.rst → lib/spack/docs/configuration.rst
View file

@ -1,6 +1,6 @@
.. _site-configuration:
.. _configuration:
Site configuration
Configuration
===================================
.. _temp-space:
@ -55,7 +55,7 @@ directory is.
External Packages
~~~~~~~~~~~~~~~~~~~~~
----------------------------
Spack can be configured to use externally-installed
packages rather than building its own packages. This may be desirable
if machines ship with system packages, such as a customized MPI
@ -70,15 +70,15 @@ directory. Here's an example of an external configuration:
packages:
openmpi:
paths:
openmpi@1.4.3%gcc@4.4.7=chaos_5_x86_64_ib: /opt/openmpi-1.4.3
openmpi@1.4.3%gcc@4.4.7=chaos_5_x86_64_ib+debug: /opt/openmpi-1.4.3-debug
openmpi@1.6.5%intel@10.1=chaos_5_x86_64_ib: /opt/openmpi-1.6.5-intel
openmpi@1.4.3%gcc@4.4.7 arch=chaos_5_x86_64_ib: /opt/openmpi-1.4.3
openmpi@1.4.3%gcc@4.4.7 arch=chaos_5_x86_64_ib+debug: /opt/openmpi-1.4.3-debug
openmpi@1.6.5%intel@10.1 arch=chaos_5_x86_64_ib: /opt/openmpi-1.6.5-intel
This example lists three installations of OpenMPI, one built with gcc,
one built with gcc and debug information, and another built with Intel.
If Spack is asked to build a package that uses one of these MPIs as a
dependency, it will use the the pre-installed OpenMPI in
the given directory.
the given directory.
Each ``packages.yaml`` begins with a ``packages:`` token, followed
by a list of package names. To specify externals, add a ``paths``
@ -108,9 +108,9 @@ be:
packages:
openmpi:
paths:
openmpi@1.4.3%gcc@4.4.7=chaos_5_x86_64_ib: /opt/openmpi-1.4.3
openmpi@1.4.3%gcc@4.4.7=chaos_5_x86_64_ib+debug: /opt/openmpi-1.4.3-debug
openmpi@1.6.5%intel@10.1=chaos_5_x86_64_ib: /opt/openmpi-1.6.5-intel
openmpi@1.4.3%gcc@4.4.7 arch=chaos_5_x86_64_ib: /opt/openmpi-1.4.3
openmpi@1.4.3%gcc@4.4.7 arch=chaos_5_x86_64_ib+debug: /opt/openmpi-1.4.3-debug
openmpi@1.6.5%intel@10.1 arch=chaos_5_x86_64_ib: /opt/openmpi-1.6.5-intel
buildable: False
The addition of the ``buildable`` flag tells Spack that it should never build
@ -118,13 +118,73 @@ its own version of OpenMPI, and it will instead always rely on a pre-built
OpenMPI. Similar to ``paths``, ``buildable`` is specified as a property under
a package name.
The ``buildable`` does not need to be paired with external packages.
It could also be used alone to forbid packages that may be
The ``buildable`` does not need to be paired with external packages.
It could also be used alone to forbid packages that may be
buggy or otherwise undesirable.
Concretization Preferences
--------------------------------
Spack can be configured to prefer certain compilers, package
versions, depends_on, and variants during concretization.
The preferred configuration can be controlled via the
``~/.spack/packages.yaml`` file for user configuations, or the
``etc/spack/packages.yaml`` site configuration.
Here's an example packages.yaml file that sets preferred packages:
.. code-block:: sh
packages:
dyninst:
compiler: [gcc@4.9]
variants: +debug
gperftools:
version: [2.2, 2.4, 2.3]
all:
compiler: [gcc@4.4.7, gcc@4.6:, intel, clang, pgi]
providers:
mpi: [mvapich, mpich, openmpi]
At a high level, this example is specifying how packages should be
concretized. The dyninst package should prefer using gcc 4.9 and
be built with debug options. The gperftools package should prefer version
2.2 over 2.4. Every package on the system should prefer mvapich for
its MPI and gcc 4.4.7 (except for Dyninst, which overrides this by preferring gcc 4.9).
These options are used to fill in implicit defaults. Any of them can be overwritten
on the command line if explicitly requested.
Each packages.yaml file begins with the string ``packages:`` and
package names are specified on the next level. The special string ``all``
applies settings to each package. Underneath each package name is
one or more components: ``compiler``, ``variants``, ``version``,
or ``providers``. Each component has an ordered list of spec
``constraints``, with earlier entries in the list being preferred over
later entries.
Sometimes a package installation may have constraints that forbid
the first concretization rule, in which case Spack will use the first
legal concretization rule. Going back to the example, if a user
requests gperftools 2.3 or later, then Spack will install version 2.4
as the 2.4 version of gperftools is preferred over 2.3.
An explicit concretization rule in the preferred section will always
take preference over unlisted concretizations. In the above example,
xlc isn't listed in the compiler list. Every listed compiler from
gcc to pgi will thus be preferred over the xlc compiler.
The syntax for the ``provider`` section differs slightly from other
concretization rules. A provider lists a value that packages may
``depend_on`` (e.g, mpi) and a list of rules for fulfilling that
dependency.
Profiling
~~~~~~~~~~~~~~~~~~~~~
------------------
Spack has some limited built-in support for profiling, and can report
statistics using standard Python timing tools. To use this feature,
@ -133,7 +193,7 @@ supply ``-p`` to Spack on the command line, before any subcommands.
.. _spack-p:
``spack -p``
^^^^^^^^^^^^^^^^^^
~~~~~~~~~~~~~~~~~
``spack -p`` output looks like this:

17
lib/spack/docs/features.rst
View file

@ -31,14 +31,21 @@ platform, all on the command line.
# Specify a compiler (and its version), with %
$ spack install mpileaks@1.1.2 %gcc@4.7.3
# Add special compile-time options with +
# Add special compile-time options by name
$ spack install mpileaks@1.1.2 %gcc@4.7.3 debug=True
# Add special boolean compile-time options with +
$ spack install mpileaks@1.1.2 %gcc@4.7.3 +debug
# Cross-compile for a different architecture with =
$ spack install mpileaks@1.1.2 =bgqos_0
# Add compiler flags using the conventional names
$ spack install mpileaks@1.1.2 %gcc@4.7.3 cppflags=\"-O3 -floop-block\"
Users can specify as many or few options as they care about. Spack
will fill in the unspecified values with sensible defaults.
# Cross-compile for a different architecture with arch=
$ spack install mpileaks@1.1.2 arch=bgqos_0
Users can specify as many or few options as they care about. Spack
will fill in the unspecified values with sensible defaults. The two listed
syntaxes for variants are identical when the value is boolean.
Customize dependencies

2
lib/spack/docs/index.rst
View file

@ -47,7 +47,7 @@ Table of Contents
basic_usage
packaging_guide
mirrors
site_configuration
configuration
developer_guide
command_index
package_list

108
lib/spack/docs/packaging_guide.rst
View file

@ -1221,11 +1221,13 @@ just as easily provide a version range:
depends_on("libelf@0.8.2:0.8.4:")
Or a requirement for a particular variant:
Or a requirement for a particular variant or compiler flags:
.. code-block:: python
depends_on("libelf@0.8+debug")
depends_on('libelf debug=True')
depends_on('libelf cppflags="-fPIC")
Both users *and* package authors can use the same spec syntax to refer
to different package configurations. Users use the spec syntax on the
@ -1623,21 +1625,21 @@ the user runs ``spack install`` and the time the ``install()`` method
is called. The concretized version of the spec above might look like
this::
mpileaks@2.3%gcc@4.7.3=linux-ppc64
^callpath@1.0%gcc@4.7.3+debug=linux-ppc64
^dyninst@8.1.2%gcc@4.7.3=linux-ppc64
^libdwarf@20130729%gcc@4.7.3=linux-ppc64
^libelf@0.8.11%gcc@4.7.3=linux-ppc64
^mpich@3.0.4%gcc@4.7.3=linux-ppc64
mpileaks@2.3%gcc@4.7.3 arch=linux-ppc64
^callpath@1.0%gcc@4.7.3+debug arch=linux-ppc64
^dyninst@8.1.2%gcc@4.7.3 arch=linux-ppc64
^libdwarf@20130729%gcc@4.7.3 arch=linux-ppc64
^libelf@0.8.11%gcc@4.7.3 arch=linux-ppc64
^mpich@3.0.4%gcc@4.7.3 arch=linux-ppc64
.. graphviz::
digraph {
"mpileaks@2.3\n%gcc@4.7.3\n=linux-ppc64" -> "mpich@3.0.4\n%gcc@4.7.3\n=linux-ppc64"
"mpileaks@2.3\n%gcc@4.7.3\n=linux-ppc64" -> "callpath@1.0\n%gcc@4.7.3+debug\n=linux-ppc64" -> "mpich@3.0.4\n%gcc@4.7.3\n=linux-ppc64"
"callpath@1.0\n%gcc@4.7.3+debug\n=linux-ppc64" -> "dyninst@8.1.2\n%gcc@4.7.3\n=linux-ppc64"
"dyninst@8.1.2\n%gcc@4.7.3\n=linux-ppc64" -> "libdwarf@20130729\n%gcc@4.7.3\n=linux-ppc64" -> "libelf@0.8.11\n%gcc@4.7.3\n=linux-ppc64"
"dyninst@8.1.2\n%gcc@4.7.3\n=linux-ppc64" -> "libelf@0.8.11\n%gcc@4.7.3\n=linux-ppc64"
"mpileaks@2.3\n%gcc@4.7.3\n arch=linux-ppc64" -> "mpich@3.0.4\n%gcc@4.7.3\n arch=linux-ppc64"
"mpileaks@2.3\n%gcc@4.7.3\n arch=linux-ppc64" -> "callpath@1.0\n%gcc@4.7.3+debug\n arch=linux-ppc64" -> "mpich@3.0.4\n%gcc@4.7.3\n arch=linux-ppc64"
"callpath@1.0\n%gcc@4.7.3+debug\n arch=linux-ppc64" -> "dyninst@8.1.2\n%gcc@4.7.3\n arch=linux-ppc64"
"dyninst@8.1.2\n%gcc@4.7.3\n arch=linux-ppc64" -> "libdwarf@20130729\n%gcc@4.7.3\n arch=linux-ppc64" -> "libelf@0.8.11\n%gcc@4.7.3\n arch=linux-ppc64"
"dyninst@8.1.2\n%gcc@4.7.3\n arch=linux-ppc64" -> "libelf@0.8.11\n%gcc@4.7.3\n arch=linux-ppc64"
}
Here, all versions, compilers, and platforms are filled in, and there
@ -1648,8 +1650,8 @@ point will Spack call the ``install()`` method for your package.
Concretization in Spack is based on certain selection policies that
tell Spack how to select, e.g., a version, when one is not specified
explicitly. Concretization policies are discussed in more detail in
:ref:`site-configuration`. Sites using Spack can customize them to
match the preferences of their own users.
:ref:`configuration`. Sites using Spack can customize them to match
the preferences of their own users.
.. _spack-spec:
@ -1666,9 +1668,9 @@ running ``spack spec``. For example:
^libdwarf
^libelf
dyninst@8.0.1%gcc@4.7.3=linux-ppc64
^libdwarf@20130729%gcc@4.7.3=linux-ppc64
^libelf@0.8.13%gcc@4.7.3=linux-ppc64
dyninst@8.0.1%gcc@4.7.3 arch=linux-ppc64
^libdwarf@20130729%gcc@4.7.3 arch=linux-ppc64
^libelf@0.8.13%gcc@4.7.3 arch=linux-ppc64
This is useful when you want to know exactly what Spack will do when
you ask for a particular spec.
@ -1682,60 +1684,8 @@ be concretized on their system. For example, one user may prefer packages
built with OpenMPI and the Intel compiler. Another user may prefer
packages be built with MVAPICH and GCC.
Spack can be configured to prefer certain compilers, package
versions, depends_on, and variants during concretization.
The preferred configuration can be controlled via the
``~/.spack/packages.yaml`` file for user configuations, or the
``etc/spack/packages.yaml`` site configuration.
Here's an example packages.yaml file that sets preferred packages:
.. code-block:: sh
packages:
dyninst:
compiler: [gcc@4.9]
variants: +debug
gperftools:
version: [2.2, 2.4, 2.3]
all:
compiler: [gcc@4.4.7, gcc@4.6:, intel, clang, pgi]
providers:
mpi: [mvapich, mpich, openmpi]
At a high level, this example is specifying how packages should be
concretized. The dyninst package should prefer using gcc 4.9 and
be built with debug options. The gperftools package should prefer version
2.2 over 2.4. Every package on the system should prefer mvapich for
its MPI and gcc 4.4.7 (except for Dyninst, which overrides this by preferring gcc 4.9).
These options are used to fill in implicit defaults. Any of them can be overwritten
on the command line if explicitly requested.
Each packages.yaml file begins with the string ``packages:`` and
package names are specified on the next level. The special string ``all``
applies settings to each package. Underneath each package name is
one or more components: ``compiler``, ``variants``, ``version``,
or ``providers``. Each component has an ordered list of spec
``constraints``, with earlier entries in the list being preferred over
later entries.
Sometimes a package installation may have constraints that forbid
the first concretization rule, in which case Spack will use the first
legal concretization rule. Going back to the example, if a user
requests gperftools 2.3 or later, then Spack will install version 2.4
as the 2.4 version of gperftools is preferred over 2.3.
An explicit concretization rule in the preferred section will always
take preference over unlisted concretizations. In the above example,
xlc isn't listed in the compiler list. Every listed compiler from
gcc to pgi will thus be preferred over the xlc compiler.
The syntax for the ``provider`` section differs slightly from other
concretization rules. A provider lists a value that packages may
``depend_on`` (e.g, mpi) and a list of rules for fulfilling that
dependency.
See the `documentation in the config section <concretization-preferences_>`_
for more details.
.. _install-method:
@ -1960,6 +1910,12 @@ the command line.
``$rpath_flag`` can be overriden on a compiler specific basis in
``lib/spack/spack/compilers/$compiler.py``.
The compiler wrappers also pass the compiler flags specified by the user from
the command line (``cflags``, ``cxxflags``, ``fflags``, ``cppflags``, ``ldflags``,
and/or ``ldlibs``). They do not override the canonical autotools flags with the
same names (but in ALL-CAPS) that may be passed into the build by particularly
challenging package scripts.
Compiler flags
~~~~~~~~~~~~~~
In rare circumstances such as compiling and running small unit tests, a package
@ -2206,12 +2162,12 @@ example:
def install(self, prefix):
# Do default install
@when('=chaos_5_x86_64_ib')
@when('arch=chaos_5_x86_64_ib')
def install(self, prefix):
# This will be executed instead of the default install if
# the package's sys_type() is chaos_5_x86_64_ib.
@when('=bgqos_0")
@when('arch=bgqos_0")
def install(self, prefix):
# This will be executed if the package's sys_type is bgqos_0
@ -2801,11 +2757,11 @@ build it:
$ spack stage libelf
==> Trying to fetch from http://www.mr511.de/software/libelf-0.8.13.tar.gz
######################################################################## 100.0%
==> Staging archive: /Users/gamblin2/src/spack/var/spack/stage/libelf@0.8.13%gcc@4.8.3=linux-ppc64/libelf-0.8.13.tar.gz
==> Created stage in /Users/gamblin2/src/spack/var/spack/stage/libelf@0.8.13%gcc@4.8.3=linux-ppc64.
==> Staging archive: /Users/gamblin2/src/spack/var/spack/stage/libelf@0.8.13%gcc@4.8.3 arch=linux-ppc64/libelf-0.8.13.tar.gz
==> Created stage in /Users/gamblin2/src/spack/var/spack/stage/libelf@0.8.13%gcc@4.8.3 arch=linux-ppc64.
$ spack cd libelf
$ pwd
/Users/gamblin2/src/spack/var/spack/stage/libelf@0.8.13%gcc@4.8.3=linux-ppc64/libelf-0.8.13
/Users/gamblin2/src/spack/var/spack/stage/libelf@0.8.13%gcc@4.8.3 arch=linux-ppc64/libelf-0.8.13
``spack cd`` here changed he current working directory to the
directory containing the expanded ``libelf`` source code. There are a

80
lib/spack/spack/cmd/find.py
View file

@ -38,71 +38,59 @@
def setup_parser(subparser):
format_group = subparser.add_mutually_exclusive_group()
format_group.add_argument('-s',
'--short',
format_group.add_argument('-s', '--short',
action='store_const',
dest='mode',
const='short',
help='Show only specs (default)')
format_group.add_argument('-p',
'--paths',
format_group.add_argument('-p', '--paths',
action='store_const',
dest='mode',
const='paths',
help='Show paths to package install directories')
format_group.add_argument(
'-d',
'--deps',
'-d', '--deps',
action='store_const',
dest='mode',
const='deps',
help='Show full dependency DAG of installed packages')
subparser.add_argument('-l',
'--long',
subparser.add_argument('-l', '--long',
action='store_true',
dest='long',
help='Show dependency hashes as well as versions.')
subparser.add_argument('-L',
'--very-long',
subparser.add_argument('-L', '--very-long',
action='store_true',
dest='very_long',
help='Show dependency hashes as well as versions.')
subparser.add_argument('-f',
'--show-flags',
subparser.add_argument('-f', '--show-flags',
action='store_true',
dest='show_flags',
help='Show spec compiler flags.')
subparser.add_argument(
'-e',
'--explicit',
'-e', '--explicit',
action='store_true',
help='Show only specs that were installed explicitly')
subparser.add_argument(
'-E',
'--implicit',
'-E', '--implicit',
action='store_true',
help='Show only specs that were installed as dependencies')
subparser.add_argument(
'-u',
'--unknown',
'-u', '--unknown',
action='store_true',
dest='unknown',
help='Show only specs Spack does not have a package for.')
subparser.add_argument(
'-m',
'--missing',
'-m', '--missing',
action='store_true',
dest='missing',
help='Show missing dependencies as well as installed specs.')
subparser.add_argument('-M',
'--only-missing',
subparser.add_argument('-M', '--only-missing',
action='store_true',
dest='only_missing',
help='Show only missing dependencies.')
subparser.add_argument('-N',
'--namespace',
subparser.add_argument('-N', '--namespace',
action='store_true',
help='Show fully qualified package names.')
@ -188,7 +176,32 @@ def fmt(s):
print(hsh + spec.format(format_string, color=True) + '\n')
else:
raise ValueError("Invalid mode for display_specs: %s. Must be one of (paths, deps, short)." % mode) # NOQA: ignore=E501
raise ValueError(
"Invalid mode for display_specs: %s. Must be one of (paths,"
"deps, short)." % mode) # NOQA: ignore=E501
def query_arguments(args):
# Check arguments
if args.explicit and args.implicit:
tty.error('You can\'t pass -E and -e options simultaneously.')
raise SystemExit(1)
# Set up query arguments.
installed, known = True, any
if args.only_missing:
installed = False
elif args.missing:
installed = any
if args.unknown:
known = False
explicit = any
if args.explicit:
explicit = True
if args.implicit:
explicit = False
q_args = {'installed': installed, 'known': known, "explicit": explicit}
return q_args
def find(parser, args):
@ -205,22 +218,7 @@ def find(parser, args):
if not query_specs:
return
# Set up query arguments.
installed, known = True, any
if args.only_missing:
installed = False
elif args.missing:
installed = any
if args.unknown:
known = False
explicit = any
if args.explicit:
explicit = False
if args.implicit:
explicit = True
q_args = {'installed': installed, 'known': known, "explicit": explicit}
q_args = query_arguments(args)
# Get all the specs the user asked for
if not query_specs:

99
lib/spack/spack/database.py
View file

@ -40,7 +40,6 @@
"""
import os
import time
import socket
import yaml
@ -56,6 +55,7 @@
from spack.error import SpackError
from spack.repository import UnknownPackageError
# DB goes in this directory underneath the root
_db_dirname = '.spack-db'
@ -69,10 +69,12 @@
def _autospec(function):
"""Decorator that automatically converts the argument of a single-arg
function to a Spec."""
def converter(self, spec_like, *args, **kwargs):
if not isinstance(spec_like, spack.spec.Spec):
spec_like = spack.spec.Spec(spec_like)
return function(self, spec_like, *args, **kwargs)
return converter
@ -92,6 +94,7 @@ class InstallRecord(object):
dependents left.
"""
def __init__(self, spec, path, installed, ref_count=0, explicit=False):
self.spec = spec
self.path = str(path)
@ -100,16 +103,19 @@ def __init__(self, spec, path, installed, ref_count=0, explicit=False):
self.explicit = explicit
def to_dict(self):
return { 'spec' : self.spec.to_node_dict(),
'path' : self.path,
'installed' : self.installed,
'ref_count' : self.ref_count,
'explicit' : self.explicit }
return {
'spec': self.spec.to_node_dict(),
'path': self.path,
'installed': self.installed,
'ref_count': self.ref_count,
'explicit': self.explicit
}
@classmethod
def from_dict(cls, spec, dictionary):
d = dictionary
return InstallRecord(spec, d['path'], d['installed'], d['ref_count'], d.get('explicit', False))
return InstallRecord(spec, d['path'], d['installed'], d['ref_count'],
d.get('explicit', False))
class Database(object):
@ -144,7 +150,7 @@ def __init__(self, root, db_dir=None):
# Set up layout of database files within the db dir
self._index_path = join_path(self._db_dir, 'index.yaml')
self._lock_path = join_path(self._db_dir, 'lock')
self._lock_path = join_path(self._db_dir, 'lock')
# Create needed directories and files
if not os.path.exists(self._db_dir):
@ -157,17 +163,14 @@ def __init__(self, root, db_dir=None):
self.lock = Lock(self._lock_path)
self._data = {}
def write_transaction(self, timeout=_db_lock_timeout):
"""Get a write lock context manager for use in a `with` block."""
return WriteTransaction(self, self._read, self._write, timeout)
def read_transaction(self, timeout=_db_lock_timeout):
"""Get a read lock context manager for use in a `with` block."""
return ReadTransaction(self, self._read, None, timeout)
def _write_to_yaml(self, stream):
"""Write out the databsae to a YAML file.
@ -183,9 +186,9 @@ def _write_to_yaml(self, stream):
# different paths, it can't differentiate.
# TODO: fix this before we support multiple install locations.
database = {
'database' : {
'installs' : installs,
'version' : str(_db_version)
'database': {
'installs': installs,
'version': str(_db_version)
}
}
@ -194,15 +197,11 @@ def _write_to_yaml(self, stream):
except YAMLError as e:
raise SpackYAMLError("error writing YAML database:", str(e))
def _read_spec_from_yaml(self, hash_key, installs, parent_key=None):
"""Recursively construct a spec from a hash in a YAML database.
Does not do any locking.
"""
if hash_key not in installs:
parent = read_spec(installs[parent_key]['path'])
spec_dict = installs[hash_key]['spec']
# Install records don't include hash with spec, so we add it in here
@ -224,7 +223,6 @@ def _read_spec_from_yaml(self, hash_key, installs, parent_key=None):
spec._mark_concrete()
return spec
def _read_from_yaml(self, stream):
"""
Fill database from YAML, do not maintain old data
@ -246,15 +244,15 @@ def _read_from_yaml(self, stream):
return
def check(cond, msg):
if not cond: raise CorruptDatabaseError(self._index_path, msg)
if not cond:
raise CorruptDatabaseError(self._index_path, msg)
check('database' in yfile, "No 'database' attribute in YAML.")
# High-level file checks
db = yfile['database']
check('installs' in db, "No 'installs' in YAML DB.")
check('version' in db, "No 'version' in YAML DB.")
check('version' in db, "No 'version' in YAML DB.")
installs = db['installs']
@ -277,25 +275,25 @@ def check(cond, msg):
# hashes are the same.
spec_hash = spec.dag_hash()
if not spec_hash == hash_key:
tty.warn("Hash mismatch in database: %s -> spec with hash %s"
% (hash_key, spec_hash))
continue # TODO: is skipping the right thing to do?
tty.warn(
"Hash mismatch in database: %s -> spec with hash %s" %
(hash_key, spec_hash))
continue # TODO: is skipping the right thing to do?
# Insert the brand new spec in the database. Each
# spec has its own copies of its dependency specs.
# TODO: would a more immmutable spec implementation simplify this?
# TODO: would a more immmutable spec implementation simplify
# this?
data[hash_key] = InstallRecord.from_dict(spec, rec)
except Exception as e:
tty.warn("Invalid database reecord:",
"file: %s" % self._index_path,
"hash: %s" % hash_key,
"cause: %s" % str(e))
"hash: %s" % hash_key, "cause: %s" % str(e))
raise
self._data = data
def reindex(self, directory_layout):
"""Build database index from scratch based from a directory layout.
@ -320,7 +318,6 @@ def reindex(self, directory_layout):
self._data = old_data
raise
def _check_ref_counts(self):
"""Ensure consistency of reference counts in the DB.
@ -342,9 +339,8 @@ def _check_ref_counts(self):
found = rec.ref_count
if not expected == found:
raise AssertionError(
"Invalid ref_count: %s: %d (expected %d), in DB %s"
% (key, found, expected, self._index_path))
"Invalid ref_count: %s: %d (expected %d), in DB %s" %
(key, found, expected, self._index_path))
def _write(self):
"""Write the in-memory database index to its file path.
@ -366,7 +362,6 @@ def _write(self):
os.remove(temp_file)
raise
def _read(self):
"""Re-read Database from the data in the set location.
@ -381,7 +376,6 @@ def _read(self):
# reindex() takes its own write lock, so no lock here.
self.reindex(spack.install_layout)
def _add(self, spec, path, directory_layout=None, explicit=False):
"""Add an install record for spec at path to the database.
@ -404,11 +398,11 @@ def _add(self, spec, path, directory_layout=None, explicit=False):
rec.path = path
else:
self._data[key] = InstallRecord(spec, path, True, explicit=explicit)
self._data[key] = InstallRecord(spec, path, True,
explicit=explicit)
for dep in spec.dependencies.values():
self._increment_ref_count(dep, directory_layout)
def _increment_ref_count(self, spec, directory_layout=None):
"""Recursively examine dependencies and update their DB entries."""
key = spec.dag_hash()
@ -438,28 +432,25 @@ def add(self, spec, path, explicit=False):
with self.write_transaction():
self._add(spec, path, explicit=explicit)
def _get_matching_spec_key(self, spec, **kwargs):
"""Get the exact spec OR get a single spec that matches."""
key = spec.dag_hash()
if not key in self._data:
if key not in self._data:
match = self.query_one(spec, **kwargs)
if match:
return match.dag_hash()
raise KeyError("No such spec in database! %s" % spec)
return key
@_autospec
def get_record(self, spec, **kwargs):
key = self._get_matching_spec_key(spec, **kwargs)
return self._data[key]
def _decrement_ref_count(self, spec):
key = spec.dag_hash()
if not key in self._data:
if key not in self._data:
# TODO: print something here? DB is corrupt, but
# not much we can do.
return
@ -472,7 +463,6 @@ def _decrement_ref_count(self, spec):
for dep in spec.dependencies.values():
self._decrement_ref_count(dep)
def _remove(self, spec):
"""Non-locking version of remove(); does real work.
"""
@ -491,7 +481,6 @@ def _remove(self, spec):
# query spec was passed in.
return rec.spec
@_autospec
def remove(self, spec):
"""Removes a spec from the database. To be called on uninstall.
@ -508,7 +497,6 @@ def remove(self, spec):
with self.write_transaction():
return self._remove(spec)
@_autospec
def installed_extensions_for(self, extendee_spec):
"""
@ -519,12 +507,11 @@ def installed_extensions_for(self, extendee_spec):
try:
if s.package.extends(extendee_spec):
yield s.package
except UnknownPackageError as e:
except UnknownPackageError:
continue
# skips unknown packages
# TODO: conditional way to do this instead of catching exceptions
def query(self, query_spec=any, known=any, installed=True, explicit=any):
"""Run a query on the database.
@ -567,14 +554,14 @@ def query(self, query_spec=any, known=any, installed=True, explicit=any):
continue
if explicit is not any and rec.explicit != explicit:
continue
if known is not any and spack.repo.exists(rec.spec.name) != known:
if known is not any and spack.repo.exists(
rec.spec.name) != known:
continue
if query_spec is any or rec.spec.satisfies(query_spec):
results.append(rec.spec)
return sorted(results)
def query_one(self, query_spec, known=any, installed=True):
"""Query for exactly one spec that matches the query spec.
@ -586,10 +573,9 @@ def query_one(self, query_spec, known=any, installed=True):
assert len(concrete_specs) <= 1
return concrete_specs[0] if concrete_specs else None
def missing(self, spec):
with self.read_transaction():
key = spec.dag_hash()
key = spec.dag_hash()
return key in self._data and not self._data[key].installed
@ -601,7 +587,10 @@ class _Transaction(object):
Timeout for lock is customizable.
"""
def __init__(self, db, acquire_fn=None, release_fn=None,
def __init__(self, db,
acquire_fn=None,
release_fn=None,
timeout=_db_lock_timeout):
self._db = db
self._timeout = timeout
@ -636,11 +625,11 @@ def _exit(self):
class CorruptDatabaseError(SpackError):
def __init__(self, path, msg=''):
super(CorruptDatabaseError, self).__init__(
"Spack database is corrupt: %s. %s" %(path, msg))
"Spack database is corrupt: %s. %s" % (path, msg))
class InvalidDatabaseVersionError(SpackError):
def __init__(self, expected, found):
super(InvalidDatabaseVersionError, self).__init__(
"Expected database version %s but found version %s"
% (expected, found))
"Expected database version %s but found version %s" %
(expected, found))

33
lib/spack/spack/fetch_strategy.py
View file

@ -1,4 +1,4 @@
##############################################################################
#
# Copyright (c) 2013-2016, Lawrence Livermore National Security, LLC.
# Produced at the Lawrence Livermore National Laboratory.
#
@ -21,7 +21,7 @@
# You should have received a copy of the GNU Lesser General Public
# License along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
##############################################################################
#
"""
Fetch strategies are used to download source code into a staging area
in order to build it. They need to define the following methods:
@ -75,11 +75,13 @@ def wrapper(self, *args, **kwargs):
class FetchStrategy(object):
"""Superclass of all fetch strategies."""
enabled = False # Non-abstract subclasses should be enabled.
required_attributes = None # Attributes required in version() args.
class __metaclass__(type):
"""This metaclass registers all fetch strategies in a list."""
def __init__(cls, name, bases, dict):
@ -126,6 +128,7 @@ def matches(cls, args):
@pattern.composite(interface=FetchStrategy)
class FetchStrategyComposite(object):
"""
Composite for a FetchStrategy object. Implements the GoF composite pattern.
"""
@ -134,6 +137,7 @@ class FetchStrategyComposite(object):
class URLFetchStrategy(FetchStrategy):
"""FetchStrategy that pulls source code from a URL for an archive,
checks the archive against a checksum,and decompresses the archive.
"""
@ -235,12 +239,13 @@ def fetch(self):
# redirects properly.
content_types = re.findall(r'Content-Type:[^\r\n]+', headers)
if content_types and 'text/html' in content_types[-1]:
tty.warn(
"The contents of " + self.archive_file + " look like HTML.",
"The checksum will likely be bad. If it is, you can use",
"'spack clean <package>' to remove the bad archive, then fix",
"your internet gateway issue and install again.")
tty.warn("The contents of ",
(self.archive_file if self.archive_file is not None
else "the archive"),
" look like HTML.",
"The checksum will likely be bad. If it is, you can use",
"'spack clean <package>' to remove the bad archive, then",
"fix your internet gateway issue and install again.")
if save_file:
os.rename(partial_file, save_file)
@ -353,6 +358,7 @@ def __str__(self):
class VCSFetchStrategy(FetchStrategy):
def __init__(self, name, *rev_types, **kwargs):
super(VCSFetchStrategy, self).__init__()
self.name = name
@ -407,6 +413,7 @@ def __repr__(self):
class GoFetchStrategy(VCSFetchStrategy):
"""
Fetch strategy that employs the `go get` infrastructure
Use like this in a package:
@ -466,6 +473,7 @@ def __str__(self):
class GitFetchStrategy(VCSFetchStrategy):
"""
Fetch strategy that gets source code from a git repository.
Use like this in a package:
@ -586,6 +594,7 @@ def __str__(self):
class SvnFetchStrategy(VCSFetchStrategy):
"""Fetch strategy that gets source code from a subversion repository.
Use like this in a package:
@ -662,6 +671,7 @@ def __str__(self):
class HgFetchStrategy(VCSFetchStrategy):
"""
Fetch strategy that gets source code from a Mercurial repository.
Use like this in a package:
@ -806,11 +816,13 @@ def for_package_version(pkg, version):
class FetchError(spack.error.SpackError):
def __init__(self, msg, long_msg=None):
super(FetchError, self).__init__(msg, long_msg)
class FailedDownloadError(FetchError):
"""Raised wen a download fails."""
def __init__(self, url, msg=""):
@ -820,16 +832,19 @@ def __init__(self, url, msg=""):
class NoArchiveFileError(FetchError):
def __init__(self, msg, long_msg):
super(NoArchiveFileError, self).__init__(msg, long_msg)
class NoDigestError(FetchError):
def __init__(self, msg, long_msg=None):
super(NoDigestError, self).__init__(msg, long_msg)
class InvalidArgsError(FetchError):
def __init__(self, pkg, version):
msg = ("Could not construct a fetch strategy for package %s at "
"version %s")
@ -838,6 +853,7 @@ def __init__(self, pkg, version):
class ChecksumError(FetchError):
"""Raised when archive fails to checksum."""
def __init__(self, message, long_msg=None):
@ -845,6 +861,7 @@ def __init__(self, message, long_msg=None):
class NoStageError(FetchError):
"""Raised when fetch operations are called before set_stage()."""
def __init__(self, method):

558
lib/spack/spack/spec.py
View file

File diff suppressed because it is too large Load diff

2
lib/spack/spack/test/__init__.py
View file

@ -39,7 +39,7 @@
'svn_fetch', 'hg_fetch', 'mirror', 'modules', 'url_extrapolate',
'cc', 'link_tree', 'spec_yaml', 'optional_deps',
'make_executable', 'configure_guess', 'lock', 'database',
'namespace_trie', 'yaml', 'sbang', 'environment',
'namespace_trie', 'yaml', 'sbang', 'environment', 'cmd.find',
'cmd.uninstall', 'cmd.test_install']

60
lib/spack/spack/test/cmd/find.py Normal file
View file

@ -0,0 +1,60 @@
##############################################################################
# Copyright (c) 2013-2016, Lawrence Livermore National Security, LLC.
# Produced at the Lawrence Livermore National Laboratory.
#
# This file is part of Spack.
# Created by Todd Gamblin, tgamblin@llnl.gov, All rights reserved.
# LLNL-CODE-647188
#
# For details, see https://github.com/llnl/spack
# Please also see the LICENSE file for our notice and the LGPL.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU Lesser General Public License (as
# published by the Free Software Foundation) version 2.1, February 1999.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the IMPLIED WARRANTY OF
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the terms and
# conditions of the GNU Lesser General Public License for more details.
#
# You should have received a copy of the GNU Lesser General Public
# License along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
##############################################################################
import spack.cmd.find
import unittest
class Bunch(object):
def __init__(self, **kwargs):
self.__dict__.update(kwargs)
class FindTest(unittest.TestCase):
def test_query_arguments(self):
query_arguments = spack.cmd.find.query_arguments
# Default arguments
args = Bunch(only_missing=False, missing=False,
unknown=False, explicit=False, implicit=False)
q_args = query_arguments(args)
self.assertTrue('installed' in q_args)
self.assertTrue('known' in q_args)
self.assertTrue('explicit' in q_args)
self.assertEqual(q_args['installed'], True)
self.assertEqual(q_args['known'], any)
self.assertEqual(q_args['explicit'], any)
# Check that explicit works correctly
args.explicit = True
q_args = query_arguments(args)
self.assertEqual(q_args['explicit'], True)
args.explicit = False
args.implicit = True
q_args = query_arguments(args)
self.assertEqual(q_args['explicit'], False)
args.explicit = True
self.assertRaises(SystemExit, query_arguments, args)

143
var/spack/repos/builtin/packages/dealii/package.py
View file

@ -25,8 +25,10 @@
from spack import *
import sys
class Dealii(Package):
"""C++ software library providing well-documented tools to build finite element codes for a broad variety of PDEs."""
"""C++ software library providing well-documented tools to build finite
element codes for a broad variety of PDEs."""
homepage = "https://www.dealii.org"
url = "https://github.com/dealii/dealii/releases/download/v8.4.0/dealii-8.4.0.tar.gz"
@ -36,7 +38,7 @@ class Dealii(Package):
variant('mpi', default=True, description='Compile with MPI')
variant('arpack', default=True, description='Compile with Arpack and PArpack (only with MPI)')
variant('doc', default=False, description='Compile with documentation')
variant('gsl' , default=True, description='Compile with GSL')
variant('gsl', default=True, description='Compile with GSL')
variant('hdf5', default=True, description='Compile with HDF5 (only with MPI)')
variant('metis', default=True, description='Compile with Metis')
variant('netcdf', default=True, description='Compile with Netcdf (only with MPI)')
@ -47,38 +49,40 @@ class Dealii(Package):
variant('trilinos', default=True, description='Compile with Trilinos (only with MPI)')
# required dependencies, light version
depends_on ("blas")
# Boost 1.58 is blacklisted, see https://github.com/dealii/dealii/issues/1591
# require at least 1.59
depends_on ("boost@1.59.0:", when='~mpi')
depends_on ("boost@1.59.0:+mpi", when='+mpi')
depends_on ("bzip2")
depends_on ("cmake")
depends_on ("lapack")
depends_on ("muparser")
depends_on ("suite-sparse")
depends_on ("tbb")
depends_on ("zlib")
depends_on("blas")
# Boost 1.58 is blacklisted, see
# https://github.com/dealii/dealii/issues/1591
# Require at least 1.59
depends_on("boost@1.59.0:+thread+system+serialization+iostreams", when='~mpi') # NOQA: ignore=E501
depends_on("boost@1.59.0:+mpi+thread+system+serialization+iostreams", when='+mpi') # NOQA: ignore=E501
depends_on("bzip2")
depends_on("cmake")
depends_on("lapack")
depends_on("muparser")
depends_on("suite-sparse")
depends_on("tbb")
depends_on("zlib")
# optional dependencies
depends_on ("mpi", when="+mpi")
depends_on ("arpack-ng+mpi", when='+arpack+mpi')
depends_on ("doxygen", when='+doc')
depends_on ("gsl", when='@8.5.0:+gsl')
depends_on ("gsl", when='@dev+gsl')
depends_on ("hdf5+mpi~cxx", when='+hdf5+mpi') #FIXME NetCDF declares dependency with ~cxx, why?
depends_on ("metis@5:", when='+metis')
depends_on ("netcdf+mpi", when="+netcdf+mpi")
depends_on ("netcdf-cxx", when='+netcdf+mpi')
depends_on ("oce", when='+oce')
depends_on ("p4est", when='+p4est+mpi')
depends_on ("petsc+mpi", when='+petsc+mpi')
depends_on ("slepc", when='+slepc+petsc+mpi')
depends_on ("trilinos", when='+trilinos+mpi')
depends_on("mpi", when="+mpi")
depends_on("arpack-ng+mpi", when='+arpack+mpi')
depends_on("doxygen+graphviz", when='+doc')
depends_on("graphviz", when='+doc')
depends_on("gsl", when='@8.5.0:+gsl')
depends_on("gsl", when='@dev+gsl')
depends_on("hdf5+mpi", when='+hdf5+mpi')
depends_on("metis@5:", when='+metis')
depends_on("netcdf+mpi", when="+netcdf+mpi")
depends_on("netcdf-cxx", when='+netcdf+mpi')
depends_on("oce", when='+oce')
depends_on("p4est", when='+p4est+mpi')
depends_on("petsc+mpi", when='+petsc+mpi')
depends_on("slepc", when='+slepc+petsc+mpi')
depends_on("trilinos", when='+trilinos+mpi')
# developer dependnecies
depends_on ("numdiff", when='@dev')
depends_on ("astyle@2.04", when='@dev')
depends_on("numdiff", when='@dev')
depends_on("astyle@2.04", when='@dev')
def install(self, spec, prefix):
options = []
@ -96,17 +100,17 @@ def install(self, spec, prefix):
'-DDEAL_II_WITH_THREADS:BOOL=ON',
'-DBOOST_DIR=%s' % spec['boost'].prefix,
'-DBZIP2_DIR=%s' % spec['bzip2'].prefix,
# CMake's FindBlas/Lapack may pickup system's blas/lapack instead of Spack's.
# Be more specific to avoid this.
# Note that both lapack and blas are provided in -DLAPACK_XYZ variables
# CMake's FindBlas/Lapack may pickup system's blas/lapack instead
# of Spack's. Be more specific to avoid this.
# Note that both lapack and blas are provided in -DLAPACK_XYZ.
'-DLAPACK_FOUND=true',
'-DLAPACK_INCLUDE_DIRS=%s;%s' %
(spec['lapack'].prefix.include,
spec['blas'].prefix.include),
'-DLAPACK_LIBRARIES=%s;%s' %
(join_path(spec['lapack'].prefix.lib,'liblapack.%s' % dsuf), # FIXME don't hardcode names
join_path(spec['blas'].prefix.lib,'libblas.%s' % dsuf)), # FIXME don't hardcode names
'-DMUPARSER_DIR=%s ' % spec['muparser'].prefix,
(spec['lapack'].lapack_shared_lib,
spec['blas'].blas_shared_lib),
'-DMUPARSER_DIR=%s' % spec['muparser'].prefix,
'-DUMFPACK_DIR=%s' % spec['suite-sparse'].prefix,
'-DTBB_DIR=%s' % spec['tbb'].prefix,
'-DZLIB_DIR=%s' % spec['zlib'].prefix
@ -116,33 +120,34 @@ def install(self, spec, prefix):
if '+mpi' in spec:
options.extend([
'-DDEAL_II_WITH_MPI:BOOL=ON',
'-DCMAKE_C_COMPILER=%s' % join_path(self.spec['mpi'].prefix.bin, 'mpicc'), # FIXME: avoid hardcoding mpi wrappers names
'-DCMAKE_CXX_COMPILER=%s' % join_path(self.spec['mpi'].prefix.bin, 'mpic++'),
'-DCMAKE_Fortran_COMPILER=%s' % join_path(self.spec['mpi'].prefix.bin, 'mpif90'),
'-DCMAKE_C_COMPILER=%s' % spec['mpi'].mpicc,
'-DCMAKE_CXX_COMPILER=%s' % spec['mpi'].mpicxx,
'-DCMAKE_Fortran_COMPILER=%s' % spec['mpi'].mpifc,
])
else:
options.extend([
'-DDEAL_II_WITH_MPI:BOOL=OFF',
])
# Optional dependencies for which librariy names are the same as CMake variables
for library in ('gsl','hdf5','p4est','petsc','slepc','trilinos','metis'):
# Optional dependencies for which librariy names are the same as CMake
# variables:
for library in ('gsl', 'hdf5', 'p4est', 'petsc', 'slepc', 'trilinos', 'metis'): # NOQA: ignore=E501
if library in spec:
options.extend([
'-D{library}_DIR={value}'.format(library=library.upper(), value=spec[library].prefix),
'-DDEAL_II_WITH_{library}:BOOL=ON'.format(library=library.upper())
'-D%s_DIR=%s' % (library.upper(), spec[library].prefix),
'-DDEAL_II_WITH_%s:BOOL=ON' % library.upper()
])
else:
options.extend([
'-DDEAL_II_WITH_{library}:BOOL=OFF'.format(library=library.upper())
'-DDEAL_II_WITH_%s:BOOL=OFF' % library.upper()
])
# doxygen
options.extend([
'-DDEAL_II_COMPONENT_DOCUMENTATION=%s' % ('ON' if '+doc' in spec else 'OFF'),
'-DDEAL_II_COMPONENT_DOCUMENTATION=%s' %
('ON' if '+doc' in spec else 'OFF'),
])
# arpack
if '+arpack' in spec:
options.extend([
@ -160,11 +165,13 @@ def install(self, spec, prefix):
options.extend([
'-DNETCDF_FOUND=true',
'-DNETCDF_LIBRARIES=%s;%s' %
(join_path(spec['netcdf-cxx'].prefix.lib,'libnetcdf_c++.%s' % dsuf),
join_path(spec['netcdf'].prefix.lib,'libnetcdf.%s' % dsuf)),
(join_path(spec['netcdf-cxx'].prefix.lib,
'libnetcdf_c++.%s' % dsuf),
join_path(spec['netcdf'].prefix.lib,
'libnetcdf.%s' % dsuf)),
'-DNETCDF_INCLUDE_DIRS=%s;%s' %
(spec['netcdf-cxx'].prefix.include,
spec['netcdf'].prefix.include),
spec['netcdf'].prefix.include),
])
else:
options.extend([
@ -200,7 +207,7 @@ def install(self, spec, prefix):
with working_dir('examples/step-3'):
cmake('.')
make('release')
make('run',parallel=False)
make('run', parallel=False)
# An example which uses Metis + PETSc
# FIXME: switch step-18 to MPI
@ -213,7 +220,7 @@ def install(self, spec, prefix):
if '^petsc' in spec and '^metis' in spec:
cmake('.')
make('release')
make('run',parallel=False)
make('run', parallel=False)
# take step-40 which can use both PETSc and Trilinos
# FIXME: switch step-40 to MPI run
@ -222,43 +229,49 @@ def install(self, spec, prefix):
print('========== Step-40 PETSc ============')
print('=====================================')
# list the number of cycles to speed up
filter_file(r'(const unsigned int n_cycles = 8;)', ('const unsigned int n_cycles = 2;'), 'step-40.cc')
filter_file(r'(const unsigned int n_cycles = 8;)',
('const unsigned int n_cycles = 2;'), 'step-40.cc')
cmake('.')
if '^petsc' in spec:
make('release')
make('run',parallel=False)
make('run', parallel=False)
print('=====================================')
print('========= Step-40 Trilinos ==========')
print('=====================================')
# change Linear Algebra to Trilinos
filter_file(r'(\/\/ #define FORCE_USE_OF_TRILINOS.*)', ('#define FORCE_USE_OF_TRILINOS'), 'step-40.cc')
filter_file(r'(\/\/ #define FORCE_USE_OF_TRILINOS.*)',
('#define FORCE_USE_OF_TRILINOS'), 'step-40.cc')
if '^trilinos+hypre' in spec:
make('release')
make('run',parallel=False)
make('run', parallel=False)
print('=====================================')
print('=== Step-40 Trilinos SuperluDist ====')
print('=====================================')
# change to direct solvers
filter_file(r'(LA::SolverCG solver\(solver_control\);)', ('TrilinosWrappers::SolverDirect::AdditionalData data(false,"Amesos_Superludist"); TrilinosWrappers::SolverDirect solver(solver_control,data);'), 'step-40.cc')
filter_file(r'(LA::MPI::PreconditionAMG preconditioner;)', (''), 'step-40.cc')
filter_file(r'(LA::MPI::PreconditionAMG::AdditionalData data;)', (''), 'step-40.cc')
filter_file(r'(preconditioner.initialize\(system_matrix, data\);)', (''), 'step-40.cc')
filter_file(r'(solver\.solve \(system_matrix, completely_distributed_solution, system_rhs,)', ('solver.solve (system_matrix, completely_distributed_solution, system_rhs);'), 'step-40.cc')
filter_file(r'(LA::SolverCG solver\(solver_control\);)', ('TrilinosWrappers::SolverDirect::AdditionalData data(false,"Amesos_Superludist"); TrilinosWrappers::SolverDirect solver(solver_control,data);'), 'step-40.cc') # NOQA: ignore=E501
filter_file(r'(LA::MPI::PreconditionAMG preconditioner;)',
(''), 'step-40.cc')
filter_file(r'(LA::MPI::PreconditionAMG::AdditionalData data;)',
(''), 'step-40.cc')
filter_file(r'(preconditioner.initialize\(system_matrix, data\);)',
(''), 'step-40.cc')
filter_file(r'(solver\.solve \(system_matrix, completely_distributed_solution, system_rhs,)', ('solver.solve (system_matrix, completely_distributed_solution, system_rhs);'), 'step-40.cc') # NOQA: ignore=E501
filter_file(r'(preconditioner\);)', (''), 'step-40.cc')
if '^trilinos+superlu-dist' in spec:
make('release')
make('run',paralle=False)
make('run', paralle=False)
print('=====================================')
print('====== Step-40 Trilinos MUMPS =======')
print('=====================================')
# switch to Mumps
filter_file(r'(Amesos_Superludist)', ('Amesos_Mumps'), 'step-40.cc')
filter_file(r'(Amesos_Superludist)',
('Amesos_Mumps'), 'step-40.cc')
if '^trilinos+mumps' in spec:
make('release')
make('run',parallel=False)
make('run', parallel=False)
print('=====================================')
print('============ Step-36 ================')
@ -267,7 +280,7 @@ def install(self, spec, prefix):
if 'slepc' in spec:
cmake('.')
make('release')
make('run',parallel=False)
make('run', parallel=False)
print('=====================================')
print('============ Step-54 ================')
@ -276,7 +289,7 @@ def install(self, spec, prefix):
if 'oce' in spec:
cmake('.')
make('release')
make('run',parallel=False)
make('run', parallel=False)
def setup_environment(self, spack_env, env):
env.set('DEAL_II_DIR', self.prefix)

18
var/spack/repos/builtin/packages/launchmon/package.py
View file

@ -24,29 +24,19 @@
##############################################################################
from spack import *
class Launchmon(Package):
"""Software infrastructure that enables HPC run-time tools to
co-locate tool daemons with a parallel job."""
homepage = "http://sourceforge.net/projects/launchmon"
url = "http://downloads.sourceforge.net/project/launchmon/launchmon/1.0.1%20release/launchmon-1.0.1.tar.gz"
homepage = "https://github.com/LLNL/LaunchMON"
url = "https://github.com/LLNL/LaunchMON/releases/download/v1.0.2/launchmon-v1.0.2.tar.gz" # NOQA: ignore=E501
version('1.0.1', '2f12465803409fd07f91174a4389eb2b')
version('1.0.1-2', git='https://github.com/llnl/launchmon.git', commit='ff7e22424b8f375318951eb1c9282fcbbfa8aadf')
version('1.0.2', '8d6ba77a0ec2eff2fde2c5cc8fa7ff7a')
depends_on('autoconf')
depends_on('automake')
depends_on('libtool')
def patch(self):
# This patch makes libgcrypt compile correctly with newer gcc versions.
mf = FileFilter('tools/libgcrypt/tests/Makefile.in')
mf.filter(r'(basic_LDADD\s*=\s*.*)', r'\1 -lgpg-error')
mf.filter(r'(tsexp_LDADD\s*=\s*.*)', r'\1 -lgpg-error')
mf.filter(r'(keygen_LDADD\s*=\s*.*)', r'\1 -lgpg-error')
mf.filter(r'(benchmark_LDADD\s*=\s*.*)', r'\1 -lgpg-error')
def install(self, spec, prefix):
configure(
"--prefix=" + prefix,

34
var/spack/repos/builtin/packages/llvm/package.py
View file

@ -23,7 +23,7 @@
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
##############################################################################
from spack import *
import os, shutil
import os, glob
class Llvm(Package):
@ -46,7 +46,9 @@ class Llvm(Package):
variant('libcxx', default=True, description="Build the LLVM C++ standard library")
variant('compiler-rt', default=True, description="Build the LLVM compiler runtime, including sanitizers")
variant('gold', default=True, description="Add support for LTO with the gold linker plugin")
variant('shared_libs', default=False, description="Build all components as shared libraries, faster, less memory to build, less stable")
variant('link_dylib', default=False, description="Build and link the libLLVM shared library rather than static")
variant('all_targets', default=True, description="Build all supported targets, default targets <current arch>,NVPTX,AMDGPU,CppBackend")
# Build dependency
depends_on('cmake @2.8.12.2:')
@ -257,6 +259,28 @@ def install(self, spec, prefix):
if '+compiler-rt' not in spec:
cmake_args.append('-DLLVM_EXTERNAL_COMPILER_RT_BUILD:Bool=OFF')
if '+shared_libs' in spec:
cmake_args.append('-DBUILD_SHARED_LIBS:Bool=ON')
if '+link_dylib' in spec:
cmake_args.append('-DLLVM_LINK_LLVM_DYLIB:Bool=ON')
if '+all_targets' not in spec: # all is default on cmake
targets = ['CppBackend', 'NVPTX', 'AMDGPU']
if 'x86' in spec.architecture.lower():
targets.append('X86')
elif 'arm' in spec.architecture.lower():
targets.append('ARM')
elif 'aarch64' in spec.architecture.lower():
targets.append('AArch64')
elif 'sparc' in spec.architecture.lower():
targets.append('sparc')
elif ('ppc' in spec.architecture.lower() or
'power' in spec.architecture.lower()):
targets.append('PowerPC')
cmake_args.append('-DLLVM_TARGETS_TO_BUILD:Bool=' + ';'.join(targets))
if '+clang' not in spec:
if '+clang_extra' in spec:
raise SpackException('The clang_extra variant requires the clang variant to be selected')
@ -267,7 +291,5 @@ def install(self, spec, prefix):
cmake(*cmake_args)
make()
make("install")
query_path = os.path.join('bin', 'clang-query')
# Manually install clang-query, because llvm doesn't...
if os.path.exists(query_path):
shutil.copy(query_path, os.path.join(prefix, 'bin'))
cp = which('cp')
cp('-a', 'bin/', prefix)

190
var/spack/repos/builtin/packages/metis/package.py
View file

@ -24,55 +24,61 @@
##############################################################################
from spack import *
import glob, sys, os
import glob
import sys
import os
class Metis(Package):
"""
METIS is a set of serial programs for partitioning graphs, partitioning finite element meshes, and producing fill
reducing orderings for sparse matrices. The algorithms implemented in METIS are based on the multilevel
recursive-bisection, multilevel k-way, and multi-constraint partitioning schemes.
"""
"""METIS is a set of serial programs for partitioning graphs, partitioning
finite element meshes, and producing fill reducing orderings for sparse
matrices. The algorithms implemented in METIS are based on the
multilevel recursive-bisection, multilevel k-way, and multi-constraint
partitioning schemes."""
homepage = 'http://glaros.dtc.umn.edu/gkhome/metis/metis/overview'
url = "http://glaros.dtc.umn.edu/gkhome/fetch/sw/metis/metis-5.1.0.tar.gz"
homepage = "http://glaros.dtc.umn.edu/gkhome/metis/metis/overview"
base_url = "http://glaros.dtc.umn.edu/gkhome/fetch/sw/metis"
version('5.1.0', '5465e67079419a69e0116de24fce58fe',
url='http://glaros.dtc.umn.edu/gkhome/fetch/sw/metis/metis-5.1.0.tar.gz')
version('4.0.3', '5efa35de80703c1b2c4d0de080fafbcf4e0d363a21149a1ad2f96e0144841a55',
url='http://glaros.dtc.umn.edu/gkhome/fetch/sw/metis/OLD/metis-4.0.3.tar.gz')
version('5.1.0', '5465e67079419a69e0116de24fce58fe')
version('5.0.2', 'acb521a4e8c2e6dd559a7f9abd0468c5')
version('4.0.3', 'd3848b454532ef18dc83e4fb160d1e10')
variant('shared', default=True, description='Enables the build of shared libraries')
variant('debug', default=False, description='Builds the library in debug mode')
variant('gdb', default=False, description='Enables gdb support')
variant('idx64', default=False, description='Use int64_t as default index type')
variant('double', default=False, description='Use double precision floating point types')
variant('real64', default=False, description='Use double precision floating point types')
depends_on('cmake @2.8:', when='@5:') # build-time dependency
depends_on('gdb', when='+gdb')
depends_on('cmake@2.8:', when='@5:') # build-time dependency
patch('install_gklib_defs_rename.patch', when='@5:')
def url_for_version(self, version):
verdir = 'OLD/' if version < Version('4.0.3') else ''
return '%s/%smetis-%s.tar.gz' % (Metis.base_url, verdir, version)
@when('@4:4.0.3')
@when('@:4')
def install(self, spec, prefix):
if '+gdb' in spec:
raise InstallError('gdb support not implemented in METIS 4!')
if '+idx64' in spec:
raise InstallError('idx64 option not implemented in METIS 4!')
if '+double' in spec:
raise InstallError('double option not implemented for METIS 4!')
# Process library spec and options
unsupp_vars = [v for v in ('+gdb', '+idx64', '+real64') if v in spec]
if unsupp_vars:
msg = 'Given variants %s are unsupported by METIS 4!' % unsupp_vars
raise InstallError(msg)
options = ['COPTIONS=-fPIC']
if '+debug' in spec:
options.append('OPTFLAGS=-g -O0')
make(*options)
# Compile and install library files
ccompile = Executable(self.compiler.cc)
mkdir(prefix.bin)
for x in ('pmetis', 'kmetis', 'oemetis', 'onmetis', 'partnmesh',
'partdmesh', 'mesh2nodal', 'mesh2dual', 'graphchk'):
install(x, prefix.bin)
binfiles = ('pmetis', 'kmetis', 'oemetis', 'onmetis', 'partnmesh',
'partdmesh', 'mesh2nodal', 'mesh2dual', 'graphchk')
for binfile in binfiles:
install(binfile, prefix.bin)
mkdir(prefix.lib)
install('libmetis.a', prefix.lib)
@ -82,106 +88,120 @@ def install(self, spec, prefix):
install(h, prefix.include)
mkdir(prefix.share)
for f in (join_path(*p)
for p in (('Programs', 'io.c'),
('Test','mtest.c'),
('Graphs','4elt.graph'),
('Graphs', 'metis.mesh'),
('Graphs', 'test.mgraph'))):
install(f, prefix.share)
sharefiles = (('Graphs', '4elt.graph'), ('Graphs', 'metis.mesh'),
('Graphs', 'test.mgraph'))
for sharefile in tuple(join_path(*sf) for sf in sharefiles):
install(sharefile, prefix.share)
if '+shared' in spec:
shared_flags = ['-fPIC', '-shared']
if sys.platform == 'darwin':
lib_dsuffix = 'dylib'
load_flag = '-Wl,-all_load'
no_load_flag = ''
shared_suffix = 'dylib'
shared_flags.extend(['-Wl,-all_load', 'libmetis.a'])
else:
lib_dsuffix = 'so'
load_flag = '-Wl,-whole-archive'
no_load_flag = '-Wl,-no-whole-archive'
shared_suffix = 'so'
shared_flags.extend(['-Wl,-whole-archive', 'libmetis.a',
'-Wl,-no-whole-archive'])
os.system(spack_cc + ' -fPIC -shared ' + load_flag +
' libmetis.a ' + no_load_flag + ' -o libmetis.' +
lib_dsuffix)
install('libmetis.' + lib_dsuffix, prefix.lib)
shared_out = '%s/libmetis.%s' % (prefix.lib, shared_suffix)
shared_flags.extend(['-o', shared_out])
ccompile(*shared_flags)
# Set up and run tests on installation
symlink(join_path(prefix.share, 'io.c'), 'io.c')
symlink(join_path(prefix.share, 'mtest.c'), 'mtest.c')
os.system(spack_cc + ' -I%s' % prefix.include + ' -c io.c')
os.system(spack_cc + ' -I%s' % prefix.include +
' -L%s' % prefix.lib + ' -lmetis mtest.c io.o -o mtest')
_4eltgraph = join_path(prefix.share, '4elt.graph')
test_mgraph = join_path(prefix.share, 'test.mgraph')
metis_mesh = join_path(prefix.share, 'metis.mesh')
kmetis = join_path(prefix.bin, 'kmetis')
os.system('./mtest ' + _4eltgraph)
os.system(kmetis + ' ' + _4eltgraph + ' 40')
os.system(join_path(prefix.bin, 'onmetis') + ' ' + _4eltgraph)
os.system(join_path(prefix.bin, 'pmetis') + ' ' + test_mgraph + ' 2')
os.system(kmetis + ' ' + test_mgraph + ' 2')
os.system(kmetis + ' ' + test_mgraph + ' 5')
os.system(join_path(prefix.bin, 'partnmesh') + metis_mesh + ' 10')
os.system(join_path(prefix.bin, 'partdmesh') + metis_mesh + ' 10')
os.system(join_path(prefix.bin, 'mesh2dual') + metis_mesh)
ccompile('-I%s' % prefix.include, '-L%s' % prefix.lib,
'-Wl,-rpath=%s' % (prefix.lib if '+shared' in spec else ''),
join_path('Programs', 'io.o'), join_path('Test', 'mtest.c'),
'-o', '%s/mtest' % prefix.bin, '-lmetis', '-lm')
test_bin = lambda testname: join_path(prefix.bin, testname)
test_graph = lambda graphname: join_path(prefix.share, graphname)
graph = test_graph('4elt.graph')
os.system('%s %s' % (test_bin('mtest'), graph))
os.system('%s %s 40' % (test_bin('kmetis'), graph))
os.system('%s %s' % (test_bin('onmetis'), graph))
graph = test_graph('test.mgraph')
os.system('%s %s 2' % (test_bin('pmetis'), graph))
os.system('%s %s 2' % (test_bin('kmetis'), graph))
os.system('%s %s 5' % (test_bin('kmetis'), graph))
graph = test_graph('metis.mesh')
os.system('%s %s 10' % (test_bin('partnmesh'), graph))
os.system('%s %s 10' % (test_bin('partdmesh'), graph))
os.system('%s %s' % (test_bin('mesh2dual'), graph))
# FIXME: The following code should replace the testing code in the
# block above since it causes installs to fail when one or more of the
# Metis tests fail, but it currently doesn't work because the 'mtest',
# 'onmetis', and 'partnmesh' tests return error codes that trigger
# false positives for failure.
"""
Executable(test_bin('mtest'))(test_graph('4elt.graph'))
Executable(test_bin('kmetis'))(test_graph('4elt.graph'), '40')
Executable(test_bin('onmetis'))(test_graph('4elt.graph'))
Executable(test_bin('pmetis'))(test_graph('test.mgraph'), '2')
Executable(test_bin('kmetis'))(test_graph('test.mgraph'), '2')
Executable(test_bin('kmetis'))(test_graph('test.mgraph'), '5')
Executable(test_bin('partnmesh'))(test_graph('metis.mesh'), '10')
Executable(test_bin('partdmesh'))(test_graph('metis.mesh'), '10')
Executable(test_bin('mesh2dual'))(test_graph('metis.mesh'))
"""
@when('@5:')
def install(self, spec, prefix):
options = []
options.extend(std_cmake_args)
build_directory = join_path(self.stage.path, 'spack-build')
source_directory = self.stage.source_path
options.append('-DGKLIB_PATH:PATH={metis_source}/GKlib'.format(metis_source=source_directory))
options.append('-DGKLIB_PATH:PATH=%s/GKlib' % source_directory)
options.append('-DCMAKE_INSTALL_NAME_DIR:PATH=%s/lib' % prefix)
if '+shared' in spec:
options.append('-DSHARED:BOOL=ON')
if '+debug' in spec:
options.extend(['-DDEBUG:BOOL=ON',
'-DCMAKE_BUILD_TYPE:STRING=Debug'])
if '+gdb' in spec:
options.append('-DGDB:BOOL=ON')
metis_header = join_path(source_directory, 'include', 'metis.h')
if '+idx64' in spec:
filter_file('IDXTYPEWIDTH 32', 'IDXTYPEWIDTH 64', metis_header)
if '+double' in spec:
if '+real64' in spec:
filter_file('REALTYPEWIDTH 32', 'REALTYPEWIDTH 64', metis_header)
# Make clang 7.3 happy.
# Prevents "ld: section __DATA/__thread_bss extends beyond end of file"
# See upstream LLVM issue https://llvm.org/bugs/show_bug.cgi?id=27059
# Adopted from https://github.com/Homebrew/homebrew-science/blob/master/metis.rb
# and https://github.com/Homebrew/homebrew-science/blob/master/metis.rb
if spec.satisfies('%clang@7.3.0'):
filter_file('#define MAX_JBUFS 128', '#define MAX_JBUFS 24', join_path(source_directory, 'GKlib', 'error.c'))
filter_file('#define MAX_JBUFS 128', '#define MAX_JBUFS 24',
join_path(source_directory, 'GKlib', 'error.c'))
with working_dir(build_directory, create=True):
cmake(source_directory, *options)
make()
make("install")
# now run some tests:
for f in ["4elt", "copter2", "mdual"]:
graph = join_path(source_directory,'graphs','%s.graph' % f)
Executable(join_path(prefix.bin,'graphchk'))(graph)
Executable(join_path(prefix.bin,'gpmetis'))(graph,'2')
Executable(join_path(prefix.bin,'ndmetis'))(graph)
make('install')
graph = join_path(source_directory,'graphs','test.mgraph')
Executable(join_path(prefix.bin,'gpmetis'))(graph,'2')
graph = join_path(source_directory,'graphs','metis.mesh')
Executable(join_path(prefix.bin,'mpmetis'))(graph,'2')
# now run some tests:
for f in ['4elt', 'copter2', 'mdual']:
graph = join_path(source_directory, 'graphs', '%s.graph' % f)
Executable(join_path(prefix.bin, 'graphchk'))(graph)
Executable(join_path(prefix.bin, 'gpmetis'))(graph, '2')
Executable(join_path(prefix.bin, 'ndmetis'))(graph)
graph = join_path(source_directory, 'graphs', 'test.mgraph')
Executable(join_path(prefix.bin, 'gpmetis'))(graph, '2')
graph = join_path(source_directory, 'graphs', 'metis.mesh')
Executable(join_path(prefix.bin, 'mpmetis'))(graph, '2')
# install GKlib headers, which will be needed for ParMETIS
GKlib_dist = join_path(prefix.include,'GKlib')
GKlib_dist = join_path(prefix.include, 'GKlib')
mkdirp(GKlib_dist)
fs = glob.glob(join_path(source_directory,'GKlib',"*.h"))
for f in fs:
install(f, GKlib_dist)
hfiles = glob.glob(join_path(source_directory, 'GKlib', '*.h'))
for hfile in hfiles:
install(hfile, GKlib_dist)

57
var/spack/repos/builtin/packages/octave/package.py
View file

@ -23,6 +23,8 @@
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
##############################################################################
from spack import *
import sys
class Octave(Package):
"""GNU Octave is a high-level language, primarily intended for numerical
@ -34,7 +36,8 @@ class Octave(Package):
homepage = "https://www.gnu.org/software/octave/"
url = "ftp://ftp.gnu.org/gnu/octave/octave-4.0.0.tar.gz"
version('4.0.0' , 'a69f8320a4f20a8480c1b278b1adb799')
version('4.0.2', 'c2a5cacc6e4c52f924739cdf22c2c687')
version('4.0.0', 'a69f8320a4f20a8480c1b278b1adb799')
# Variants
variant('readline', default=True)
@ -62,33 +65,35 @@ class Octave(Package):
# Required dependencies
depends_on('blas')
depends_on('lapack')
# Octave does not configure with sed from darwin:
depends_on('sed', sys.platform == 'darwin')
depends_on('pcre')
depends_on('pkg-config')
# Strongly recommended dependencies
depends_on('readline', when='+readline')
depends_on('readline', when='+readline')
# Optional dependencies
depends_on('arpack', when='+arpack')
depends_on('curl', when='+curl')
depends_on('fftw', when='+fftw')
depends_on('fltk', when='+fltk')
depends_on('fontconfig', when='+fontconfig')
depends_on('freetype', when='+freetype')
depends_on('glpk', when='+glpk')
depends_on('gl2ps', when='+gl2ps')
depends_on('gnuplot', when='+gnuplot')
depends_on('ImageMagick', when='+magick')
depends_on('hdf5', when='+hdf5')
depends_on('jdk', when='+jdk')
depends_on('llvm', when='+llvm')
#depends_on('opengl', when='+opengl') # TODO: add package
depends_on('qhull', when='+qhull')
depends_on('qrupdate', when='+qrupdate')
#depends_on('qscintilla', when='+qscintilla) # TODO: add package
depends_on('qt', when='+qt')
depends_on('suite-sparse',when='+suitesparse')
depends_on('zlib', when='+zlib')
depends_on('arpack', when='+arpack')
depends_on('curl', when='+curl')
depends_on('fftw', when='+fftw')
depends_on('fltk', when='+fltk')
depends_on('fontconfig', when='+fontconfig')
depends_on('freetype', when='+freetype')
depends_on('glpk', when='+glpk')
depends_on('gl2ps', when='+gl2ps')
depends_on('gnuplot', when='+gnuplot')
depends_on('ImageMagick', when='+magick')
depends_on('hdf5', when='+hdf5')
depends_on('jdk', when='+jdk')
depends_on('llvm', when='+llvm')
# depends_on('opengl', when='+opengl') # TODO: add package
depends_on('qhull', when='+qhull')
depends_on('qrupdate', when='+qrupdate')
# depends_on('qscintilla', when='+qscintilla) # TODO: add package
depends_on('qt', when='+qt')
depends_on('suite-sparse', when='+suitesparse')
depends_on('zlib', when='+zlib')
def install(self, spec, prefix):
config_args = [
@ -154,7 +159,8 @@ def install(self, spec, prefix):
config_args.append("--without-glpk")
if '+magick' in spec:
config_args.append("--with-magick=%s" % spec['ImageMagick'].prefix.lib)
config_args.append("--with-magick=%s"
% spec['ImageMagick'].prefix.lib)
if '+hdf5' in spec:
config_args.extend([
@ -187,7 +193,8 @@ def install(self, spec, prefix):
if '+qrupdate' in spec:
config_args.extend([
"--with-qrupdate-includedir=%s" % spec['qrupdate'].prefix.include,
"--with-qrupdate-includedir=%s"
% spec['qrupdate'].prefix.include,
"--with-qrupdate-libdir=%s" % spec['qrupdate'].prefix.lib
])
else:

31
var/spack/repos/builtin/packages/openmpi/package.py
View file

@ -27,6 +27,26 @@
from spack import *
def _verbs_dir():
"""
Try to find the directory where the OpenFabrics verbs package is
installed. Return None if not found.
"""
try:
# Try to locate Verbs by looking for a utility in the path
ibv_devices = which("ibv_devices")
# Run it (silently) to ensure it works
ibv_devices(output=str, error=str)
# Get path to executable
path = ibv_devices.exe[0]
# Remove executable name and "bin" directory
path = os.path.dirname(path)
path = os.path.dirname(path)
return path
except:
return None
class Openmpi(Package):
"""Open MPI is a project combining technologies and resources from
several other projects (FT-MPI, LA-MPI, LAM/MPI, and PACX-MPI)
@ -54,7 +74,7 @@ class Openmpi(Package):
variant('psm', default=False, description='Build support for the PSM library.')
variant('psm2', default=False, description='Build support for the Intel PSM2 library.')
variant('pmi', default=False, description='Build support for PMI-based launchers')
variant('verbs', default=False, description='Build support for OpenFabrics verbs.')
variant('verbs', default=_verbs_dir() is not None, description='Build support for OpenFabrics verbs.')
variant('mxm', default=False, description='Build Mellanox Messaging support')
variant('thread_multiple', default=False, description='Enable MPI_THREAD_MULTIPLE support')
@ -113,7 +133,6 @@ def install(self, spec, prefix):
# Fabrics
'--with-psm' if '+psm' in spec else '--without-psm',
'--with-psm2' if '+psm2' in spec else '--without-psm2',
('--with-%s' % self.verbs) if '+verbs' in spec else ('--without-%s' % self.verbs),
'--with-mxm' if '+mxm' in spec else '--without-mxm',
# Other options
'--enable-mpi-thread-multiple' if '+thread_multiple' in spec else '--disable-mpi-thread-multiple',
@ -121,6 +140,14 @@ def install(self, spec, prefix):
'--with-sqlite3' if '+sqlite3' in spec else '--without-sqlite3',
'--enable-vt' if '+vt' in spec else '--disable-vt'
])
if '+verbs' in spec:
path = _verbs_dir()
if path is not None:
config_args.append('--with-%s=%s' % (self.verbs, path))
else:
config_args.append('--with-%s' % self.verbs)
else:
config_args.append('--without-%s' % self.verbs)
# TODO: use variants for this, e.g. +lanl, +llnl, etc.
# use this for LANL builds, but for LLNL builds, we need:

44
var/spack/repos/builtin/packages/parmetis/package.py
View file

@ -26,33 +26,36 @@
from spack import *
import sys
class Parmetis(Package):
"""
ParMETIS is an MPI-based parallel library that implements a variety of algorithms for partitioning unstructured
graphs, meshes, and for computing fill-reducing orderings of sparse matrices.
"""
"""ParMETIS is an MPI-based parallel library that implements a variety of
algorithms for partitioning unstructured graphs, meshes, and for
computing fill-reducing orderings of sparse matrices."""
homepage = 'http://glaros.dtc.umn.edu/gkhome/metis/parmetis/overview'
url = 'http://glaros.dtc.umn.edu/gkhome/fetch/sw/parmetis/parmetis-4.0.3.tar.gz'
base_url = 'http://glaros.dtc.umn.edu/gkhome/fetch/sw/parmetis'
version('4.0.3', 'f69c479586bf6bb7aff6a9bc0c739628')
version('4.0.2', '0912a953da5bb9b5e5e10542298ffdce')
variant('shared', default=True, description='Enables the build of shared libraries')
variant('debug', default=False, description='Builds the library in debug mode')
variant('gdb', default=False, description='Enables gdb support')
depends_on('cmake @2.8:') # build dependency
depends_on('cmake@2.8:') # build dependency
depends_on('mpi')
patch('enable_external_metis.patch')
depends_on('metis@5:')
patch('enable_external_metis.patch')
# bug fixes from PETSc developers
# https://bitbucket.org/petsc/pkg-parmetis/commits/1c1a9fd0f408dc4d42c57f5c3ee6ace411eb222b/raw/
# https://bitbucket.org/petsc/pkg-parmetis/commits/1c1a9fd0f408dc4d42c57f5c3ee6ace411eb222b/raw/ # NOQA: ignore=E501
patch('pkg-parmetis-1c1a9fd0f408dc4d42c57f5c3ee6ace411eb222b.patch')
# https://bitbucket.org/petsc/pkg-parmetis/commits/82409d68aa1d6cbc70740d0f35024aae17f7d5cb/raw/
# https://bitbucket.org/petsc/pkg-parmetis/commits/82409d68aa1d6cbc70740d0f35024aae17f7d5cb/raw/ # NOQA: ignore=E501
patch('pkg-parmetis-82409d68aa1d6cbc70740d0f35024aae17f7d5cb.patch')
depends_on('gdb', when='+gdb')
def url_for_version(self, version):
verdir = 'OLD/' if version < Version('3.2.0') else ''
return '%s/%sparmetis-%s.tar.gz' % (Parmetis.base_url, verdir, version)
def install(self, spec, prefix):
options = []
@ -60,30 +63,27 @@ def install(self, spec, prefix):
build_directory = join_path(self.stage.path, 'spack-build')
source_directory = self.stage.source_path
metis_source = join_path(source_directory, 'metis')
# FIXME : Once a contract is defined, MPI compilers should be retrieved indirectly via spec['mpi'] in case
# FIXME : they use a non-standard name
options.extend(['-DGKLIB_PATH:PATH={metis_source}/GKlib'.format(metis_source=spec['metis'].prefix.include),
'-DMETIS_PATH:PATH={metis_source}'.format(metis_source=spec['metis'].prefix),
'-DCMAKE_C_COMPILER:STRING=mpicc',
'-DCMAKE_CXX_COMPILER:STRING=mpicxx'])
options.extend([
'-DGKLIB_PATH:PATH=%s/GKlib' % spec['metis'].prefix.include,
'-DMETIS_PATH:PATH=%s' % spec['metis'].prefix,
'-DCMAKE_C_COMPILER:STRING=%s' % spec['mpi'].mpicc,
'-DCMAKE_CXX_COMPILER:STRING=%s' % spec['mpi'].mpicxx
])
if '+shared' in spec:
options.append('-DSHARED:BOOL=ON')
if '+debug' in spec:
options.extend(['-DDEBUG:BOOL=ON',
'-DCMAKE_BUILD_TYPE:STRING=Debug'])
if '+gdb' in spec:
options.append('-DGDB:BOOL=ON')
with working_dir(build_directory, create=True):
cmake(source_directory, *options)
make()
make("install")
make('install')
# The shared library is not installed correctly on Darwin; correct this
# The shared library is not installed correctly on Darwin; fix this
if (sys.platform == 'darwin') and ('+shared' in spec):
fix_darwin_install_name(prefix.lib)

16
var/spack/repos/builtin/packages/py-autopep8/package.py Normal file
View file

@ -0,0 +1,16 @@
from spack import *
class PyAutopep8(Package):
"""Automatic pep8 formatter"""
homepage = "https://github.com/hhatto/autopep8"
url = "https://github.com/hhatto/autopep8/archive/ver1.2.2.tar.gz"
version('1.2.2', 'def3d023fc9dfd1b7113602e965ad8e1')
extends('python')
depends_on('py-setuptools')
depends_on('py-pep8')
def install(self, spec, prefix):
python('setup.py', 'install', '--prefix=%s' % prefix)

15
var/spack/repos/builtin/packages/py-pep8/package.py Normal file
View file

@ -0,0 +1,15 @@
from spack import *
class PyPep8(Package):
"""python pep8 format checker"""
homepage = "https://github.com/PyCQA/pycodestyle"
url = "https://github.com/PyCQA/pycodestyle/archive/1.7.0.tar.gz"
version('1.7.0', '31070a3a6391928893cbf5fa523eb8d9')
extends('python')
depends_on('py-setuptools')
def install(self, spec, prefix):
python('setup.py', 'install', '--prefix=%s' % prefix)

18
var/spack/repos/builtin/packages/rust-bindgen/package.py Normal file
View file

@ -0,0 +1,18 @@
from spack import *
import os
class RustBindgen(Package):
"""The rust programming language toolchain"""
homepage = "http://www.rust-lang.org"
url = "https://github.com/crabtw/rust-bindgen"
version('0.16', tag='0.16', git='https://github.com/crabtw/rust-bindgen')
extends("rust")
depends_on("llvm")
def install(self, spec, prefix):
env = dict(os.environ)
env['LIBCLANG_PATH'] = os.path.join(spec['llvm'].prefix, 'lib')
cargo('install', '--root', prefix, env=env)

63
var/spack/repos/builtin/packages/rust/package.py Normal file
View file

@ -0,0 +1,63 @@
from spack import *
import os
def get_submodules():
git = which('git')
git('submodule', 'update', '--init', '--recursive')
class Rust(Package):
"""The rust programming language toolchain"""
homepage = "http://www.rust-lang.org"
url = "https://github.com/rust-lang/rust"
version('1.8.0', tag='1.8.0', git="https://github.com/rust-lang/rust")
resource(name='cargo',
git="https://github.com/rust-lang/cargo.git",
tag='0.10.0',
destination='cargo')
extendable = True
# Rust
depends_on("llvm")
depends_on("curl")
depends_on("git")
depends_on("cmake")
depends_on("python@:2.8")
# Cargo
depends_on("openssl")
def install(self, spec, prefix):
configure('--prefix=%s' % prefix,
'--llvm-root=' + spec['llvm'].prefix)
make()
make("install")
# Install cargo, rust package manager
with working_dir(os.path.join('cargo', 'cargo')):
get_submodules()
configure('--prefix=' + prefix,
'--local-rust-root=' + prefix)
make()
make("install")
def setup_dependent_package(self, module, ext_spec):
"""
Called before python modules' install() methods.
In most cases, extensions will only need to have one or two lines::
cargo('build')
cargo('install', '--root', prefix)
or
cargo('install', '--root', prefix)
"""
# Rust extension builds can have a global cargo executable function
module.cargo = Executable(join_path(self.spec.prefix.bin, 'cargo'))

5
var/spack/repos/builtin/packages/scotch/Makefile.esmumps
View file

@ -1,5 +0,0 @@
esmumps : scotch
(cd esmumps ; $(MAKE) scotch && $(MAKE) install)
ptesmumps : ptscotch
(cd esmumps ; $(MAKE) ptscotch && $(MAKE) ptinstall)

83
var/spack/repos/builtin/packages/scotch/package.py
View file

@ -23,7 +23,6 @@
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
##############################################################################
import os
import re
from spack import *
@ -32,7 +31,7 @@ class Scotch(Package):
partitioning, graph clustering, and sparse matrix ordering."""
homepage = "http://www.labri.fr/perso/pelegrin/scotch/"
url = "http://gforge.inria.fr/frs/download.php/latestfile/298/scotch_6.0.3.tar.gz"
url = "http://gforge.inria.fr/frs/download.php/latestfile/298/scotch_6.0.3.tar.gz"
base_url = "http://gforge.inria.fr/frs/download.php/latestfile/298"
list_url = "http://gforge.inria.fr/frs/?group_id=248"
@ -44,6 +43,7 @@ class Scotch(Package):
variant('compression', default=True, description='Activate the posibility to use compressed files')
variant('esmumps', default=False, description='Activate the compilation of esmumps needed by mumps')
variant('shared', default=True, description='Build a shared version of the library')
variant('metis', default=True, description='Build metis and parmetis wrapper libraries')
depends_on('flex')
depends_on('bison')
@ -56,37 +56,15 @@ class Scotch(Package):
# from the Scotch hosting site. These alternative archives include a
# superset of the behavior in their default counterparts, so we choose to
# always grab these versions for older Scotch versions for simplicity.
def url_for_version(self, version):
return super(Scotch, self).url_for_version(version)
@when('@:6.0.0')
def url_for_version(self, version):
return '%s/scotch_%s_esmumps.tar.gz' % (Scotch.base_url, version)
@when('@6.0.1:')
def url_for_version(self, version):
return super(Scotch, self).url_for_version(version)
# NOTE: Several of the 'esmumps' enabled Scotch releases up to version
# 6.0.0 have broken build scripts that don't properly build 'esmumps' as a
# separate target, so we need a patch procedure to remove 'esmumps' from
# existing targets and to add it as a standalone target.
@when('@:6.0.0')
def patch(self):
makefile_path = os.path.join('src', 'Makefile')
with open(makefile_path, 'r') as makefile:
esmumps_enabled = any(re.search(r'^esmumps(\s*):(.*)$', line)
for line in makefile.readlines())
if not esmumps_enabled:
mff = FileFilter(makefile_path)
mff.filter(r'^.*((esmumps)|(ptesmumps)).*(install).*$', '')
mfesmumps_dir = os.path.dirname(os.path.realpath(__file__))
mfesmumps_path = os.path.join(mfesmumps_dir, 'Makefile.esmumps')
with open(makefile_path, 'a') as makefile:
makefile.write('\ninclude %s\n' % mfesmumps_path)
@when('@6.0.1:')
def patch(self):
pass
self.configure()
# NOTE: Configuration of Scotch is achieved by writing a 'Makefile.inc'
# file that contains all of the configuration variables and their desired
@ -103,13 +81,13 @@ def configure(self):
]
# Library Build Type #
if '+shared' in self.spec:
makefile_inc.extend([
# todo change for Darwin systems
'LIB = .so',
'CLIBFLAGS = -shared -fPIC',
'RANLIB = echo',
'AR = $(CC)',
'AR = $(CC)',
'ARFLAGS = -shared $(LDFLAGS) -o'
])
cflags.append('-fPIC')
@ -118,7 +96,7 @@ def configure(self):
'LIB = .a',
'CLIBFLAGS = ',
'RANLIB = ranlib',
'AR = ar',
'AR = ar',
'ARFLAGS = -ruv '
])
@ -171,20 +149,49 @@ def configure(self):
fh.write('\n'.join(makefile_inc))
def install(self, spec, prefix):
self.configure()
targets = ['scotch']
if '+mpi' in self.spec:
targets.append('ptscotch')
if '+esmumps' in self.spec:
targets.append('esmumps')
if '+mpi' in self.spec:
targets.append('ptesmumps')
if self.spec.version >= Version('6.0.0'):
if '+esmumps' in self.spec:
targets.append('esmumps')
if '+mpi' in self.spec:
targets.append('ptesmumps')
with working_dir('src'):
for target in targets:
make(target, parallel=(target != 'ptesmumps'))
# It seams that building ptesmumps in parallel fails, for
# version prior to 6.0.0 there is no separated targets force
# ptesmumps, this library is built by the ptscotch target. This
# should explain the test for the can_make_parallel variable
can_make_parallel = \
not (target == 'ptesmumps' or
(self.spec.version < Version('6.0.0') and
target == 'ptscotch'))
make(target, parallel=can_make_parallel)
# todo change this to take into account darwin systems
lib_ext = '.so' if '+shared' in self.spec else '.a'
# It seams easier to remove metis wrappers from the folder that will be
# installed than to tweak the Makefiles
if '+metis' not in self.spec:
with working_dir('lib'):
lib_ext = '.so' if '+shared' in self.spec else '.a'
force_remove('libscotchmetis{0}'.format(lib_ext))
force_remove('libptscotchparmetis{0}'.format(lib_ext))
with working_dir('include'):
force_remove('metis.h')
force_remove('parmetis.h')
if '~esmumps' in self.spec and self.spec.version < Version('6.0.0'):
with working_dir('lib'):
force_remove('libesmumps{0}'.format(lib_ext))
force_remove('libptesmumps{0}'.format(lib_ext))
with working_dir('include'):
force_remove('esmumps.h')
install_tree('bin', prefix.bin)
install_tree('lib', prefix.lib)

62
var/spack/repos/builtin/packages/stream/package.py Normal file
View file

@ -0,0 +1,62 @@
##############################################################################
# Copyright (c) 2013-2016, Lawrence Livermore National Security, LLC.
# Produced at the Lawrence Livermore National Laboratory.
#
# This file is part of Spack.
# Created by Todd Gamblin, tgamblin@llnl.gov, All rights reserved.
# LLNL-CODE-647188
#
# For details, see https://github.com/llnl/spack
# Please also see the LICENSE file for our notice and the LGPL.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU Lesser General Public License (as
# published by the Free Software Foundation) version 2.1, February 1999.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the IMPLIED WARRANTY OF
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the terms and
# conditions of the GNU Lesser General Public License for more details.
#
# You should have received a copy of the GNU Lesser General Public
# License along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
##############################################################################
from spack import *
class Stream(Package):
"""The STREAM benchmark is a simple synthetic benchmark program that
measures sustainable memory bandwidth (in MB/s) and the corresponding
computation rate for simple vector kernels."""
homepage = "https://www.cs.virginia.edu/stream/ref.html"
version('5.10', git='https://github.com/jeffhammond/STREAM.git')
variant('openmp', default=False, description='Build with OpenMP support')
def patch(self):
makefile = FileFilter('Makefile')
# Use the Spack compiler wrappers
makefile.filter('CC = .*', 'CC = cc')
makefile.filter('FC = .*', 'FC = f77')
cflags = '-O2'
fflags = '-O2'
if '+openmp' in self.spec:
cflags += ' ' + self.compiler.openmp_flag
fflags += ' ' + self.compiler.openmp_flag
# Set the appropriate flags for this compiler
makefile.filter('CFLAGS = .*', 'CFLAGS = {0}'.format(cflags))
makefile.filter('FFLAGS = .*', 'FFLAGS = {0}'.format(fflags))
def install(self, spec, prefix):
make()
# Manual installation
mkdir(prefix.bin)
install('stream_c.exe', prefix.bin)
install('stream_f.exe', prefix.bin)

51
var/spack/repos/builtin/packages/tbb/package.py
View file

@ -23,9 +23,9 @@
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
##############################################################################
from spack import *
import os
import glob
class Tbb(Package):
"""Widely used C++ template library for task parallelism.
Intel Threading Building Blocks (Intel TBB) lets you easily write parallel
@ -35,35 +35,39 @@ class Tbb(Package):
homepage = "http://www.threadingbuildingblocks.org/"
# Only version-specific URL's work for TBB
version('4.4.3', '80707e277f69d9b20eeebdd7a5f5331137868ce1', url='https://www.threadingbuildingblocks.org/sites/default/files/software_releases/source/tbb44_20160128oss_src_0.tgz')
version('4.4.4', 'd4cee5e4ca75cab5181834877738619c56afeb71', url='https://www.threadingbuildingblocks.org/sites/default/files/software_releases/source/tbb44_20160413oss_src.tgz') # NOQA: ignore=E501
version('4.4.3', '80707e277f69d9b20eeebdd7a5f5331137868ce1', url='https://www.threadingbuildingblocks.org/sites/default/files/software_releases/source/tbb44_20160128oss_src_0.tgz') # NOQA: ignore=E501
def coerce_to_spack(self,tbb_build_subdir):
for compiler in ["icc","gcc","clang"]:
fs = glob.glob(join_path(tbb_build_subdir,"*.%s.inc" % compiler ))
for f in fs:
lines = open(f).readlines()
of = open(f,"w")
for l in lines:
if l.strip().startswith("CPLUS ="):
def coerce_to_spack(self, tbb_build_subdir):
for compiler in ["icc", "gcc", "clang"]:
fs = glob.glob(join_path(tbb_build_subdir,
"*.%s.inc" % compiler))
for f in fs:
lines = open(f).readlines()
of = open(f, "w")
for l in lines:
if l.strip().startswith("CPLUS ="):
of.write("# coerced to spack\n")
of.write("CPLUS = $(CXX)\n")
elif l.strip().startswith("CPLUS ="):
elif l.strip().startswith("CPLUS ="):
of.write("# coerced to spack\n")
of.write("CONLY = $(CC)\n")
else:
of.write(l);
else:
of.write(l)
def install(self, spec, prefix):
#
# we need to follow TBB's compiler selection logic to get the proper build + link flags
# but we still need to use spack's compiler wrappers
if spec.satisfies('%gcc@6.1:') and spec.satisfies('@:4.4.3'):
raise InstallError('Only TBB 4.4.4 and above build with GCC 6.1!')
# We need to follow TBB's compiler selection logic to get the proper
# build + link flags but we still need to use spack's compiler wrappers
# to accomplish this, we do two things:
#
# * Look at the spack spec to determine which compiler we should pass to tbb's Makefile
# * Look at the spack spec to determine which compiler we should pass
# to tbb's Makefile;
#
# * patch tbb's build system to use the compiler wrappers (CC, CXX) for
# icc, gcc, clang
# (see coerce_to_spack())
# icc, gcc, clang (see coerce_to_spack());
#
self.coerce_to_spack("build")
@ -74,7 +78,6 @@ def install(self, spec, prefix):
else:
tbb_compiler = "gcc"
mkdirp(prefix)
mkdirp(prefix.lib)
@ -82,10 +85,10 @@ def install(self, spec, prefix):
# tbb does not have a configure script or make install target
# we simply call make, and try to put the pieces together
#
make("compiler=%s" %(tbb_compiler))
make("compiler=%s" % (tbb_compiler))
# install headers to {prefix}/include
install_tree('include',prefix.include)
install_tree('include', prefix.include)
# install libs to {prefix}/lib
tbb_lib_names = ["libtbb",
@ -94,10 +97,10 @@ def install(self, spec, prefix):
for lib_name in tbb_lib_names:
# install release libs
fs = glob.glob(join_path("build","*release",lib_name + ".*"))
fs = glob.glob(join_path("build", "*release", lib_name + ".*"))
for f in fs:
install(f, prefix.lib)
# install debug libs if they exist
fs = glob.glob(join_path("build","*debug",lib_name + "_debug.*"))
fs = glob.glob(join_path("build", "*debug", lib_name + "_debug.*"))
for f in fs:
install(f, prefix.lib)