hlrs-software-stack/spack
7
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

A new subclass StagedPackage(Package) is introduced. This PR should

Browse source 
not change the behavior for existing packages that subclass from
spack.Package.

If a package subclasses spack.StagedPackage instead of spack.Package,
the install() phase (run inside a forked process) is now separated
into sub-stages:

    a) spconfig: Generate a script spconfig.py that will configure the
       package (eg by running CMake or ./configure) This is for use if
       the user wishes to build externally from Spack.  Therefore, the
       Spack compiler wrappers are NOT used here.  Currently, that
       means that RPATH support is up to the user.

    b) configure: Configure the project (eg: runs configure, CMake,
       etc).  This will configure it for use within Spack, using the
       Spack wrapper.

    c) build: eg: "make"

    d) install: eg: "install"

If one chooses to use StagedPackage instead of Package, then one must
implement each of the install sub-stages as a separate method.
StagedPackage.install() then calls each of the sub-stages as
appropriate.

StagedPackage can be configured to only run certain sub-stages.  This
is done by setting the optional kwarg install_phases when calling
do_install().  Setting install_phases() ONLY has an effect on
StagedPackage, not on any existing packages.  By default,
install_phases is set to make StagedPackage run the same stages that
are normally run for any package: configure, build, install (and NOT
spconfig).

The ability for Spack to run stages selectively for StagedPackage
instances will enable new functionality.  For example, explicit
CMake/Autotools helpers that allow Spack to help configure a user's
project without fetching, building or installing it.

-------------

One implementation of StagedPackage is provided, CMakePackage.  This
has the following advantage for CMake-based projects over using the
standard Package class:

  a) By separating out the phases, it enables future new functionality
     for packages that use it.

  b) It provides an implementation of intall_spconfig(), which will
     help users configure their CMake-based projects.

CMakePackage expects users to implement configure_args() and
configure_env().  These methods provide the package-specific arguments
and environment needed to properly configure the package.  They are
placed in separated functions because they are used in both the
spconfig and configure stages.

TODO:

1. Generate spconfig.py scripts that are more readable.  This allows
   the users to understand how their project is configured.

2. Provide a practical way for the user to ACCESS the spconfig stage
   without building the project through Spack.

3. The CMAKE_TRANSITIVE_INCLUDE_PATH stuff needs to be reworked; it
   should be considered provisional for now.

4. User of Autotools might wish to make a similar ConfigurePackage
   subclass of StagedPackage.

---------------

One package using CMakePackage is introduced.  See ibmisc/package.py.
This commit is contained in:
Elizabeth F 2016-03-11 23:30:38 -05:00
parent 081bdd0828
commit 90bb855ffa
2 changed files with 145 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

100
lib/spack/spack/package.py
View file

@ -66,7 +66,7 @@
from spack.version import *
from spack.stage import Stage, ResourceStage, StageComposite
from spack.util.compression import allowed_archive, extension
from spack.util.executable import ProcessError
from spack.util.executable import ProcessError, which
from spack.util.environment import dump_environment
"""Allowed URL schemes for spack packages."""
@ -826,7 +826,8 @@ def _resource_stage(self, resource):
def do_install(self,
keep_prefix=False, keep_stage=False, ignore_deps=False,
skip_patch=False, verbose=False, make_jobs=None, fake=False):
skip_patch=False, verbose=False, make_jobs=None, fake=False,
install_phases = {'spconfig', 'configure', 'build', 'install'}):
"""Called by commands to install a package and its dependencies.
Package implementations should override install() to describe
@ -881,6 +882,10 @@ def build_process():
tty.msg("Building %s" % self.name)
self.stage.keep = keep_stage
self.install_phases = install_phases
self.build_directory = join_path(self.stage.path, 'spack-build')
self.source_directory = self.stage.source_path
with self.stage:
# Run the pre-install hook in the child process after
# the directory is created.
@ -1291,6 +1296,97 @@ def _hms(seconds):
if s: parts.append("%.2fs" % s)
return ' '.join(parts)
class StagedPackage(Package):
"""A Package subclass where the install() is split up into stages."""
def install_spconfig(self):
"""Creates an spconfig.py script to configure the package later if we like."""
raise InstallError("Package %s provides no install_spconfig() method!" % self.name)
def install_configure(self):
"""Runs the configure process."""
raise InstallError("Package %s provides no install_configure() method!" % self.name)
def install_build(self):
"""Runs the build process."""
raise InstallError("Package %s provides no install_build() method!" % self.name)
def install_install(self):
"""Runs the install process."""
raise InstallError("Package %s provides no install_install() method!" % self.name)
def install(self, spec, prefix):
if 'spconfig' in self.install_phases:
self.install_spconfig()
if 'configure' in self.install_phases:
self.install_configure()
if 'build' in self.install_phases:
self.install_build()
if 'install' in self.install_phases:
self.install_install()
else:
# Create a dummy file so the build doesn't fail.
# That way, the module file will also be created.
with open(os.path.join(prefix, 'dummy'), 'w') as fout:
pass
class CMakePackage(StagedPackage):
def configure_args(self):
"""Returns package-specific arguments to be provided to the configure command."""
return list()
def configure_env(self):
"""Returns package-specific environment under which the configure command should be run."""
return dict()
def cmake_transitive_include_path(self):
return ';'.join(
os.path.join(dep, 'include')
for dep in os.environ['SPACK_DEPENDENCIES'].split(os.pathsep)
)
def install_spconfig(self):
cmd = [str(which('cmake'))] + \
spack.build_environment.get_std_cmake_args(self) + \
['-DCMAKE_INSTALL_PREFIX=%s' % os.environ['SPACK_PREFIX'],
'-DCMAKE_C_COMPILER=%s' % os.environ['SPACK_CC'],
'-DCMAKE_CXX_COMPILER=%s' % os.environ['SPACK_CXX'],
'-DCMAKE_Fortran_COMPILER=%s' % os.environ['SPACK_FC']] + \
self.configure_args()
env = dict()
env['PATH'] = os.environ['PATH']
env['CMAKE_TRANSITIVE_INCLUDE_PATH'] = self.cmake_transitive_include_path()
env['CMAKE_PREFIX_PATH'] = os.environ['CMAKE_PREFIX_PATH']
with open('spconfig.py', 'w') as fout:
fout.write('import sys\nimport os\nimport subprocess\n')
fout.write('env = {}\n'.format(repr(env)))
fout.write('cmd = {} + sys.argv[1:]\n'.format(repr(cmd)))
fout.write('proc = subprocess.Popen(cmd, env=env)\nproc.wait()\n')
def install_configure(self):
cmake = which('cmake')
with working_dir(self.build_directory, create=True):
os.environ.update(self.configure_env())
os.environ['CMAKE_TRANSITIVE_INCLUDE_PATH'] = self.cmake_transitive_include_path()
options = self.configure_args() + spack.build_environment.get_std_cmake_args(self)
cmake(self.source_directory, *options)
def install_build(self):
with working_dir(self.build_directory, create=False):
make()
def install_install(self):
with working_dir(self.build_directory, create=False):
make('install')
class FetchError(spack.error.SpackError):
"""Raised when something goes wrong during fetch."""

47
var/spack/repos/builtin/packages/ibmisc/package.py Normal file
View file

@ -0,0 +1,47 @@
from spack import *
class Ibmisc(CMakePackage):
"""Misc. reusable utilities used by IceBin."""
homepage = "https://github.com/citibeth/ibmisc"
url = "https://github.com/citibeth/ibmisc/tarball/123"
version('0.1.0', '12f2a32432a11db48e00217df18e59fa')
variant('everytrace', default=False, description='Report errors through Everytrace')
variant('proj', default=True, description='Compile utilities for PROJ.4 library')
variant('blitz', default=True, description='Compile utilities for Blitz library')
variant('netcdf', default=True, description='Compile utilities for NetCDF library')
variant('boost', default=True, description='Compile utilities for Boost library')
variant('udunits2', default=True, description='Compile utilities for UDUNITS2 library')
variant('googletest', default=True, description='Compile utilities for Google Test library')
variant('python', default=True, description='Compile utilities for use with Python/Cython')
extends('python')
depends_on('eigen')
depends_on('everytrace', when='+everytrace')
depends_on('proj', when='+proj')
depends_on('blitz', when='+blitz')
depends_on('netcdf-cxx4', when='+netcdf')
depends_on('udunits2', when='+udunits2')
depends_on('googletest', when='+googletest')
depends_on('py-cython', when='+python')
depends_on('py-numpy', when='+python')
depends_on('boost', when='+boost')
# Build dependencies
depends_on('cmake')
depends_on('doxygen')
def configure_args(self):
spec = self.spec
return [
'-DUSE_EVERYTRACE=%s' % ('YES' if '+everytrace' in spec else 'NO'),
'-DUSE_PROJ4=%s' % ('YES' if '+proj' in spec else 'NO'),
'-DUSE_BLITZ=%s' % ('YES' if '+blitz' in spec else 'NO'),
'-DUSE_NETCDF=%s' % ('YES' if '+netcdf' in spec else 'NO'),
'-DUSE_BOOST=%s' % ('YES' if '+boost' in spec else 'NO'),
'-DUSE_UDUNITS2=%s' % ('YES' if '+udunits2' in spec else 'NO'),
'-DUSE_GTEST=%s' % ('YES' if '+googletest' in spec else 'NO')]