foam-extend4.1-coherent-io/ThirdParty/README.ThirdParty

A new design for the ThirdParty packages:
=========================================

The main purpose of this new development is to build a complete ThirParty
packages set for OpenFOAM 1.6-ext using only the original package source 
tarball and some patch files ( when necessary).

A useful by-product of this development is also to provide some kind of binary
packaging of the ThirdParty packages.  

The RPM suite of tools was selected to develop a first prototype.  
The whole process needs to run and install in user-space, without the need to
be root for installing the packages.  
 
Here is what's available:
 
 a: A set of rpm spec files for specific ThirdParty packages.
 b: A suite of bash scripts to automate the complete sequence of  downloading,
    compiling, installing and generating RPMs. 
 c: An empty directory structure pre-configured and ready to proceed with the
    download, compilation and installation of chosen ThirdParty packages for
    OF-1.6-ext. 
 d: A simple mecanism that allows replacing any of the ThirdParty packages by
    a system installed package instead. 

Quick description of the main scripts:
-------------------------------------- 
   a: AllMake:
      Main wrapper script that will call AllMake.stage0 to AllMake.stage4
      scripts in sequence. 
 
   b: AllMake.stage0:
      This script is useful only for populating what I am calling the local
      "RPM vault" with pre-generated RPMs. This is the script written to
      address the use-case: "I have some pre-generated RPM files, now what"
      Basically, you call this script with a list of RPMs generated by the
      AllMake.stage(1-4) in order to populate the local  RPMS  vault.
      Once in place, these are the RPMs will be installed instead of
      proceeding with the standard compilation process. 
 
   c: AllMake.stage1:
      This script is taking care of the basic ThirdParty tools like compilers,
      cmake , python, etc. If we ever need to override the local version of
      flex or bison, this is where we will add those additional packages.  
      This stage will also generate a .sh and .csh file one needs to source in
      order to initialize the PATH and LD_LIBRARY_PATH env. variable. If you
      need to compile the rest of the ThirdParty packages with a new gcc
      compiler, you will need to source those .sh or .csh file in before
      activating the other AllMake.stage(2-4) scripts.
 
   d: AllMake.stage2:
      This script is taking care of the MPI communication libraries. Right
      now, only OpenMPI is supported. 
      This stage will also generate a .sh and .csh file one needs to source in
      order to initialize the PATH and LD_LIBRARY_PATH env. variable. You will
      need to source those .sh or .csh file in before activating the other
      AllMake.stage(3-4) scripts because some packages depends on the
      communication library. 
 
   e: AllMake.stage3:
      This script is taking care of the "standard" ThirdParty libraries like
      metis, scotch, mesquite, etc. 
      This stage will also generate a .sh and .csh file one needs to source in
      order to initialize the PATH and LD_LIBRARY_PATH env. variable. You will
      need to source those .sh or .csh file in before compiling OpenFOAM
      because some "Make/options" files will refer to environment variable
      that are specific to those packages
 
   f: AllMake.stage4:
      This script is taking care of Paraview and QT (and takes an awfull long
      time to compile, honest...). 
      This stage will also generate a .sh and .csh file one needs to source in
      order to initialize the PATH and LD_LIBRARY_PATH env. variable. You will
      need to source those .sh or .csh file in before compiling OpenFOAM
      because some "Make/options" files will refer to environment variable
      that are specific to those packages 
 
   g: tools/makeThirdPartyFunctionsForRPM:
       A suite of bash functions useful for wrapping the rpmbuild and rpm
       commands 
 
 
For every packages, this is the basic process we will be going through when
    starting building the ThirdParty packages from scratch:
    a: Verify in the local "RPM vault" if  a RPM is available for a given
       package 
    b: If it is, simply install this RPM and move on to the next package
    c: if the RPM is absent:
        i: verify if the source tarbal is available from the SOURCES "vault"
        ii: if it is not, download the tarball using the specified URL.
        iii: proceed with the extraction, patching, configuration,
             compilation, RPM generation and installation of the package. The
             generated RPM is always used for installation.  
    d: The default installation root  directory is "$WM_THIRD_PARTY_DIR". This
       can be overriden though when installing the RPM. 
 
Relocating the RPM root installation directory: 
    a: All the generated RPMs can be relocated, meaning that you can override
       the hard-coded root installation directory when using those RPMs for
       installation.  
    b: You can check that the RPM is relocatable by using the command rpm -qip
       thePackage.rpm. For example, from the cmake-2.8.3 package generated on
       one of my machine: 
 
      > rpm -qip cmake-2.8.3-darwinIntelGccDPOpt.i386.rpm| grep Relocations
         Name        : cmake
	 Relocations: /Users/beaudoin/Projets/SAMH/OpenFOAM/ThirdParty-1.6-ext-RPM-based
 
    The Relocations path is the actual location pointed by the
    $WM_THIRD_PARTY_DIR on my laptop when I generated the RPM. 
    It is the indication that the RPM is relocatable. This path will turn to
    be hardcoded in the RPM because the environment variable was expanded
    before generating the RPM. This is the default root directory where the
    RPM will install its "payload". This can be overridden using the 'rpm'
    command-line parameter --relocate OLDPATH=NEWPATH . 
 
    For example, let's say you want to install the RPM under the root
    directory /tmp/someDir instead, you will call the 'rpm' command like this: 
 
    rpm -ivh ./cmake-2.8.3-darwinIntelGccDPOpt.i386.rpm \
        --relocate /Users/beaudoin/Projets/SAMH/OpenFOAM/ThirdParty-1.6-ext-RPM-based=/tmp/someDir
 
    Even better, you can dig down the hard-coded path even deeper in order to
    relocate the whole installation directory, down to the last hard-coded
    subdirectory. 
    Just specify the whole path when using the --relocate  parameter
    So basically, you can install the RPM right under /usr if you want, hence
    bypassing the default sequence of package subdirectories I have chosen in
    order to stay close to the "traditional" ThirdParty layout.

9: Using system installed package
   It is possible to replace most of the ThirdParty packages by a system
   installed version simply by activating some specific environment variables
   in your file prefs.sh or prefs.csh.

   The files prefs.csh-EXAMPLE and prefs.sh-EXAMPLE provides a list of
   environment variables you need to activate in order to use a system
   installed version of a given package.

   For example, in order to use a system installed version of Scotch, here is
   the list of environment variable you must declare in your etc/prefs.sh file

   # System installed Scotch
   export SCOTCH_SYSTEM=1
   export SCOTCH_DIR=path_to_system_installed_scotch
   export SCOTCH_BIN_DIR=$SCOTCH_DIR/bin
   export SCOTCH_LIB_DIR=$SCOTCH_DIR/lib
   export SCOTCH_INCLUDE_DIR=$SCOTCH_DIR/include

   When the XXXX_SYSTEM environment variable is activated for package XXXX,
   the compilation and installation from the source tarball of the ThirdParty
   package  will be skipped. Just make sure all the necessary package specific
   environment variables are properly initialized, and that the system
   installed version of the package is compatible with the version made
   available through the source tarball. 
 
10: Things to do:
    a: Testing testing testing. This prototype was tested on the following
       platforms: 
 
       Mac OS X 10.6 (Snow Leopard) (non RPM based)
       Ubuntu 10.04 32bit (non RPM based)
       Centos 5.5 64bit (RPM based)
       OpenSUSE 11.3 64bit (RPM based)
   
    b: Maybe adding some RPM dependencies might be useful. I have not explored
       this yet. 
 
To be continued...
 
 
Martin Beaudoin

Last update: December 2010