250 lines
11 KiB
Text
250 lines
11 KiB
Text
|
A new design for the ThirdParty packages compilation and installation process:
|
||
|
|
==============================================================================
|
||
|
|
|
||
|
|
The main purpose of this new development is to build a complete ThirParty
|
||
|
|
packages set for foam-extend-3.1 using only the original package source
|
||
|
|
tarball and some patch files (when necessary).
|
||
|
|
|
||
|
|
A useful by-product of this development is also to provide some kind of
|
||
|
|
binary packaging of the ThirdParty packages. There are currently two types of
|
||
|
|
binary packaging generated by this compilation process: RPMs and compressed
|
||
|
|
tarballs (TGZs)
|
||
|
|
|
||
|
|
The RPM package manager was selected to develop a first prototype. RPM is
|
||
|
|
available for multiple flavors of Unix/Linux, and offers all the necessary
|
||
|
|
functionnality to configured, patch, compile and install source code packages.
|
||
|
|
|
||
|
|
One of the main requirement for this new development is that the whole process
|
||
|
|
needs to run and install in user-space, without the need to be root for
|
||
|
|
installing the packages.
|
||
|
|
|
||
|
|
Here is what's available:
|
||
|
|
|
||
|
|
a: A set of RPM spec files for specific ThirdParty packages.
|
||
|
|
b: A set of bash scripts to automate the complete sequence of downloading,
|
||
|
|
compiling, installing and generating RPMs.
|
||
|
|
c: An directory structure pre-configured and ready to proceed with the
|
||
|
|
download, compilation and installation of chosen ThirdParty packages for
|
||
|
|
OF-1.6-ext.
|
||
|
|
|
||
|
|
|
||
|
|
1: Quick description of the main scripts:
|
||
|
|
-----------------------------------------
|
||
|
|
a: AllMake:
|
||
|
|
Main wrapper script that will call AllMake.stage0 to AllMake.stage4
|
||
|
|
scripts in sequence.
|
||
|
|
|
||
|
|
b: AllMake.stage0:
|
||
|
|
This script is useful only for populating what I am calling the local "RPM
|
||
|
|
vault" with pre-generated RPMs.
|
||
|
|
|
||
|
|
This is the script written to address the following use case:
|
||
|
|
"I have some pre-generated RPM files, now what"
|
||
|
|
|
||
|
|
Basically, you call this script with a list of RPMs already generated by
|
||
|
|
whichever of the AllMake.stage(1-4) in order to populate the local RPMS
|
||
|
|
vault. Once in place in the RPM vault, these are the RPMs that will get
|
||
|
|
directly installed by the AllMake.stage(1-4) scripts, instead of being
|
||
|
|
regenerated by the normal compilation and install process of the
|
||
|
|
ThirdParty packages.
|
||
|
|
|
||
|
|
c: AllMake.stage1:
|
||
|
|
This script is taking care of the basic ThirdParty tools like compilers,
|
||
|
|
flex, bison, cmake , python, etc.
|
||
|
|
This compilation stage will generate .sh and .csh files that will be
|
||
|
|
sourced by your file settings.sh or settings.csh in order to initialize
|
||
|
|
the PATH and LD_LIBRARY_PATH environment variable for the stage1 packages.
|
||
|
|
|
||
|
|
d: AllMake.stage2:
|
||
|
|
This script is taking care of the MPI communication libraries. Right now,
|
||
|
|
only OpenMPI is supported.
|
||
|
|
This compilation stage will generate .sh and .csh files that will be
|
||
|
|
sourced by your file settings.sh or settings.csh in order to initialize
|
||
|
|
the PATH and LD_LIBRARY_PATH environment variable for the stage2 packages.
|
||
|
|
|
||
|
|
e: AllMake.stage3:
|
||
|
|
This script is taking care of the "standard" ThirdParty libraries like
|
||
|
|
metis, scotch, mesquite, etc.
|
||
|
|
This compilation stage will generate .sh and .csh files that will be
|
||
|
|
sourced by your file settings.sh or settings.csh in order to initialize
|
||
|
|
the PATH and LD_LIBRARY_PATH environment variable for the stage3 packages.
|
||
|
|
|
||
|
|
f: AllMake.stage4:
|
||
|
|
This script is taking care of Paraview and QT (and this step will take an
|
||
|
|
awfull long time to compile, honest...).
|
||
|
|
This compilation stage will generate .sh and .csh files that will be
|
||
|
|
sourced by your file settings.sh or settings.csh in order to initialize
|
||
|
|
the PATH and LD_LIBRARY_PATH environment variable for the stage4 packages.
|
||
|
|
|
||
|
|
g: tools/makeThirdPartyFunctionsForRPM:
|
||
|
|
A set of bash functions useful for wrapping the rpmbuild and rpm commands
|
||
|
|
|
||
|
|
|
||
|
|
2: Important notice:
|
||
|
|
--------------------
|
||
|
|
Every single ThirdParty package will be installed with a set of companion .sh
|
||
|
|
and .csh files one needs to source in order to properly initialize the PATH and
|
||
|
|
LD_LIBRARY_PATH environment variables for running the packages.
|
||
|
|
|
||
|
|
Some packages will also create environments variables necessary for the
|
||
|
|
compilation of some foam-extend libraries.
|
||
|
|
|
||
|
|
The scripts AllMake.stage[1-4] are taking care of refreshing the environment
|
||
|
|
variable after the installation of every single package, so you will not need to
|
||
|
|
manually refresh your foam-extend environment yourself when building individual
|
||
|
|
ThirdParty packages through the AllMake.stage[1-4] scripts.
|
||
|
|
|
||
|
|
The main Allwmake script for the compilation of foam-extend will also take care of
|
||
|
|
refreshing the environment variable before moving on to the compilation of the
|
||
|
|
main foam-extend libraries and applications.
|
||
|
|
|
||
|
|
However, if you decide to compile the libraries and/or applications manually, or
|
||
|
|
through the src/Allwmake or applications/Allwmake scripts, YOU MUST MAKE SURE
|
||
|
|
YOUR ENVIRONMENT VARIABLES FOR THE THIRDPARTY PACKAGES ARE UPDATED FIRST!
|
||
|
|
|
||
|
|
In that case, simply resource your bashrc or cshrc file once, before moving on
|
||
|
|
with the rest of your compilation activities.
|
||
|
|
|
||
|
|
|
||
|
|
3: The building and installation process overview:
|
||
|
|
--------------------------------------------------
|
||
|
|
For every ThirdParty packages, this is the basic process we will be going
|
||
|
|
through when starting the compilation from scratch:
|
||
|
|
|
||
|
|
a: Verify in the local "RPM vault" if a RPM file is available for the
|
||
|
|
required ThirdParty package.
|
||
|
|
|
||
|
|
b: If the RPM is available, and the package already installed, then there is
|
||
|
|
nothing to do for this package, and we proceed with the next package.
|
||
|
|
|
||
|
|
c: If the RPM is available, and the package is NOT already installed, then
|
||
|
|
we simply install it using the RPM and move on to the next package.
|
||
|
|
|
||
|
|
d: if the RPM is absent:
|
||
|
|
1: we verify if the source tarbal is available from the SOURCES "vault"
|
||
|
|
2: if it is not, we download the tarball using the specified URL.
|
||
|
|
3: we then proceed with the extraction, patching, configuration,
|
||
|
|
compilation, RPM generation, TGZ generation and installation of the
|
||
|
|
package.
|
||
|
|
|
||
|
|
Important notices:
|
||
|
|
- The generated RPM is always used for the installation, not the TGZ file.
|
||
|
|
|
||
|
|
- The compressed tarball (.tgz) file is generated for systems where
|
||
|
|
using rpm for installing packages as a non-root user might be a problem.
|
||
|
|
|
||
|
|
- The .rpm files will be located under the directory
|
||
|
|
$WM_THIRD_PARTY_DIR/rpmBuild/RPMS
|
||
|
|
|
||
|
|
- The .tgz files will be located under the directory
|
||
|
|
$WM_THIRD_PARTY_DIR/rpmBuild/TGZS
|
||
|
|
|
||
|
|
e: The default installation root directory is "$WM_THIRD_PARTY_DIR".
|
||
|
|
This can be overridden though when installing the RPM.
|
||
|
|
|
||
|
|
f: Through the modification of the AllMake.stage[1-4] scripts, it is
|
||
|
|
possible to specify a modified version of a .spec file for a given
|
||
|
|
package in order to tweak its local compilation or installation.
|
||
|
|
|
||
|
|
The modified .spec file will have to be installed under the SPECS
|
||
|
|
directory, and the corresponding invocation or the command rpm_make in
|
||
|
|
the AllMake.stage[1-4] file will need to be modified accordingly.
|
||
|
|
|
||
|
|
|
||
|
|
4: Relocating the RPM root installation directory:
|
||
|
|
--------------------------------------------------
|
||
|
|
|
||
|
|
Warning: This section is a bit advanced, and as of yet, I have no evidence that
|
||
|
|
this feature has ever been used by anybody. So please feel free to skip
|
||
|
|
this section if you don't need to relocate your ThirdParty packages
|
||
|
|
installation directory, which you should not have do on most
|
||
|
|
installation anyway.
|
||
|
|
|
||
|
|
a: All the generated RPMs can be relocated, meaning that you can override
|
||
|
|
the hard-coded root installation directory when using those RPMs for
|
||
|
|
installation.
|
||
|
|
|
||
|
|
b: You can check that the RPM is relocatable by using the command rpm -qip
|
||
|
|
thePackage.rpm. For example, from the cmake-2.8.3 package generated on one
|
||
|
|
of my machines, I will get:
|
||
|
|
|
||
|
|
> rpm -qip cmake-2.8.3-darwinIntelGccDPOpt.i386.rpm| grep Relocations
|
||
|
|
Name : cmake Relocations: /home/beaudoin/foam/foam-extend-3.1/ThirdParty
|
||
|
|
|
||
|
|
The Relocations path is the actual location pointed by the
|
||
|
|
$WM_THIRD_PARTY_DIR on my virtual machine when I generated the RPM. It is
|
||
|
|
the indication that the RPM is relocatable. This path will end up being
|
||
|
|
hardcoded in the RPM because the environment variable was expanded before
|
||
|
|
generating the RPM.
|
||
|
|
|
||
|
|
This is the default root directory where the RPM will install its
|
||
|
|
"payload". This can be overridden using the 'rpm' command-line parameter
|
||
|
|
--relocate OLDPATH=NEWPATH .
|
||
|
|
|
||
|
|
For example, let's say I want to install my cmake-2.8.3 RPM under the root
|
||
|
|
directory /tmp/someDir instead. I will need to call the 'rpm' command like
|
||
|
|
this:
|
||
|
|
|
||
|
|
rpm -ivh ./cmake-2.8.3-darwinIntelGccDPOpt.i386.rpm \
|
||
|
|
--relocate /home/beaudoin/foam/foam-extend-3.1/ThirdParty=/tmp/someDir
|
||
|
|
|
||
|
|
Even better, you can dig down the hard-coded path even deeper in order to
|
||
|
|
relocate the whole installation directory, down to the last hard-coded
|
||
|
|
subdirectory.
|
||
|
|
Just specify the whole path when using the --relocate parameter
|
||
|
|
So basically, you can install the RPM right under /usr if you want, hence
|
||
|
|
bypassing the default sequence of package subdirectories I have chosen in
|
||
|
|
order to stay close to the "traditional" ThirdParty layout.
|
||
|
|
|
||
|
|
|
||
|
|
5: Things to do:
|
||
|
|
----------------
|
||
|
|
a: Testing testing testing. This prototype was tested on the following
|
||
|
|
platforms:
|
||
|
|
|
||
|
|
Centos 5.5 64-bit (RPM based)
|
||
|
|
Mac OS X 10.6 (Snow Leopard) (non RPM based)
|
||
|
|
OpenSUSE 11.3 64-bit (RPM based)
|
||
|
|
RedHat Enterprise Linux 6 64-bit (RPM based)
|
||
|
|
Ubuntu 10.04 32-bit (non RPM based)
|
||
|
|
Ubuntu 10.04 64-bit (non RPM based)
|
||
|
|
|
||
|
|
b: Maybe adding some RPM dependencies might be useful. I have not explored
|
||
|
|
this yet.
|
||
|
|
|
||
|
|
c: Supplying compilation recipes with the source code for various OSes.
|
||
|
|
Lots of traffic on the Forum about this. This does not need to be that
|
||
|
|
complicated.
|
||
|
|
|
||
|
|
|
||
|
|
|
||
|
|
6: Gotchas and solutions:
|
||
|
|
-------------------------
|
||
|
|
|
||
|
|
1: It was reported that version 5.2.0 of rpm does not work well with the provided scripts.
|
||
|
|
rpm version 5.2.1 is working nicely.
|
||
|
|
|
||
|
|
|
||
|
|
2: Error messages from rpm:
|
||
|
|
|
||
|
|
Problem : error: can't create transaction lock on /var/lock/rpm/transaction
|
||
|
|
|
||
|
|
Solution: add the following entry in your file ~/.rpmmacros
|
||
|
|
|
||
|
|
%_rpmlock_path %{_dbpath}/__db.000
|
||
|
|
|
||
|
|
|
||
|
|
7: Reporting errors:
|
||
|
|
--------------------
|
||
|
|
|
||
|
|
Please report ThirdParty packages related bugs to the Bug tracking system
|
||
|
|
for foam-extend-3.1
|
||
|
|
http://sourceforge.net/apps/mantisbt/openfoam-extend/main_page.php
|
||
|
|
|
||
|
|
|
||
|
|
|
||
|
|
Martin Beaudoin
|
||
|
|
Rev 0: 05/2011
|
||
|
|
Rev 1: 08/2011
|
||
|
|
Rev 2: 05/2014
|