* Module files now are generated using a template engine refers #2902#3173
jinja2 has been hooked into Spack.
The python module `modules.py` has been splitted into several modules
under the python package `spack/modules`. Unit tests stressing module
file generation have been refactored accordingly.
The module file generator for Lmod has been extended to multi-providers
and deeper hierarchies.
* Improved the support for templates in module files.
Added an entry in `config.yaml` (`template_dirs`) to list all the
directories where Spack could find templates for `jinja2`.
Module file generators have a simple override mechanism to override
template selection ('modules.yaml' beats 'package.py' beats 'default').
* Added jinja2 and MarkupSafe to vendored packages.
* Spec.concretize() sets mutual spec-package references
The correct place to set the mutual references between spec and package
objects at the end of concretization. After a call to concretize we
should now be ensured that spec is the same object as spec.package.spec.
Code in `build_environment.py` that was performing the same operation
has been turned into an assertion to be defensive on the new behavior.
* Improved code and data layout for modules and related tests.
Common fixtures related to module file generation have been extracted
in `conftest.py`. All the mock configurations for module files have been
extracted from python code and have been put into their own yaml file.
Added a `context_property` decorator for the template engine, to make
it easy to define dictionaries out of properties.
The default for `verbose` in `modules.yaml` is now False instead of True.
* Extendable module file contexts + short description from docstring
The contexts that are used in conjunction with `jinja2` templates to
generate module files can now be extended from package.py and
modules.yaml.
Module files generators now infer the short description from package.py
docstring (and as you may expect it's the first paragraph)
* 'module refresh' regenerates all modules by default
`module refresh` without `--module-type` specified tries to
regenerate all known module types. The same holds true for `module rm`
Configure options used at build time are extracted and written into the
module files where possible.
* Fixed python3 compatibility, tests for Lmod and Tcl.
Added test for exceptional paths of execution when generating Lmod
module files.
Fixed a few compatibility issues with python3.
Fixed a bug in Tcl with naming_scheme and autoload + unit tests
* Updated module file tutorial docs. Fixed a few typos in docstrings.
The reference section for module files has been reorganized. The idea is
to have only three topics at the highest level:
- shell support + spack load/unload use/unuse
- module file generation (a.k.a. APIs + modules.yaml)
- module file maintenance (spack module refresh/rm)
Module file generation will cover the entries in modules.yaml
Also:
- Licenses have been updated to include NOTICE and extended to 2017
- docstrings have been reformatted according to Google style
* Removed redundant arguments to RPackage and WafPackage.
All the callbacks in `RPackage` and `WafPackage` that are not build
phases have been modified not to accept a `spec` and a `prefix`
argument. This permits to leverage the common `configure_args` signature
to insert by default the configuration arguments into the generated
module files. I think it's preferable to handling those packages
differently than `AutotoolsPackage`. Besides only one package seems
to override one of these methods.
* Fixed broken indentation + improved resiliency of refresh
Fixed broken indentation in `spack module refresh` (probably a rebase
gone silently wrong?). Filter the writers for blacklisted specs before
searching for name clashes. An error with a single writer will not
stop regeneration, but instead will print a warning and continue
the command.
* Add --with-mpi=spec['mpi'].name at configure step to avoid error when finding 2 MPI implementations
* Improve scorep package with different compilers and mpi providers
Merge work from multiple contributors on the mesa package. This
includes the following changes and decisions:
* Added in mesa +hwrender support
* make +swrender the default selection since this represents a very
common usage case (ie, cluster usage)
* leave ~llvm as the default, since inadvertently building this package
could take a very long time. May need to revisit this assumption in
the future since the performance difference can be quite substantial.
* build libGLESv1+2 and libGL for all variants
* avoid dependence on dri/libdrm when ~hwrender
* disable gallium drivers if none are requested
- converted `log_path` and `env_path` to properties of PackageBase.
- InstallErrors in build_environment are now annotated with the package
that caused them, in the 'pkg' attribute.
- Add `--show-log-on-error` option to `spack install` that catches
InstallErrors and prints the log to stderr if it exists.
Note that adding a reference to the Pakcage allows a lot of stuff
currently handled by do_install() and build_environment to be handled
externally.
- '\b' in regular expression needs to be in a raw string (r'\b')
- Regression test that would've caught this was unintentionally disabled
- This fixes the string and the test
Fixes a problem in ymake-filter: The line buffer is currently hardcoded to be
2048 bytes large but some Makefiles contain lines longer than that. This
caused the Makefiles to sometimes not be generated, consequently failing parts
of the build.
setup-env.sh adds the 'module' command to the user's environment
if it is not defined and if there is a Spack installation of
environment-modules available. This commit updates that logic to
perform these checks and updates quietly.
The correct place to set the mutual references between spec and
package objects is at the end of concretization. After a call to
concretize we should now be ensured that spec is the same object
as spec.package.spec.
Code in `build_environment.py` that was performing the same
operation has been turned into an assertion to be defensive on
the new behavior.
- Fixes bugs where concretization would fail due to an erroneously cached
_concrete attribute.
- Ripped out a bunch of code in spec.py that isn't needed/valid anymore:
- The various concrete() methods on different types of Specs would
attempt to statically compute whether the Spec was concrete.
- This dates back to when DAGs were simpler and there were no optional
dependencies. It's actually NOT possible to compute statically
whether a Spec is concrete now. The ONLY way you know is if it goes
through concretization and is marked concrete once that completes.
- This commit removes all simple concreteness checks and relies only on
the _concrete attribute. This should make thinking about
concreteness simpler.
- Fixed a couple places where Specs need to be marked concrete explicitly.
- Specs read from files and Specs that are destructively copied from
concrete Specs now need to be marked concrete explicitly.
- These spots may previously have "worked", but they were brittle and
should be explcitly marked anyway.
- Dependencies in concrete specs did not previously have their cache
fields (_concrete, _normal, etc.) preserved.
- _dup and _dup_deps weren't passing each other enough information to
preserve concreteness properly, so only the root was properly
preserved.
- cached concreteness is now preserved properly for the entire DAG, not
just the root.
- added method docs.
