Rework Spack's continuous integration workflow to be environment-based.
- Add the `spack ci` command, which replaces the many scripts in `bin/`
- `spack ci` decouples the CI workflow from the spack instance:
- CI is defined in a spack environment
- environment is in its own (single) git repository, separate from Spack
- spack instance used to run the pipeline is up to the user
- A new `gitlab-ci` section in environments allows users to configure how
specs in the environment should be mapped to runners
- Compilers can be bootstrapped in the new pipeline workflow
- Add extensive documentation on pipelines (see `pipelines.rst` for further details)
- Add extensive tests for pipeline code
* Update and fix samtools
This PR adds samtools-1.10 and sets the htlib directory so that the
spack built htslib can be used. This PR also arranges the dependencies
so that the htslib sequence is grouped on its own. Finally, the bzip2
dependency is removed and python and perl run dependencies are added.
* Fix samtools when built with ncurses+termlib
* The CI flake8 tests require lowercase variable
Interestingly, this did not show up when I ran `spack flake8` locally.
Unfortunately UCX 1.7.0 is appearing in RPMS before it's officially released.
There's a problem with Open MPI 4.0.x where x < 3 and this version of UCX,
namely that the UCT BTL fails to compile.
See https://github.com/open-mpi/ompi/issues/7128
This patch works around the problem by disabling the build of the UCT BTL
for releases 4.0.0 to 4.0.2.
add hppritcha (me) as maintainer
Signed-off-by: Howard Pritchard <howardp@lanl.gov>
* Add +cfitsio variant to wcslib dependency
* Replace ncurses dependency with readline dependency
casacore explicitly may depend on readline, not ncurses
* Add workaround for casacore's readline dependency
casacore optionally depends upon readline, but it's CMakeLists.txt provides no
user control over whether or not readline becomes a dependency. As readline is
often present by default on systems, it's better for this package to explicitly
depend on readline in order to prevent linking to whatever system version of the
library happens to be found during the build process. This should be considered
a workaround until casacore's CMakeLists.txt is fixed.
* Apply workaround for casacore's dependency on SOFA
Similar to the issues with casacore's readline dependency, casacore's optional
dependency on SOFA does not provide the user with a means of controlling the
dependency during build time. Unlike the readline library, the SOFA library is
unlikely to exist on most systems by default. As the SOFA dependency is only
optionally used for testing casacore, requiring it by default is not a good
workaround. Until casacore's CMakeLists.txt is fixed, this variant has been
removed to avoid unexpected library dependencies in the installed package.
* Add newer casacore versions
* Add mpokorny to maintainer field
- The suite-sparse author publishes new versions starting with 5.5.0 on GitHub, see https://github.com/DrTimothyAldenDavis/SuiteSparse/releases and http://faculty.cse.tamu.edu/davis/SuiteSparse/
- change spack to download from there
- updated sha256 checksums from GitHub for all available releases
- For versions 5.4.0, 5.5.0, 5.6.0 there is a slightly different compilation necessary: first `make default` then `make install`.
Summary of the version changes (+ added, -removed [because not available on GitHub]):
```
+ 5.6.0
+ 5.5.0
+ 5.4.0
5.3.0
5.2.0
+ 5.1.2
5.1.0
+ 5.0.0
+ 4.5.6
4.5.5
- 4.5.4
4.5.3
- 4.5.1
```
to disabled use of libunwind. Without this mesa fails to build
using recent Cray compilers - cce 9 and higher - on aarch64 systems.
Signed-off-by: Howard Pritchard <hppritcha@gmail.com>
It seems that stable versions of perl also install a `perlX.Y.Z` binary.
However, it seems that this binary can hang if used in conjunction with
Spack's sbang workaround, as observed during automake's build.
* Update py-csvkit
This PR updates the py-csvkit package. This version requires a python
stack based on agate and this PR includes the new dependency packages.
- py-agate-dbf
- py-agate-excel
- py-agate-sql
- py-agate
- py-dbfread
- py-isodate
- py-leather
- py-parsedatetime
- py-python-slugify
- py-pytimeparse
- py-text-unidecode
* Replace the copy/pasted apostrophes
Python 2 can not process the copy/pasted apostrophes so replace them
with standard single quote character.
* Add version constraints on dependencies
* Add version contraint to graphviz patches
This PR restricts the graphviz version that the patches for building
with the Intel compiler apply to. The two patches that were needed for
building graphviz-2.40.1 with the Intel compiler are not needed for
graphviz-2.42.2.
* Adjust the qt dependencies
The qt5 patch is only needed for graphviz-2.40.1. However, that version
will only compile with GCC-6 or greater.
* add variant for enabling testing
* add variant for enabling testing
* enable tests and clean up other options
* enable tests and clean up other options
* add numbered versions
* add numbered versions
* updates to avoid enable_tests variant; correct versioning
* updates to avoid enable_tests variant; correct versioning
* fixes for style
* appropriate partitioners are enabled if 'all' is specified - so no need to check in spec
* revert accidental change to copyright
* remove erroneously re-introduced line about tests
* new spack recipe for build Jali - unstructured mesh infrastructure for multiphysics applications
* remove the +parallel condition for mstk, update 1.1.1 sha256sum and whitespace cleanup
* reformat description
* cut down description
* Fixes:
1. MPI_THREAD_MULTIPLE problem with OpenMPI and UCX.
Changes:
1. OpenMPI provides two new depends_on options which result in UCX being compiled with multiple threads support. One implicit when OpenMPI 3.x is used, MPI_THREAD_MULTIPLE is enabled by default, and one explicit for OpenMPI <= 2.x, MPI_THREAD_MULTIPLE is disabled by default.
2. Extends UCX package to allow "Enable thread support in UCP and UCT" option.
3. Adds sha256 sums of UCX releases 1.6.1 and 1.2.0.
More details:
Fixes the issue with OpenMPI where programs which use MPI_THREAD_MULTIPLE will fail to execute because UCP worker didn't support it.
During the OpenMPI package installation it's the +thread_multiple spec was not propagated to UCX nor UCX handled it at all.
Now, the OpenMPI package is capable of handling +thread_multiple spec when UCX is request and the UCX package correctly handles +thread_multiple and compiles with the --enable-mt option.
Error message during runtime:
pml_ucx.c:226 Error: UCP worker does not support MPI_THREAD_MULTIPLE
* Adapts check of specs to read better and is the suggested form in the docs.
* Explicitly disables multithreading of UCX if +thread_multiple option is not used.
* Rework texlive package to install from source
This PR reworks the texlive package so that it installs from versioned
source distibution files. This is preferred over installing the binary
package for several reasons. For the binary installation:
1. Each component is downloaded, so can not use a spack mirror.
2. Changes in components over time are not reflected in spack hash.
3. Some of the binaries do not run due to glibc issues, depending on OS.
This PR keeps the binary installation as an option but it should be
considered deprecated, and probably rewmoved at some point.
This PR depends on zziplib from PR #14318.
* Fix flake8 issues
One of the perl scripts was encoded with ISO-8859-1, which caused the
sbang replacement process to fail when spack uses python3. This PR
converts the ps_scan script to UTF-8 encoding.
This PR converts ISO-8859 encoding to UTF-8 encoding for three scripts in
repeatmasker.
- the main RepeatMasker script
- SimpleBatcher.pm
- wublastToCrossmatch.pl
The ISO-8859 encoding prevented the sbang replacement of long paths when
spack uses python3.
* Update the icu4c package
This PR makes several changes to the icu4c package
1. add updated version to 65.1
2. modify the default url as project has moved to github
3. set UTF-8 locale to support building from source files in UTF-8
format
Note that the older versions are not available on github so explicit urls
were used. This PR will close#14399.
* Consolidate the urls
Consolidate the URLs in the `version` directives by using an if test in
`url_for_version`.
* Put version and sha256 on same line
* Put top level url back
* Update and fix bcftools package
This PR updates bcftools to 1.10.2 and is dependent on PR #14504.
This PR also fixes builds of other versions. Versions 1.2-1.4 did not
use autotools so when the packaeg was converted to use autotools with
version 1.6 those older versions could no longer build. Also, those
versions needed to be patched to use an external htsllib. The method of
finding the external htslib is also different for those older versions.
In addition, this PR adds two variants to bcftools:
- libgsl
- perl-filters
Finally, dependencies for perl and python are added, and an unused
dependency for libzip was removed.
* Do not use '@' in variant description
The '@' character in a variant description will cause a problem with
`spack info`.
```
==> Error: Incomplete color format: '@' in
expressions, for @1.8:
```
* Fix error with python2 processing this package
* Update htslib and add libcurl variant
This PR updates htslib to version 1.10.2 and adds a libcurl variant. The
libcurl variant defaults to True because, while it is optional, it is
highly recommended by the project developers.
Other things done:
- be consistent with quotes
* Change version in variant description
Apparently, `spack info` does not like the `@` character in a variant
description.
```
==> Error: Incomplete color format: '@' in
@1.3:.
```
* libcircle: add develop version from git master branch
* Update var/spack/repos/builtin/packages/libcircle/package.py
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* libcircle: flake8 fix i think
* libcircle: naming things
* libcircle: 🐑 my sacrifice to the flake8 gods
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Add 20181226 release of pgmath
* 20190329 release of pgmath requires match for CMPLX macro.
* Add llvm-flang package for Flang fork of LLVM
* Add new and old flang releases.
* Add cmake and python dependencies.
* Update dependencies on llvm-flang and pgmath.
* Fix cmake args and change spec to reflect llvm-flang package.
* change copyright date through 2020.
* Reference Flang package more explicitly.
* More robust support for python executable.
* import os no longer needed, picked up by flake8.
* Use built-in target spec. Variant and targets follow style in main LLVM package.
* Get rid of targets list and only support one target for now.
* Sparc does not appear to be supported in Flang.
* Raise InstallError if architecture not supported.
* fixes#967
* Version bump to 0.9.1
- Bugfixes for spack find
- 0.9.1 can read specs from current develop.
* Don't assume spack is in the path when building docs.
* Quick fix for relocation issues.
* elf relocation fix: cherry-picked from develop branch (#6889)
* Revert "Quick fix for relocation issues."
This reverts commit 57608a6dc4.
* Buildcache: relocate fixes (#6512)
* Updated function which checks if a binary file needs relocation.
Previously this was incorrectly identifying ELF binaries as symbolic
links (so they were being excluded from relocation). Added test to
check that ELF binaries are not considered symlinks.
* relocate_text was not replacing paths in text files. Added test to
check that text files are relocated properly (i.e. paths in the file
are converted to the new prefix).
* Exclude backup files created by filter_file when installing from
binary cache.
* Update write_buildinfo_file method signature to distinguish between
the spec prefix and the working directory for the binary cache
package.
* Final changes for v0.11.0 (#6318)
* Fix logo link in README.md to point to the develop branch. (#6969)
* Compiler flag handlers (#6415)
This adds the ability for packages to apply compiler flags in one of
three ways: by injecting them into the compiler wrapper calls (the
default in this PR and previously the only automated choice);
exporting environment variable definitions for variables with
corresponding names (e.g. CPPFLAGS=...); providing them as arguments
to the build system (e.g. configure).
When applying compiler flags using build system arguments, a package
must implement the 'flags_to_build_system_args" function. This is
provided for CMake and autotools packages, so for packages which
subclass those build systems, they need only update their flag
handler method specify which compiler flags should be specified as
arguments to the build system.
Convenience methods are provided to specify that all flags be applied
in one of the 3 available ways, so a custom implementation is only
required if more than one method of applying compiler flags is
needed.
This also removes redundant build system definitions from tutorial
examples
* Fix type issues with setting flag handlers (#6960)
The flag_handlers method was being set as a bound method, but when
reset in the package.py file it was being set as an unbound method
(all python2 issues). This gets the underlying function information,
which is the same in either case.
The bug was uncovered for parmetis in #6858. This is a partial fix.
Included are changes to the parmetis package.py file to make use of
flag_handlers.
* Bump version to 0.11.1
* Added flags to unit tests + OSX build done once per day (#6988)
* Adding flags to codecov reports
* OSX builds are triggered once a day
* Pull R list_urls from upstream.
* travis: removed /usr/local/include/c++ before installing gcc on OSX (#6515) (#7027)
"brew install gcc" fails for travis build because of an existing
/usr/local/include/c++. This commit removes the offending file
as suggested by brew.
* Fix gfortran 7 detection (#7017)
* Add NameError to exceptions caught from configure_args in module generation (#7173)
* Revert "Binary caching: remove symlinks, copy files instead (#9747)"
This reverts commit 058cf81312.
* Make Spack relocate text files in build caches with relative binaries
* add the tfel package
* fix the tfel package
* fix the tfel package
* fix the tfel package
* Taking Adam J. Steward' remarks into account
* fixes trailing white spaces
* Update description
* Update dependencies following @adamjstewart adices
* Style fixes
* Style fixes
* Add java optional support
* add the maintainers attribute (following @alalazo advice), disable interface not selected (following @adamjstewart advice)
* flake8 fixes
* Fix Cast3M and python-bindings support. Python detection is made compatible with cmake'FindPythonLibs module (at least how it is used in TFEL)
* Style fixes
* Style fixes
* Fix test on python version
* Follow @adamjstewart advices: code is much cleaner and readable
* Small fix
* Small fix
* Add comment
* Small fix in cmake option
* try again (trying to overcome Travis CI unstable build process)
* Add support for the MFrontGenericInterfaceSupport project (MGIS)
* Style fixes
* Package documentation update
* Package documentation update
* Fix a typo thanks to Andreas Baumbach review
* Follow Adam J. Stewart advices
* Fix type
* bugfix: add back r's for invalid regexes
* tutorial basics section: fix gcc install version
* version bump: v0.12.1
* bugfix: bring in .travis.yml from develop
* Add new TFEL' versions (3.0.4, 3.1.4 and 3.2.1). Add new MGIS version (1.0.1). Fix MGIS dependency
* merge with spack:develop
* add missing dependency
* new versions of and
* Fix MGIS url. Fix duplicate variant in TFEL
* Fix tfel packaging according to Adam J. Stewart' advices
* Fix flake8 warning
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
Co-authored-by: Peter Scheibel <scheibel1@llnl.gov>
Co-authored-by: Greg Becker <becker33@llnl.gov>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Add ADF
* Fix typo and lint
* fix lint again
* one more lint fix
* fix identation
* still stying to fix identation
* one final fix
* import needed libraries
* changes as per reviewer's request
fix setup environment function, enhance recipe
* add import os once again
* chnages as per reviewer's request
* Set conflicts for qt5 and the Intel compiler
This PR sets a `conflicts` statement for QT5 and the Intel compiler.
* New patches for intel compiles
This commit adds two patches to get QT5 to compile with the intel
compilers. The two patches are very similar but the file being patched
was changed substantially between qt-5.11 and qt-5.12. The patch checks
versions of both GCC and Intel compilers to know when to use overflow
builtis. Essentially, GCC must be >= 5 and Intel must be >= 18.
The sqlite dependency needs the `+column_metadata` variant when the
Intel compiler is used. That is made conditional on the compiler but it
might make sense to make that the default for the sqlite dependency.
Some other changes were made based on testing builds of various QT5
versions with several Intel compilers.
- The libxext dependency is still needed for QT5
- A dependency on libxrender is needed
- The gtk option format needs to be constrained at the qt@5.7 level, not
qt@5.8.
- An extra configure option is needed for the sql plugins RPATH
* Adding a new package, scikit-build, which is very useful for building python extensions
* Update package.py
* Update package.py
Trying to address flake8 corrections
* Update var/spack/repos/builtin/packages/py-scikit-build/package.py
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/py-scikit-build/package.py
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* Update package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-rapidjson: new package at 0.9.1
* py-rapidjson: rename to py-python-rapidjson, use PyPI mirror
* py-python-rapidjson: add missing deps
* python-rapidjson: use short PyPI url
* py-python-rapidjson: remove extra dependencies
* Add platform flag to QT for linux+clang
* Extend QT platform support to more compilers and systems
* Unify QT5 configure options
* fixup! Unify QT5 configure options
* fixup! Unify QT5 configure options
* fixup! Unify QT5 configure options
* Fix newer flake8 and mac qt5 configure
* Add Thirdorder recipe
* Remove white spaces
* Converting recipe to a PythonPackage base class
* remove trailing spaces
* remove line at end of file
* enhance recipe as per reviewer
* fix post_install as requested by reviewer
* rename dir to py-thirderorder
* change checksum to sha256
* py-intervaltree: new package at 3.0.2
* py-intervaltree: fix checksum
* py-intervaltree: add py-setuptools dep
* py-intervaltree: use inclusive ranges
* py-intervaltree: change py-test dep type
Beginning with numpy > 1.16 when using older versions of gcc the
`std=c99` flag must be used. The Intel compiler depends on gcc for its
language extensions so the version of gcc is important. If the version
of gcc used by the Intel compiler is one that requires the `-std=c99`
flag then that flag will have to be used for a build with the Intel
compiler as well.
This PR tests the version of gcc used by the Intel compiler and will
abort the build if the gcc version is < 4.8 and inject the `-std=c99`
flag if >= 4.8 and < 5.1. This will cover the system gcc compiler and
any gcc environment module loaded at build time.
Due to formatting differences, the older version of perl-bioperl was
getting picked up as the preferred version. This PR explicitly sets the
newer version to be preferred.
Because of a bug in the current concretizer,
spack install gromacs
fails because gromacs depends on hwloc (default is v2), and Open MPI
(the default MPI library) depends on hwloc v1.
As discussed in https://github.com/spack/spack/issues/14339, this
workaround should be removed once the concretizer is fixed
Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>
* create package py-zarr
* specify setuptools versions
* add more dependencies, improve style
* Update var/spack/repos/builtin/packages/py-zarr/package.py
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/py-zarr/package.py
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/py-zarr/package.py
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* add dependencies, remove python version constraint
* remove windows specific dependency
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Updated versions and more variants
- Added 'develop' and '3.0.0' versions
- Added 'tau', 'upcxx', 'gotcha', and 'likwid'
* Added conflict handling for +cupti~cuda
* Removed extra cmake args line
* Add Avizo Recipe
* make changes as per review
* fix home url and linting
* Fix url
* fix identation
* change checksum to sha256 instead of md5
* fix installation
* fix lint
* fix identation
* make it compatible with python 2.6
* enhancing recipe and fixing avizo licensing
changes as per suggestions from reviewer; fix licensing
* fix identation
* use new setup_run_environment function
Users can now list mirrors of the main url in packages.
- [x] Instead of just a single `url` attribute, users can provide a list (`urls`) in the package, and these will be tried by in order by the fetch strategy.
- [x] To handle one of the most common mirror cases, define a `GNUMirrorPackage` mixin to handle all the standard GNU mirrors. GNU packages can set `gnu_mirror_path` to define the path within a mirror, and the mixin handles setting up all the requisite GNU mirror URLs.
- [x] update all GNU packages in `builtin` to use the `GNUMirrorPackage` mixin.
* Add symbols patch
* Apply symbols patch to pgmath
* Add github issue number for symbols patch.
* Add naromero77 as a maintainer.
* Patch only applied to March 2019 release and master.
* Record that old versions of ROOT don't support modern GCC
* Well, actually I don't know about 6.07
* Fix typo and follow odd version recommendation from @chissg
* Add QE 6.5
* Support for serial HDF5 case with serial (no mpi) QE is now supported but requires a patch for 6.4.1 and 6.5.
* Add naromero77 as a maintainer.
* Start cinema package
* Remove boilerplate and add description
* Formatting for pep8
* Correct milestone tag
* 'master' instead of 'develop'
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* Two variants, both with numpy and other small changes
* When +image for scikit
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
- Add an optional argument so that `possible_dependencies()` will report
missing dependencies.
- Add a test to ensure it works.
- Ignore missing dependencies in `possible_dependencies()` by default.
* Add dependencies for hpcrypt
* address review comments
* flake
* license-fix
* fix checksums
* Update py-hvace homepage
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* update py-hvac url
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* add new package : filebench
* remove alpha version and duplicated autoheader cmd
* refine automake cmd in sh()
* refine filebench url as a stable tarball link
* root: Rationalize and improve version, variant and ROOT option handling.
* Completely re-vamp CMake option handling for readability and maintainability:
* Three categories of option: control, builtin and feature, alphabetically sorted.
* Each option is described as a list: an option name followed by an optional value which is either Boolean or a string representing the name of a variant. If the value is omitted, it defaults to the option name.
* New functions `_process_opts()` and `_process_opt()` (nested) to turn all supplied option/value specifications into CMake arguments.
* Remove overly-terse per-option comments in favor of (much) more comprehensive notes in README.md.
* Variants and conflicts:
* Remove `test` variant in favor of pegging ROOT `testing` option to the value of `self.run_tests` since the install is unaffected, per ROOT developer.
* Remove commented-out and never-functional variants: `asimage`, `avahi`, `kerberos`, `ldap`, `libcxx`, `odbc`, `oracle`, `pythia8`, `xinetd`.
* New variant `vmc` (default `OFF`) to control the Virtual Monte Carlo interface.
* Conflict: `+opengl` is incompatible with `~x`.
* Conflict: `http` is now an unconditional conflict due to dependency issues (see README.md).
* Remove commented-out and non-existent dependencies `avahi`, `kerberos`, `ldap`, `libcxx`, `odbc`, `oracle`, `pythia`, `veccore` (per #13949).
* New and changed options:
* Option `pch` was inadvertently set to `OFF` due to its dependence on a nonexistent variant `pch`. As it happens its value is ignored in the ROOT configuration handling, so there was no deleterious effect. It has been fixed to `ON` to better reflect actual behavior pending enablement of tuntime C++ modules.
* Add new versions 6.18.0{0,2,4}:
* Require CMake 3.9 for 6.18.00+.
* Add conflicts for variants `qt4` and `table` representing ROOT build options for which support was discontinued. Remove redundant conflict on \@master.
* C++ standard is now specified with `-DCMAKE_CXX_STANDARD=X` rather than `-Dcxx=X`.
* Remove old version 5.34.38 (wrong build system).
See README.md for more details of option-related changes.
* Flake8
* `rpath` option is a control option rather than a feature.
* Add new DD4hep release and some forgotten build requirements
* PR review suggestions
Use master naming convention for development branch, and put versions in decreasing order.
* rose: Update boost dependency for rose
* rose: Updated rose to version 0.9.12.45
* rose: Updated jdk dependency
* rose: Updated rose to version 0.9.13.0
* rose: Fixed formatting
* rose: Added maintainer and switch dependency to java@8
* Added package for Half C++ header-only library.
Fixed an checksum for Hydrogen 1.3.2. Cleaned up the Clara package to
not create an empty bin directory.
* Fixed flake8
* Added maintainer
* add new package : cosbench
* add cosbench depends and remove unstable version
* Update var/spack/repos/builtin/packages/cosbench/package.py
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/cosbench/package.py
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* when constructing package hash, default to including a method in the content hash if we can't determine whether it would be included by examining the AST
* add a test for updated content-hash calculations
* refactor content hash tests to eliminate repeated lines
