PowerShell requires explicit shell and env support in Spack.
This is due to the distinct differences in shell interactions between
cmd and pwsh. Add a doskey in pwsh piping 'spack' commands to a
powershell script similar to the sh function 'spack'. Add
support for PowerShell-specific shell interactions from Spack
(set/unset shell variables).
* WarpX 23.06
Update WarpX and related Python packages to the lastest releases.
WarpX 23.06 introduces multi-dimension support in a single package,
which will ease deployment in E4S et al. that can ship now a single,
full-feature module/package that is NOT incompatible with itself
anymore.
* e4s ci stacks: multiple specs for each dim variant no longer required
* [@spackbot] updating style on behalf of ax3l
* WarpX: Update CMake CLI and Test/Check
* Add Missing `build-directory`
* [@spackbot] updating style on behalf of ax3l
* Remove `build_directory` again
---------
Co-authored-by: eugeneswalker <eugenesunsetwalker@gmail.com>
Co-authored-by: ax3l <ax3l@users.noreply.github.com>
Update Tcl modulefile template to use the `depends-on` command to
autoload modules if Lmod is the current module tool.
Autoloading modules with `module load` command in Tcl modulefile does
not work well for Lmod at some extend. An attempt to unload then load
designated module is performed each time such command is encountered. It
may lead to a load storm that may not end correctly with large number of
module dependencies.
`depends-on` command should be used for Lmod instead of `module load`,
as it checks if module is already loaded, and does not attempt to reload
this module.
Lua modulefile template already uses `depends_on` command to autoload
dependencies. Thus it is already considered that to use Lmod with Spack,
it must support `depends_on` command (version 7.6+).
Environment Modules copes well with `module load` command to autoload
dependencies (version 3.2+). `depends-on` command is supported starting
version 5.1 (as an alias of `prereq-all` command) which was relased last
year.
This change introduces a test to determine if current module tool that
evaluates modulefile is Lmod. If so, autoload dependencies are defined
with `depends-on` command. Otherwise `module load` command is used.
Test is based on `LMOD_VERSION_MAJOR` environment variable, which is set
by Lmod starting version 5.1.
Fixes#36764
Update modulefile templates to append a trailing delimiter to MANPATH
environment variable, if the modulefile sets it.
With a trailing delimiter at ends of MANPATH's value, man will search
the system man pages after searching the specific paths set.
Using append-path/append_path to add this element, the module tool
ensures it is appended only once. When modulefile is unloaded, the
number of append attempt is decreased, thus the trailing delimiter is
removed only if this number equals 0.
Disclaimer: no path element should be appended to MANPATH by generated
modulefiles. It should always be prepended to ensure this variable's
value ends with the trailing delimiter.
Fixes#11355.
The pcluster image has am internal buildcache without an index.
Also, we need to force reuse to avoid rebuilding GCC, since the default is
to only reuse dependencies - and that is subject to changes in the GCC
recipe.
* e4s oneapi ci: use official intel oneapi-derived runner image
* update oneapi image
* tau builds ok, but only with libdrm - comment out for now, follow up with pr later
* e4s cray ci stack
* e4s ci: add cray
* add zen4 tag
* WIP: new defintions just for cray
* updates
* remove ci signing job overrride, not necessary
* echo $PATH and show modules loaded
* add mirror
* add external def for cray-libsci
* comment out quantum-espresso
* use /etc/protected-runner as key path
* cray ci stack: do not remove tags: [spack, public]
* make cray stack composable
* generate job should run on public tagged runner, override default config:install_tree:root
* CI: Use relative path in default script
* CI: Use relative includes paths for shell runners
* Use concrete_env_dir for relpath
* ml-darwin-aarch64-mps: jax has bazel codesign issue
---------
Co-authored-by: Scott Wittenburg <scott.wittenburg@kitware.com>
Co-authored-by: Ryan Krattiger <ryan.krattiger@kitware.com>
* llvm: replace +omp_as_runtime with omp=runtime
* llvm: fetch 'libomp-libflags-as-list.patch' from upstream repo
* llvm: replace 'llvm14-hwloc-ompd.patch' with the official fix from upstream repo
* llvm: fix-up for the black reformatting
* llvm: fetch 'constexpr_longdouble.patch' from upstream repo
* llvm: optionally build libcxx as a runtime
* llvm: fetch 'llvm5-sanitizer-ustat.patch' from upstream repo
* llvm: update 'sanitizer-ipc_perm_mode.patch'
* llvm: refactor compiler conflicts when libcxx=project
* llvm: fetch 'llvm_python_path.patch' from upstream repo
* llvm: update comments and condition for 'xray_buffer_queue-cstddef.patch'
* llvm: optionally build compiler-rt as a runtime
* llvm: fetch 'lldb_external_ncurses-10.patch' from upstream repo
* llvm: fetch 'llvm_py37.patch' from upstream repo
* llvm: rename variant 'internal_unwind' to 'libunwind'
* llvm: optionally build libunwind as a runtime
* llvm: extend the list of maintainers
* llvm: allow for explicit '~clang~flang~libomptarget~lldb~omp_debug~z3'
* llvm: fetch 'llvm5-lld-ELF-Symbols.patch' from FreeBSD port repo
* llvm: fetch most of 'missing-includes.patch' from upstream repo and reuse 'llvm-gcc11.patch'
* llvm: regroup patches for missing include directives and drop compiler constraints for them
* llvm: fetch 'llvm-gcc11.patch' from upstream repo
* llvm: fetch 'no_cyclades.patch' from upstream repo
* llvm: update comments and condition for 'no_cyclades9.patch'
* llvm: rename variant 'omp' to 'openmp'
* llvm: constrain and rename variant 'omp_tsan' to 'libomp_tsan'
* llvm: rename variant 'omp_debug' to 'libomptarget_debug'
* llvm: do not apply same patch twice
* llvm: constrain and document the '*-thread.patch' patches
* llvm: document the '~lld+libomptarget' conflict
* llvm: update comments for the 'D133513.diff' patch
fa7719a changed syntax for specifying exact versions, which are
required for some compiler specs (including those read as part
of parsing a Cray manifest). This fixes that and also makes a
couple other improvements to manifest parsing.
* Instantiate compiler specs with exact versions (fixes#37893)
* fix slingshot network detection (CPE 22.10+ has libcxi.so
in /usr/lib64)
* "spack external find": add arg to ignore default dir for cray
manifests
* Add macOS ML CI stacks
* torchmeta is no longer maintained and requires ancient PyTorch
* Add MXNet
* update darwin aarch64 stacks
* add darwin-aarch64 scoped config.yaml
* remove unnecessary cleanup job
* fix specifications
* fix labels
* fix labels
* fix indent on tags specification
* no tags for trigger jobs
* try overriding tags in stack spack.yaml
* do not use CI_STACK_CONFIG_SCOPES
* incorporate config:install_tree:root: overrides and compiler defs
* copy relevant ci-scoped config settings directly into stack spack.yaml
* remove build-job-remove
* spack ci generate: add debug flag
* include cdash config directly in stack spack.yaml
* customize build-job script section to avoid absolute paths
* add any-job specification
* tags: use aarch64-macos instead of aarch64
* generate tags: use aarch64-macos instead of aarch64
* do not add morepadding
* use shared mirror; comment out known failures
* remove any-job
* nproc || true
* comment out specs failing due to bazel from cache codesign issue
---------
Co-authored-by: eugeneswalker <eugenesunsetwalker@gmail.com>
* [pcluster pipeline] Use local buildcache instead of upstream spack
Spack currently does not relocate compiler references from upstream spack
installations. When using a buildcache we don't need an upstream spack.
* gcc needs to be installed via postinstall to get correct deps
* quantum-espresso@gcc@12.3.0 returns ICE on neoverse_{n,v}1
* Force gitlab to pull the new container
* Revert "Force gitlab to pull the new container"
This reverts commit 3af5f4cd88245138992deb2a46c17e6f85858d68.
Seems the gitlab version does not yet support "pull_policy" in .gitlab-ci.yml
* Gitlab keeps picking up wrong container. Renaming
* Update containers once more after failed build
Add aws-plcuster[-aarch64] stacks. These stacks build packages defined in
https://github.com/spack/spack-configs/tree/main/AWS/parallelcluster
They use a custom container from https://github.com/spack/gitlab-runners which
includes necessary ParallelCluster software to link and build as well as an
upstream spack installation with current GCC and dependencies.
Intel and ARM software is installed and used during the build stage but removed
from the buildcache before the signing stage.
Files `configs/linux/{arch}/ci.yaml` select the necessary providers in order to
build for specific architectures (icelake, skylake, neoverse_{n,v}1).
* CI: Expand E4S ROCm stack to include missing DaV packages
Ascent: Fixup for VTK-m with Kokkos backend
* DaV SDK: Removed duplicated openmp variant for ascent
* Drop visit and add conflict for Kokkos
* E4S: Drop ascent from CUDA builds
Ensure that requirements `packages:*:require:@x` and preferences `packages:*:version:[x]`
fail concretization when no version defined in the package satisfies `x`. This always holds
except for git versions -- they are defined on the fly.
* gitlab ci: release fixes and improvements
- use rules to reduce boilerplate in .gitlab-ci.yml
- support copy-only pipeline jobs
- make pipelines for release branches rebuild everything
- make pipelines for protected tags copy-only
* gitlab ci: remove url changes used in testing
* gitlab ci: tag mirrors need public key
Make sure that mirrors associated with release branches and tags
contain the public key needed to verify the signed binaries. This
also ensures that when stack-specific mirror contents are copied
to the root, the root mirror has the public key as well.
* review: be more specific about tags, curl flags
* Make the check in ci.yaml consistent with the .gitlab-ci.yml
---------
Co-authored-by: Ryan Krattiger <ryan.krattiger@kitware.com>
The flags --mirror-name / --mirror-url / --directory were deprecated in
favor of just passing a positional name, url or directory, and letting spack
figure it out.
---------
Co-authored-by: Scott Wittenburg <scott.wittenburg@kitware.com>
`spack buildcache create` is a misnomer cause it's the only way to push to
an existing buildcache (and it in fact calls binary_distribution.push).
Also we have `spack buildcache update-index` but for create the flag is
`--rebuild-index`, which is confusing (and also... why "rebuild"
something if the command is "create" in the first place, that implies it
wasn't there to begin with).
So, after this PR, you can use either
```
spack buildcache create --rebuild-index
```
or
```
spack buildcache push --update-index
```
Also, alias `spack buildcache rebuild-index` to `spack buildcache
update-index`.
Paths with spaces are an issue on Windows and our current powershell
scripts are not sufficiently hardended against their use.
This PR removes promlematic commandlets that do not work well with paths
with spaces and adds escape quotes in other areas where this could be an
issue.
* DaV SDK: Enable ParaView raytracing with in SDK
* CI: Drop swr testing from Data Vis SDK
* ISPC: extend LLVM requirement to main
* DaV SDK: Disallow concretizing develop unifyfs
No longer needed after mochi-margo patch
