Commit graph

34618 commits

Author SHA1 Message Date
Vanessasaurus
b1ab01280a
Automated deployment to update package flux-sched 2023-12-14 (#41658)
Co-authored-by: github-actions <github-actions@users.noreply.github.com>
2023-12-14 12:52:31 -08:00
Arne Becker
ab84876e2c
perl-proc-daemon: New package (#41666) 2023-12-14 12:51:43 -08:00
Arne Becker
e2d5be83e7
perl-test-base: New package (#41668)
Adds Test::Base
2023-12-14 12:49:43 -08:00
Arne Becker
85cdf37d3b
perl-datetime-format-strptime: New package (#41676)
* perl-datetime-format-strptime: New package
  Adds package:
  - perl-datetime-format-strptime
  And adds these because they are test dependencies:
  - perl-test-file-sharedir
  - perl-test2-plugin-nowarnings
  - perl-test2-suite
  And modifies these to enable build time tests:
  - perl-b-hooks-endofscope
  - perl-class-singleton
  - perl-datetime-locale
  - perl-datetime-timezone
  - perl-file-sharedir
  - perl-namespace-autoclean
  - perl-namespace-clean
  - perl-params-validationcompiler
  - perl-specio
* Add myself as maintainer
2023-12-14 12:47:40 -08:00
Arne Becker
06521b44b6
perl-common-sense: New package (#41677) 2023-12-14 12:20:47 -08:00
Arne Becker
1e5325eea0
perl-time-clock: New package (#41678) 2023-12-14 12:19:06 -08:00
Arne Becker
0995a29c5c
perl-sql-reservedwords: New package (#41685) 2023-12-14 12:07:12 -08:00
Adam J. Stewart
133d6e2656
py-pandas: add v2.1.4 (#41590) 2023-12-14 13:53:05 -06:00
Adam J. Stewart
36117444aa
py-black: add v23.12.0 (#41589) 2023-12-14 13:52:47 -06:00
Andrey Perestoronin
330a9a7c9a
intel-oneapi-compilers 2023.2.3: added new version to dpcpp package (#41680)
* add new cpp compiler version

* empty ftn for 2023.2.3

* OLD ftn in 2023.2.3 version

* tolerate missing fortran compiler

---------

Co-authored-by: Robert Cohn <robert.s.cohn@intel.com>
2023-12-14 11:03:18 -07:00
Todd Gamblin
0dc3fc2d21
spec parser: precompile quoting-related regular expressions (#41657)
This adds a small (~5%) performance improvement to Spec parsing.

Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2023-12-14 05:08:08 -07:00
Dom Heinzeller
a972314fa6
Fix spack compiler wrappers in ESMF's esmf.mk on Cray when using cc, CC, ftn (#41640) 2023-12-14 09:42:34 +01:00
Wouter Deconinck
16d1ed3591
geant4: new version 11.2.0 (#41643)
* geant4: new version 11.2.0

* geant4: depends_on geant4-data@11.2:

* geant4-data: new version 11.2.0

* g4abla: new version 3.3

* g4emlow: new version 8.4

* g4incl: new version 1.1

* geant4: depends_on vecgeom@1.2.6:

* geant4: depends_on qt@5.9: when @11.2: +qt

* vecgeom: new version 1.2.6
2023-12-14 09:18:04 +01:00
Arne Becker
5c25f16df2
perl-cpanel-json-xs: New package (#41646) 2023-12-13 18:13:58 -08:00
Arne Becker
b3ccaa81a7
perl-mock-config: New package (#41647) 2023-12-13 18:13:04 -08:00
Arne Becker
a0041731a3
perl-ref-util: New package (#41648) 2023-12-13 18:12:02 -08:00
Todd Gamblin
a690b8c27c
Improve parsing of quoted flags and variants in specs (#41529)
This PR does several things:

- [x] Allow any character to appear in the quoted values of variants and flags.
- [x] Allow easier passing of quoted flags on the command line, e.g. `cflags="-O2 -g"`.
- [x] Handle quoting better in spec output, using single quotes around double 
      quotes and vice versa.
- [x] Disallow spaces around `=` and `==` when parsing variants and flags.

## Motivation

This PR is motivated by the issues above and by ORNL's 
[tips for launching at scale on Frontier](https://docs.olcf.ornl.gov/systems/frontier_user_guide.html#tips-for-launching-at-scale).
ORNL recommends using `sbcast --send-libs` to broadcast executables and their
libraries to compute nodes when running large jobs (e.g., 80k ranks). For an
executable named `exe`, `sbcast --send-libs` stores the needed libraries in a
directory alongside the executable called `exe_libs`. ORNL recommends pointing
`LD_LIBRARY_PATH` at that directory so that `exe` will find the local libraries and
not overwhelm the filesystem.

There are other ways to mitigate this problem:
* You could build with `RUNPATH` using `spack config add config:shared_linking:type:runpath`,
  which would make `LD_LIBRARY_PATH` take precedence over Spack's `RUNPATHs`.
  I don't recommend this one because `RUNPATH` can cause many other things to go wrong.
* You could use `spack config add config:shared_linking:bind:true`, added in #31948, which
  will greatly reduce the filesystem load for large jobs by pointing `DT_NEEDED` entries in
  ELF *directly* at the needed `.so` files instead of relying on `RPATH` search via soname.
  I have not experimented with this at 80,000 ranks, but it should help quite a bit.
* You could use [Spindle](https://github.com/hpc/Spindle) (as LLNL does on its machines)
  which should transparently fix this without any changes to your executable and without
  any need to use `sbcast` or other tools.

But we want to support the `sbcast` use case as well.

## `sbcast` and Spack

Spack's `RPATHs` break the `sbcast` fix because they're considered with higher precedence
than `LD_LIBRARY_PATH`. So Spack applications will still end up hitting the shared filesystem
when searching for libraries. We can avoid this by injecting some `ldflags` in to the build, e.g.,
if were were going to launch, say, `LAMMPS` at scale, we could add another `RPATH`
specifically for use with `sbcast`:

    spack install lammps ldflags='-Wl,-rpath=$ORIGIN/lmp_libs'

This will put the `lmp_libs` directory alongside `LAMMPS`'s `lmp` executable first in the
`RPATH`, so it will be searched before any directories on the shared filesystem.

## Issues with quoting

Before this PR, the command above would've errored out for two reasons:

1. `$` wasn't an allowed character in our spec parser.
2. You would've had to double quote the flags to get them to pass through correctly:

       spack install lammps ldflags='"-Wl,-rpath=$ORIGIN/lmp_libs"'

This is ugly and I don't think many users will easily figure it out. The behavior was added in
#29282, and it improved parsing of specs passed as a single string, e.g.:

    spack install 'lammps ldflags="-Wl,-rpath=$ORIGIN/lmp_libs"'

but a lot of users are naturally going to try to quote arguments *directly* on the command
line, without quoting their entire spec. #29282 used a heuristic to detect unquoted flags
and warn the user, but the warning could be confusing. In particular, if you wrote
`cflags="-O2 -g"` on the command line, it would break the flags up, warn, and tell you
that you could fix the issue by writing `cflags="-O2 -g"` even though you just wrote
that. It's telling you to *quote* that value, but the user has to know to double quote.

## New heuristic for quoted arguments from the CLI

There are only two places where we allow arbitrary quoted strings in specs: flags and
variant values, so this PR adds a simpler heuristic to the CLI parser: if an argument in
`sys.argv` starts with `name=...`, then we assume the whole argument is quoted.

This means you can write:

    spack install bzip2 cflags="-O2 -g"

directly on the command line, without multiple levels of quoting. This also works:

    spack install 'bzip2 cflags="-O2 -g"'

The only place where this heuristic runs into ambiguity is if you attempt to pass
anonymous specs that start with `name=...` as one large string. e.g., this will be
interpreted as one large flag value:

    spack find 'cflags="-O2 -g" ~bar +baz'

This sets `cflags` to `"-O2 -g" ~bar +baz`, which is likely not what you wanted. You
can fix this easily by either removing the quotes:

    spack find cflags="-O2 -g" ~bar +baz

Or by adding a space at the start, which has the same effect:

    spack find ' cflags="-O2 -g" ~bar +baz'

You may wonder why we don't just look for quotes inside of flag arguments, and the
reason is that you *might* want them there.  If you are passing arguments like:

    spack install zlib cppflags="-D DEBUG_MSG1='quick fox' -D DEBUG_MSG2='lazy dog'"

You *need* the quotes there. So we've opted for one potentially confusing, but easily
fixed outcome vs. limiting what you can put in your quoted strings.

## Quotes in formatted spec output

In addition to being more lenient about characters accepted in quoted strings, this PR fixes
up spec formatting a bit. We now format quoted strings in specs with single quotes, unless
the string has a single quote in it, in which case we JSON-escape the string (i.e., we add
`\` before `"` and `\`).  

    zlib cflags='-D FOO="bar"'
    zlib cflags="-D FOO='bar'"
    zlib cflags="-D FOO='bar' BAR=\"baz\""
2023-12-13 16:36:22 -08:00
yizeyi18
a1fa862c3f
camp: fixing build issue (#41400)
* adding necessary headers, to fix https://github.com/spack/spack/issues/41398

* deleting something imported by accident

* [@spackbot] updating style on behalf of yizeyi18

* undo commit 7688fed according to suggestion from @msimberg

* patching camp@:2022.10.1 for compatibility with gcc-13

* adding the patch

* fixing paths in the patch

* [@spackbot] updating style on behalf of yizeyi18

* Update camp patch using LLNL/camp@05e1c35

Co-authored-by: Mikael Simberg <mikael.simberg@iki.fi>

* changing patch name

---------

Co-authored-by: Mikael Simberg <mikael.simberg@iki.fi>
2023-12-13 16:28:57 -08:00
Harmen Stoppels
80f31829a8
python: don't run mkdirp in setup_dependent_package (#41603)
`setup_dependent_package` is not a build phase, it should just set
globals for a package.

It's called during setup of runtime environment of packages, and there
have been reports of it actually failing due to a read only file system
(not sure under what exact conditions that is possible).
2023-12-13 23:54:28 +01:00
Arne Becker
84436f10ba
perl-config-tiny: New package (#41584) 2023-12-13 14:43:03 -07:00
Arne Becker
660485709d
perl-b-cow: New package (#41596) 2023-12-13 12:06:14 -08:00
Arne Becker
251dce05c9
perl-throwable: New package (#41597) 2023-12-13 12:05:17 -08:00
Arne Becker
ecd05fdfb4
perl-scope-guard: New package (#41598) 2023-12-13 12:02:50 -08:00
Arne Becker
9ffcf36444
perl-test-sharedfork: New package (#41599) 2023-12-13 12:01:54 -08:00
Arne Becker
07258a7c80
perl-safe-isa: New package (#41600) 2023-12-13 12:01:10 -08:00
Arne Becker
395e53a5e0
perl-proc-processtable: New package (#41601) 2023-12-13 11:59:05 -08:00
Arne Becker
77c331c753
perl-net-ip: New package (#41606) 2023-12-13 11:57:17 -08:00
Arne Becker
ee5481a861
perl-any-uri-escape: New package (#41607) 2023-12-13 11:56:11 -08:00
Arne Becker
f7ec061c64
perl-term-table: New package (#41608) 2023-12-13 11:47:08 -08:00
Arne Becker
7cb70ff4b1
perl-test-pod: New package (#41609) 2023-12-13 11:45:59 -08:00
Arne Becker
4a661f3255
perl-spiffy: New package (#41610) 2023-12-13 11:44:36 -08:00
Arne Becker
7037240879
perl-ipc-system-simple: New package (#41611) 2023-12-13 11:43:21 -08:00
Arne Becker
0e96dfaeef
perl-mime-types: New package (#41612) 2023-12-13 11:41:48 -08:00
Arne Becker
a0a2cd6a1a
perl-convert-nls-date-format: New package (#41613) 2023-12-13 11:39:11 -08:00
Arne Becker
170c05bebb
perl-module-pluggable: New package (#41614) 2023-12-13 11:04:27 -08:00
Arne Becker
bdf68b7ac0
perl-email-date-format: New package (#41617) 2023-12-13 11:03:24 -08:00
Arne Becker
c176de94e2
perl-heap: New package (#41618) 2023-12-13 11:00:55 -08:00
Harmen Stoppels
f63dbbe75d
spack mirror create --all: include patches (#41579) 2023-12-13 20:00:44 +01:00
Arne Becker
a0c7b10c76
perl-log-any: New package (#41619) 2023-12-13 10:59:54 -08:00
Arne Becker
2dc3bf0164
perl-http-cookiejar: New package (#41620) 2023-12-13 10:57:55 -08:00
Harmen Stoppels
9bf6e05d02
Revert "[protobuf] New versions, explicit cxxstd variant (#41459)" (#41635)
This reverts commit b82bd8e6b6.
2023-12-13 15:02:25 +01:00
Massimiliano Culpo
cd283846af
mysql: add v8.0.35, fix build (#41602) 2023-12-13 12:10:33 +01:00
Taillefumier Mathieu
03625c1c95
Add pic variant when building the library (#41631)
* Add pic variant when building the library

* make pretty

* Probably better approach
2023-12-13 01:38:13 -07:00
Wouter Deconinck
f01774f1d4
hepmc3: fix from_variant -> self.define (#41605)
* hepmc3: fix from_variant -> self.define
* hepmc3: str on versions
2023-12-13 07:02:48 +01:00
Christopher Christofi
965860d1f8
perl-getopt-argvfile: add new package with version 1.11 (#41625) 2023-12-12 20:33:25 -07:00
Christopher Christofi
c4baf4e199
perl-parselex: add new package with version 2.21 (#41626) 2023-12-12 20:08:28 -07:00
Aiden Grossman
dd82227ae7
Remove MCT license annotation (#41593)
This license annotation is currently invalid as it specifies a URL
rather than an SPDX expression. Remove it for now until we have a
consensus on how to represent this case.
2023-12-12 17:52:42 -07:00
fpruvost
a9028630a5
pastix: new release v6.3.2 (#41585) 2023-12-12 15:57:11 -07:00
Arne Becker
789c85ed8b
perl-clone-pp: New package (#41586) 2023-12-12 15:46:25 -07:00
James Taliaferro
cf9d36fd64
kakoune: add v2023.08.05 (#41443) 2023-12-12 23:02:20 +01:00