The required hash of a submodule might point to the
non-HEAD commit of the current main branch and hence
would lead to a "no such remote ref" at checkout in
a shallow submodule.
## Motivation
Python installations are both important and unfortunately inconsistent. Depending on the Python version, OS, and the strength of the Earth's magnetic field when it was installed, the name of the Python executable, directory containing its libraries, library names, and the directory containing its headers can vary drastically.
I originally got into this mess with #3274, where I discovered that Boost could not be built with Python 3 because the executable is called `python3` and we were telling it to use `python`. I got deeper into this mess when I started hacking on #3140, where I discovered just how difficult it is to find the location and name of the Python libraries and headers.
Currently, half of the packages that depend on Python and need to know this information jump through hoops to determine the correct information. The other half are hard-coded to use `python`, `spec['python'].prefix.lib`, and `spec['python'].prefix.include`. Obviously, none of these packages would work for Python 3, and there's no reason to duplicate the effort. The Python package itself should contain all of the information necessary to use it properly. This is in line with the recent work by @alalazo and @davydden with respect to `spec['blas'].libs` and friends.
## Prefix
For most packages in Spack, we assume that the installation directory is `spec['python'].prefix`. This generally works for anything installed with Spack, but gets complicated when we include external packages. Python is a commonly used external package (it needs to be installed just to run Spack). If it was installed with Homebrew, `which python` would return `/usr/local/bin/python`, and most users would erroneously assume that `/usr/local` is the installation directory. If you peruse through #2173, you'll immediately see why this is not the case. Homebrew actually installs Python in `/usr/local/Cellar/python/2.7.12_2` and symlinks the executable to `/usr/local/bin/python`. `PYTHONHOME` (and presumably most things that need to know where Python is installed) needs to be set to the actual installation directory, not `/usr/local`.
Normally I would say, "sounds like user error, make sure to use the real installation directory in your `packages.yaml`". But I think we can make a special case for Python. That's what we decided in #2173 anyway. If we change our minds, I would be more than happy to simplify things.
To solve this problem, I created a `spec['python'].home` attribute that works the same way as `spec['python'].prefix` but queries Python to figure out where it was actually installed. @tgamblin Is there any way to overwrite `spec['python'].prefix`? I think it's currently immutable.
## Command
In general, Python 2 comes with both `python` and `python2` commands, while Python 3 only comes with a `python3` command. But this is up to the OS developers. For example, `/usr/bin/python` on Gentoo is actually Python 3. Worse yet, if someone is using an externally installed Python, all 3 commands may exist in the same directory! Here's what I'm thinking:
If the spec is for Python 3, try searching for the `python3` command.
If the spec is for Python 2, try searching for the `python2` command.
If neither are found, try searching for the `python` command.
## Libraries
Spack installs Python libraries in `spec['python'].prefix.lib`. Except on openSUSE 13, where it installs to `spec['python'].prefix.lib64` (see #2295 and #2253). On my CentOS 6 machine, the Python libraries are installed in `/usr/lib64`. Both need to work.
The libraries themselves change name depending on OS and Python version. For Python 2.7 on macOS, I'm seeing:
```
lib/libpython2.7.dylib
```
For Python 3.6 on CentOS 6, I'm seeing:
```
lib/libpython3.so
lib/libpython3.6m.so.1.0
lib/libpython3.6m.so -> lib/libpython3.6m.so.1.0
```
Notice the `m` after the version number. Yeah, that's a thing.
## Headers
In Python 2.7, I'm seeing:
```
include/python2.7/pyconfig.h
```
In Python 3.6, I'm seeing:
```
include/python3.6m/pyconfig.h
```
It looks like all Python 3 installations have this `m`. Tested with Python 3.2 and 3.6 on macOS and CentOS 6
Spack has really nice support for libraries (`find_libraries` and `LibraryList`), but nothing for headers. Fixed.
* Make dia build w/ Spack's X bits (and misc)
X related
- need to depend on the +X variant of gtkplus
- need to depend on freetype
misc
- fix path to tarball
* Make freetype a "build" dependency
* Freetype is not just a build dep
* ncurses package will build ncurses and ncursesw
* Added libs property to ncurses, added fix for hstr
* flake8 is a harsh mistress
* make libs() more robust
* atop depends on ncurses
* fish depends on ncurses
* libtermkey and nano depend on ncurses
* Adjust url spacing
* Added a patch to the openblas package to change the openmp flag for
icc to qopenmp.
* Fixed a linking problem where when using Intel compilers, it was still
pulling in -lgfortran
* depend on readline, remove hardcoded -ltermcap
Bowtie should use Spack's readline and not explicitly depend on the
system termcap (which, on CentOS, leads to linking against the
system's tinfo library).
* Add depends_on('zlib')
* Add conflict with gcc@6:
Build seems to have trouble with 6's migration to -std=gnu++14.
When a compiler was not found a stacktrace was displayed to user because
there were three arguments to be substituted in a string with only two
substitutions to be done.
Fixes#4026#1167 updated Database.reindex to keep old installation records to
support external packages. However, when a user manually removes a
prefix and reindexes this kept the records so the packages were
still installed according to "spack find" etc. This adds a check
for non-external packages to ensure they are properly installed
according to the directory layout.
* Dia requires libxml2.
* Clean up dependencies for Dia (and add X11 deps).
+ Remove dependencies on cairo and libpng. The will be satisfied via gtkplus.
+ Add dependencies on X11 libraries: libsm, libuuid, libxinerama, libxrender.
+ From a dependency diagram, it doesn't appear that we need libxml2 since this
dependency should be come in through cairo (via gtkplus). However, Dia will
not build without it.
* Separate build integration tests; simplify test scripts
- Move build tests out of the regular Travis unit tests, add more smoke
test packages to build.
- Run all test scripts with bash -e, which fails on error.
- Factor coverage out into a Travis environment variable, so it's more
obvious from .travis.yml which tests contribute to coverage and which
don't.
- Factor dependency checking and much of the front-matter in tests
scripts into a setup.sh script, which is sourced by all the test
scripts. Extra cruft in each tests script now reduced to 2 lines at
the beginning.
* Hackery to get gtkplus to build
PR #3077 broke gtkplus by introducing gobject-introspection.
This big hack makes things work. It has problems.
1. Rather than deal with the nasty sbang fooey in the
g-ir-tool-template.in derived scripts, it just adds a python
dependency to each package that runs one of the scripts. This lets
the `/usr/bin/env python` sbang do the right thing.
2. It stuffs a several directories on to the XDG_DATA_DIRS environment
variable, which is used for (among other things) locating the .gir
files.
3. It avoids building the gtkplus demos because I can't make the bit
that calls `gdk-pixbuf-csource` work. It doesn't think that it can
load `.png` files and all of the google hits I found suggest a bad
`loader.cache` file. The file's fine and I can strace the command
and watch it read it in... Many, many hours wasted here.
In spite of the demo failing, the tests pass and an emacs built
with this lib seems to work.
* Fix sbang so everyone needn't depend_on python
Rather than have every package that
`depends_on('gobject-introspection')` also need to
`depend_on('python')`, this commit fixes the
scripts (e.g. `g-ir-scanner`).
The interesting bit is in the gobject-introspection package. There is
a beefy comment there that is included below.
The commit also removes the now un-necessary dependencies from various
packages.
I have two reservations about this commit:
1. How portable is the "insertion" sed command? I'm particularly
worried that some sed's might need the line to insert to be on a
different line, which I can't imagine how to cram into the
Makefile.in.
The solution I see to this is rather than extending the existing
sed command in the Makefile I could shim in another line in the
rule and e.g. call a bit of Perl (or Python, I suppose) which would
end up being much neater.
2. As written it always uses Spack's `.../bin/sbang`, which might or
might not be a good idea.
If I use "the solution" from number 1 above, then I can check the
line length before I munge it. Otherwise???
---
This package creates several scripts from |
toosl/g-ir-tool-template.in. In their original form these |
scripts end up with a sbang line like |
|
`#!/usr/bin/env /path/to/spack/python`. |
|
These scripts are generated and then used as part of the build |
(other packages also use the scripts after they've been |
installed). |
|
The path to the spack python can become too long. Because these |
tools are used as part of the build, the normal hook that fixes |
this problem can't help us. |
This package fixes the problem in two steps: |
- it rewrites the g-ir-tool-template so that its sbang line |
refers directly to spack's python (filter_file step below); and |
- it patches the Makefile.in so that the generated Makefile has an |
extra sed expression in its TOOL_SUBSTITUTION that results in |
an `#!/bin/bash /path/to/spack/bin/sbang` unconditionally being |
inserted into the scripts as they're generated. |
* Cairo needs python when it's +X
Cairo needs to depend_on python when it's +X. I think it's an
indirect requirement that's coming in via libxcb).
* Flake8 cleanup
* Make cairo's dep on python be type=build
This seems to be the right thing and seems to produce a result
that works (I can build gtk+ and then emacs+X on top of it).
* ghostscript: add version 9.21 and handle new URL format
* ghostscript: add url of latest version to fix `spack versions`
* ghostscript: use github for all versions our package provides (>= 9.18)
- add Version.__format__ to support new-style formatting.
- Python3 doesn't handle this well -- it delegates to
object.__format__(), which raises an error for fancy format strings.
- not sure why it doesn't call str(self).__format__ instead, but that's
hwo things are.
