Fixes: #8258#8090 altered import behavior so that import spack no longer
provides access to many other Spack modules. This addresses
a case which depended on the prior behavior and was not
updated as part of #8090. This particular import error only
came up when users were setting compiler flags on specs.
See also: #8194
- there were some leftover spack.* names being used after we removed
globals and moved everything in the top-level namespace to spack.pkgkit
- point those references to their new homes
- remove most `import spack` statements, except for files that need
`spack_version`
- import spack is no longer sufficient to use submodules
(e.g. spack.directives).
- these submodules must be imported directly. Update references
accordingly.
- Spack packages were originally expected to call `from spack import *`
themselves, but it has become difficult to manage imports in the
Spack core.
- the top-level namespace polluted by package symbols, and it's not
possible to avoid circular dependencies and unnecessary module loads in
the core, given all the stuff the packages need.
- This makes the top-level `spack` package essentially empty, save for a
version tuple and a version string, and `from spack import *` is now
essentially a no-op.
- The common routines and directives that packages need are now in
`spack.pkgkit`, and the import system forces packages to automatically
include this so that old packages that call `from spack import *`
will continue to work without modification.
- Since `from spack import *` is no longer required, we could consider
removing ``from spack import *`` from packages in the future and
shifting to ``from spack.pkgkit import *``, but we can wait a while to
do this.
- spack.util.lock behaves the same as llnl.util.lock, but Lock._lock and
Lock._unlock do nothing.
- can be disabled with a control variable.
- configuration options can enable/disable locking:
- `locks` option in spack configuration controls whether Spack will use filesystem locks or not.
- `-l` and `-L` command-line options can force-disable or force-enable locking.
- Spack will check for group- and world-writability before disabling
locks, and it will not allow a group- or world-writable instance to
have locks disabled.
- update documentation
- Spack core has long used llnl.util.filesystem.join_path, but
os.path.join is pretty much the same thing, and is more efficient.
- Use os.path.join in the core Spack code from now on.
- simplify the singleton pattern across the codebase
- reduce lines of code needed for crufty initialization
- reduce functions that need to mess with a global
- Singletons whose semantics changed:
- spack.store.store() -> spack.store
- spack.repo.path() -> spack.repo.path
- spack.config.config() -> spack.config.config
- spack.caches.fetch_cache() -> spack.caches.fetch_cache
- spack.caches.misc_cache() -> spack.caches.misc_cache
- `spack.cmd.all_commands` does a directory listing on
`lib/spack/spack/cmd`, regardless of whether it is needed
- make this lazy so that the directory listing won't happen unless it's
necessary.
- It turns out that jsonschema is one of the more expensive imports.
- move imports of jsonschema into functions to avoid the performance hits
for calls that don't need config.
- spack.store was previously initialized at the spack.store module level,
but this means the store has to be initialized on every spack call.
- this moves the state in spack.store to a singleton so that the store is
only initialized when needed.
- spack.repository module is now spack.repo
- `spack.repo` is now `spack.repo.path()` and loaded lazily
- Added `spack.repo.get()` and `spack.repo.all_package_names()` as
convenience functions to simplify the new lazy interface.
- updated tests and code
- no longer require `spack_version` to be a Version (it isn't used that
way anyway)
- use a simple tuple `spack_version_info` with major, minor, patch
versions
- generate `spack_version` from the tuple
- replace `spack.config.get_configuration()` with `spack.config.config()`
- replace `get_config`/`update_config` with `get`, `set`
- add a path syntax that can be used to refer to specific config options
without firt getting the entire configuration dict
- update usages of `get_config` and `update_config` to use `get` and `set`
- Current configuration code forces the config system to be initialized
at module scope, so configs are parsed on every Spack run, essentially
before anything else.
- We need more control over configuration init order, so move the config
scopes into a class and reduce global state in config.py
Fixes#2781
This PR introduces a new attribute for packages called
`archive_files`, which designates files that should be saved from
a package build (e.g. the config.log generated during autotools
builds).
The attribute contains a list of glob expressions; Any file that
matches will be archived in the `<prefix>/.spack/archived-files`
directory. Errors that occur when archiving files are collected and
reported in a file named `<prefix>/.spack/archived-files/errors.txt`.
`AutotoolsPackage` and `CMakePackage` provide a sensible default
override for this attribute.
fixes#7941
Modified string representation of Specs to add a space before deps
Unit-tests have been modified accordingly
Added a test for regression on #7941
Fixes#7924
Spack's yaml schema validator was initializing all instances of
unspecified properties with the same object, so that updating the
property for one entry was updating it for others (e.g. updating
versions for one package would update it for other packages).
This updates the schema validator to instantiate defaults with
separate object instances and adds a test to confirm this behavior
(and also confirms #7924 without this change).
* Use reported version of IBM XL Fortran compiler for compiler versions >= 16.0.
Starting with the April 2018 release, the IBM XL C and Fortran compilers report the same version, 16.0. Consequently, there is no need to downgrade the Fortran compiler version to match that of the C compiler.
* Use GitLab's API endpoint for fetching a git snapshot.
* More GitLab packages use the API.
* find_list_url for GitLab's API URLs.
* Flake8
* Url for 'hacckernels'.
* Check GitLab API regexps before the non-API ones.
Activating a package that is already activated now sends a `tty.msg`
and returns.
```
-bash-4.2$ ~/spack/bin/spack activate aspell6-en
==> Package aspell6-en/lc4v24f is already activated.
```
* Better error message for spack providers
fixes#1355
`spack providers` now outputs a sensible error message if non-virtual
specs are provided as arguments:
```
$ spack providers mpi zlib petsc
==> Error: non-virtual specs cannot be part of the query [zlib, petsc]
```
Formatting of the output changed slightly.
* Calling 'spack providers' without arguments print the virtual pkg list
Also, the error message in case of a wrong parameter has been improved
to show the list of valid packages.
* Avoid printing headers if stdout is not a tty
* The provider list is formatted with colify if not in a tty
* Added a test to check the list of providers returned from the command
Popping the when spec from kwargs in the extends directive breaks
class inheritance. Inheriting classes do not find their when spec.
We now get the when spec from kwargs instead, leaving it to be found
by any downstream package classes.
fixes#7705
Package.provides now checks constraints to ensure that a spec provides
a given virtual package. Note that 'strict=True' is not passed to
satisfies as this function is also used during concretization.
Fixes#7548
This updates the "spack view" command to use the same parsing logic
as "spack install" on the user-provided specs. For example you can
provide a DAG hash to refer to an exact installed spec instead of
specifying name, compiler, etc.
This fixes a check that decides when to skip buildcache relocation.
Originally the check was flawed in two ways: it would skip if the
source prefix matched the destination prefix, which no longer
matters since the source prefix is replaced with a placeholder
(so it always needs to be updated); it also would skip relocation
if the rpaths were not relative, when in fact it should be the
opposite (binaries without relative rpaths *should* be relocated,
and those without don't need it).
- FastPackageChecker was being called at startup every time Spack runs,
which takes a long time on networked filesystems. Startup was taking
5-7 seconds due to this call.
- The checker was intended to avaoid importing all packages (which is
really expensive) when all it needs is to stat them. So it's only
"fast" for parts of the code that *need* it.
- This commit makes repositories instantiate the checker lazily, so it's
only constructed when needed.
- This was needed when we transitioned to all lowercase packages because
git didn't handle case changes well on case-insensitive filesystems.
- Now it just adds extra stat calls to startup, and we check for
all-lowercase package names in tests, so we'll remove it.
- people using really old versions of Spack can re-clone.
* Create unload_module method
Extract code from load_module into unload_module.
* Unload modules to create a clean env on Cray
removes cray-libsci, cray-mpich and darshan to prevent any silent
linking with those packages.
* Add format to separate target and os for path
spec format can now handle separations of target and os for setting
up the path.
* Added ${PLATFORM} et al to spec.format()
${PLATFORM}, ${OS}, ${TARGET}
* Update tests
Updated tests and got rid of unnecessary code.
* Also update documentation to reflect this new ability.
* Add default path scheme to config.yaml
Added default path scheme to config.yaml. Users can overwrite this
section if they want.
* Speedup the default 'libs' property search - important for external
packages.
* As advised by @alalazo, use tuples instead of lists inside
_libs_default_handler.
* Added installation date and time to the database
Information on the date and time of installation of a spec is recorded
into the database. The information is retained on reindexing.
* Expose the possibility to query for installation date
The DB can now be queried for specs that have been installed in a given
time window. This query possibility is exposed to command line via two
new options of the `find` command.
* Extended docstring for Database._add
* Use timestamps since the epoch instead of formatted date in the DB
* Allow 'pretty date' formats from command line
* Substituted kwargs with explicit arguments
* Simplified regex for pretty date strings. Added unit tests.
This updates architecture concretization to
* Search for the nearest parent in the DAG for architecture information
rather than defaulting to the root of the DAG
* Propagate architecture settings transitively, such that if for
example the target is set at the root of the dag it will set the
same target on indirect dependencies (assuming no intermediate
dependency specifies a separate target). Previously this occurred
in general but under some conditions did not, for example if an
intermediate dependency specified some subset of architecture
properties.
* Create mirror for system with different compilers
Spack concretizes the spec provided by the user in
"spack mirror create" to ensure downloading the right
dependencies. Under normal circumstances concretization
requires that the chosen compiler exists on the system,
but this is not required when creating download mirrors
for other systems, so this requirement is removed in that
case.
* Add test for disabling compiler existence check
* Update compiler existence checking logic
* improve test for disabling compiler existence check
* make py-setuptools a run-time-only dep for py-basemap and patch python package to only apply setuptools flag for build deps
* py-qtconsole does not require setuptools
* This allows Spack to work with MD5 hashes on machines with openssl in FIPS mode.
* We are still using MD5 for validation in many places, and a later PR will replace all uses of MD5 with SHA256.
* This is a quick fix until that happens.
- transitive dependencies were not being handled correctly
- restructure code to do recursion and mark visited packages properly
- add `-V` option to *not* expand virtuals in spack dependencies
This re-adds the option to create and install unsigned tarballs, now
with the -u option (--unsigned) rather than the -y option.
This also changes the "keys" command, replacing the -y/--yes-to-all
option with the -t/--trust option (which has the same effect but is
more-clearly named).
Fixes#7130
shutil.move expects a source path like "/x/y/" to be a directory and
fails if "/x/y" is a symlink. This invokes realpath on the source
path to avoid the issue.
Fixes#7356
In some cases OperatingSystem (e.g. LinuxDistro) was getting
instantiated with a version that contains dashes. This breaks because
the concretizer later converts this value to a string and re-parses
it, and the '-' character is used to separate architecture components.
This adds a guard in the initializer to convert '-' to '_'.
Fixes#7237Fixes#6404Fixes#6418Fixes#6369
Identify when binary relocation fails to remove all instances of the
source prefix (and report an error to the user unless they specify
-a to allow the old root to appear). This check occurs at two stages:
during "bincache create" all instances of the root are replaced with
a special placeholder string ("@@@@..."), and a failure occurs if the
root is detected at this point; when the binary package is extracted
there is a second check. This addresses #7237 and #6418.
This is intended to be compatible with previously-created binary
packages.
This also adds:
* Better error messages for "spack install --use-cache" (#6404)
* Faster relocation on Mac OS (using a single call to
install_name_tool for all files rather than a call for each file)
* Clean up when "buildcache create" fails (addresses #6369)
* Explicit error message when the spack instance extracting the binary
package uses a different install layout than the spack instance that
created the binary package (since this is currently not supported)
* Remove the option to create unsigned binary packages with -y
