Major stuff:
- Created a FileCache for managing user cache files in Spack. Currently just
handles virtuals.
- Moved virtual cache from the repository to the home directory so that users do
not need write access to Spack repositories to use them.
- Refactored `Transaction` class in `database.py` -- moved it to
`LockTransaction` in `lock.py` and made it reusable by other classes.
Other additions:
- Added tests for file cache and transactions.
- Added a few more tests for database
- Fixed bug in DB where writes could happen even if exceptions were raised
during a transaction.
- `spack uninstall` now attempts to repair the database when it discovers that a
prefix doesn't exist but a DB record does.
- Spack will check if the index needs updating, and will only parse
all package files if it does.
- Spack tries to parse as few package files as necessary.
- Don't need to list all packages unless we have to.
- Only use the list of all packages for existence checks if we have
generated it for some other purpose.
This PR will catch the error where the url can not be determined from a
VCS URL, such as git. It will print a message to the console and move on
because it should not be a fatal error at this point in the process.
This should fix#1459.
The archive_version variable should only get set if versions will be
checked and that is only if there is a list_url in the package file. For
VCS repos setting the variable triggers an error from web.py as it
parses the default_fetcher object.
This should fix#1422.
Assuming a bash interactive environment will be correctly formed on login, we
should prefer to probe the environment using a shell that reports itself as
`bash` instead of `sh` which may not source files that set the
environment modules in statements like:
```
case "$is" in
bash) test -s /etc/bash.bashrc.local && . /etc/bash.bashrc.local ;;
ksh) test -s /etc/ksh.kshrc.local && . /etc/ksh.kshrc.local ;;
zsh) test -s /etc/zsh.zshrc.local && . /etc/zsh.zshrc.local ;;
ash) test -s /etc/ash.ashrc.local && . /etc/ash.ashrc.local
esac
test -s /etc/sh.shrc.local && . /etc/sh.shrc.local
```
The list of default environment modules is obtained by calling
`module list -lt`
from a subshell with a wiped environment. This allows `/etc/profile` and other
init scripts to be fully sourced which should generally include loading the
default modules.
The list of default modules is then parsed for the first acceptable CPU target,
assumed to be the back_end target.
A platform to generically cover all Cray systems is introduced to avoid having
specific platforms for each of XK (XE, XT), XC, and future systems using CrayPE
and CNL.
The platform searches for 'front_end' and 'back_end' targets, in order, from:
* Environment variables 'SPACK_FRONT_END' and 'SPACK_BACK_END'
* A spack configuration file 'targets.yaml'
* Parsing `/etc/bash.bashrc.local` for first loaded CrayPE CPU target.
If a back_end target is not found through one of these methods, an exception is
raised. Otherwise, the back_end is set as the default target. The shell init
script search is based on recommendations in Crayports case #144359.
No automagic way of determining front_end targets has been found (to date) so if
a front_end is not specified through configuration, it is ignored which should
leave a spack instance in a consistant state.
In addition to `list_url` make sure the package has the `url` attribute
set before attempting to add urls from a list. This is to cover the case
where there may be a `list_url` specified in tandem with a
`url_for_version`.
This commit will make urls from list_url only checked if `list_url` is
set in the package file. This makes more sense as there is no need to
check for those if the attribute is not present. If `url` is present and
`list_url` is not then it would result in the same url. If
`url_for_version` is used then that will not work anyway.
This PR allows archive file retrieval from urls derived from the
`list_url` setting in a package file. This allows for continued
retrieval of checksummed archive files even when they are moved to a new
remote location when a package is updated upstream.
Set up the environment for the Intel compilers and tools. This commit
does the following:
- Unset variables that were incorrect from the auto guess prefix
inspections.
- Add a RemovePath environment_modifications_formats for dotkit.
- Set the module environment variables appropriate for the different
variants.
- Change the component logic so that the '+all' variant works. It was
getting split by letter and leaving COMPONENTS empty.
- Added a variant checking function.
- Added NONRPM_DB_DIR to the silent.cfg so that the product database
goes to the installation directory.
- With the product database in prefix the code to remove the product
database file from the home directory is no longer needed and was
removed.
- Reformat the 'tools' variant description.
There are probably more variables needed for the '+tools' for the
'professional' product version but I do not have access to that.
- Default scope is versioned with spack and can be overridden by site
or user config.
- Default scope provides sensible default concretization preferences
for all of Spack.
- per-platform concretization scope can be added later (to force a
particular MPI on, e.g., Cray systems)
