Commit graph

2828 commits

Author SHA1 Message Date
Denis Davydov
939eedcb37 improve Advanced Compiler Configuration part of tutorial (#8325) 2018-05-31 10:28:38 -07:00
scheibelp
43114c2e06 more-flexible user-specified dependency constraints (#8162)
* allow user to constrain dependencies that are added conditionally

* remove check for not-visited deps from normalize, move it to concretize. The check now runs after the concretization loop completes (so an error is only reported if the user-mentioned spec doesnt appear anywhere in the dag)

* remove separate full_spec_deps variable; rename spec_deps to all_spec_deps to clarify that it merges user-specified dependencies with derived dependencies

* add unit test to confirm new functionality
2018-05-30 11:07:13 -07:00
Todd Gamblin
1a677e30f3
config: add spack config blame command (#8081)
- `spack config blame` is similar to `spack config get`, but it prints
  out the config file and line number that each line of the merged
  configuration came from.

- This is a debugging tool for understanding where Spack config settings
  come from.

- add tests for config blame
2018-05-30 10:25:22 -07:00
Adam J. Stewart
87223a531b
Document specs with multiple version ranges (#8284) 2018-05-30 10:36:56 -05:00
Michael Kuhn
c77bc613eb Fix missing spack.compiler import (#8240)
Fixes: #8258

#8090 altered import behavior so that import spack no longer
provides access to many other Spack modules. This addresses
a case which depended on the prior behavior and was not
updated as part of #8090. This particular import error only
came up when users were setting compiler flags on specs.

See also: #8194
2018-05-24 10:59:20 -07:00
Axel Huebl
cd72312940 CUDA 9.2 (#8238)
Add the latest CUDA release, v9.2 and updated CUDA Compatibility Matrix
2018-05-22 12:45:47 +02:00
George Hartzell
e954686e5a Add info about haveged to gpg/randomness note (#8227) 2018-05-21 15:57:03 -05:00
Adam J. Stewart
0ccbc3a479
URL parsing: strip x86-64 from filename (#8200) 2018-05-20 15:01:58 -05:00
Todd Gamblin
22b551f84f fix leftover references to spack.<symbol>
- there were some leftover spack.* names being used after we removed
  globals and moved everything in the top-level namespace to spack.pkgkit

- point those references to their new homes
2018-05-19 07:37:24 -07:00
Todd Gamblin
8d7873e8b2 imports: remove unnecessary import spack statements
- remove most `import spack` statements, except for files that need
  `spack_version`

- import spack is no longer sufficient to use submodules
  (e.g. spack.directives).

  - these submodules must be imported directly. Update references
    accordingly.
2018-05-18 16:06:21 -07:00
Todd Gamblin
6b2c49648a init: move symbols in spack to spack.pkgkit
- Spack packages were originally expected to call `from spack import *`
  themselves, but it has become difficult to manage imports in the
  Spack core.

- the top-level namespace polluted by package symbols, and it's not
  possible to avoid circular dependencies and unnecessary module loads in
  the core, given all the stuff the packages need.

- This makes the top-level `spack` package essentially empty, save for a
  version tuple and a version string, and `from spack import *` is now
  essentially a no-op.

- The common routines and directives that packages need are now in
  `spack.pkgkit`, and the import system forces packages to automatically
  include this so that old packages that call `from spack import *`
  will continue to work without modification.

- Since `from spack import *` is no longer required, we could consider
  removing ``from spack import *`` from packages in the future and
  shifting to ``from spack.pkgkit import *``, but we can wait a while to
  do this.
2018-05-18 16:06:21 -07:00
Todd Gamblin
54201e3c02
locks: add configuration and command-line options to enable/disable locks (#7692)
- spack.util.lock behaves the same as llnl.util.lock, but Lock._lock and
  Lock._unlock do nothing.

- can be disabled with a control variable.

- configuration options can enable/disable locking:
  - `locks` option in spack configuration controls whether Spack will use filesystem locks or not.
  - `-l` and `-L` command-line options can force-disable or force-enable locking.

- Spack will check for group- and world-writability before disabling
  locks, and it will not allow a group- or world-writable instance to
  have locks disabled.

- update documentation
2018-05-18 14:41:03 -07:00
Todd Gamblin
5ef30e0a51 make check_for_compiler_existence an instance variable 2018-05-17 14:10:30 -07:00
Todd Gamblin
e9a3e3bfbb use os.path.join instead of join_path in core
- Spack core has long used llnl.util.filesystem.join_path, but
  os.path.join is pretty much the same thing, and is more efficient.

- Use os.path.join in the core Spack code from now on.
2018-05-17 14:10:30 -07:00
Todd Gamblin
f202198777 Convert lazy singleton functions to Singleton object
- simplify the singleton pattern across the codebase
- reduce lines of code needed for crufty initialization
- reduce functions that need to mess with a global

- Singletons whose semantics changed:
  - spack.store.store() -> spack.store
  - spack.repo.path() -> spack.repo.path
  - spack.config.config() -> spack.config.config
  - spack.caches.fetch_cache() -> spack.caches.fetch_cache
  - spack.caches.misc_cache() -> spack.caches.misc_cache
2018-05-17 14:10:30 -07:00
Todd Gamblin
3493f7e793 init: make spack.cmd.all_commands lazy
- `spack.cmd.all_commands` does a directory listing on
  `lib/spack/spack/cmd`, regardless of whether it is needed

- make this lazy so that the directory listing won't happen unless it's
  necessary.
2018-05-17 14:10:30 -07:00
Todd Gamblin
1fe5dbf338 init: only imoprt jsonschema if needed
- It turns out that jsonschema is one of the more expensive imports.

- move imports of jsonschema into functions to avoid the performance hits
  for calls that don't need config.
2018-05-17 14:10:30 -07:00
Todd Gamblin
6f2cac9565 init: initialize spack.store lazily
- spack.store was previously initialized at the spack.store module level,
  but this means the store has to be initialized on every spack call.

- this moves the state in spack.store to a singleton so that the store is
  only initialized when needed.
2018-05-17 14:10:30 -07:00
Todd Gamblin
c615d2be06 init: move spack.repo global variable into its own module
- spack.repository module is now spack.repo

- `spack.repo` is now `spack.repo.path()` and loaded lazily

- Added `spack.repo.get()` and `spack.repo.all_package_names()` as
  convenience functions to simplify the new lazy interface.

- updated tests and code
2018-05-17 14:10:30 -07:00
Todd Gamblin
2a09b627c9 tests: rename builtin_mock to "mock_packages" for clarity.
- rename `builtin_mock` and `refresh_builtin_mock` to the more clear
  `mock_packages` and `mutable_mock_packages`
2018-05-17 14:10:30 -07:00
Todd Gamblin
d1903f3bf3 init: convert spack.debug global variable to config option 2018-05-17 14:10:30 -07:00
Todd Gamblin
47dc96224d init: remove package_testing global
- refactor the way test dependencies are passed to the concretizer
- remove global state
- update tests
2018-05-17 14:10:30 -07:00
Todd Gamblin
bc9f5f084f init: remove dependency on spack.version
- no longer require `spack_version` to be a Version (it isn't used that
  way anyway)

- use a simple tuple `spack_version_info` with major, minor, patch
  versions

- generate `spack_version` from the tuple
2018-05-17 14:10:30 -07:00
Todd Gamblin
678639749c init: remove binary_cache_retrieved_specs global variable
- remove variable from spack/__init__.py
- clean up imports and some code structure in binary_distribution.py
2018-05-17 14:10:30 -07:00
Todd Gamblin
3e6d85c404 init: remove spack.concretizer global variable 2018-05-17 14:10:30 -07:00
Todd Gamblin
d4709673b5 init: remove build_jobs global variable 2018-05-17 14:10:30 -07:00
Todd Gamblin
2ab7b7a5a0 init: convert spack.dirty global to config option 2018-05-17 14:10:30 -07:00
Todd Gamblin
278933e824 init: remove template_dirs global
- remove template_dirs global variable from __init__.py

- also remove update_template_dirs fixture, which had no effect on test
  correctness
2018-05-17 14:10:30 -07:00
Todd Gamblin
77bd2dd706 init: replace global spack.do_checksum with config option 2018-05-17 14:10:30 -07:00
Todd Gamblin
73ab0e5dd7 init: replace spack.insecure global with spack.config.get 2018-05-17 14:10:30 -07:00
Todd Gamblin
863ccd5e94 config: simplify configuration API to just get and set
- replace `spack.config.get_configuration()` with `spack.config.config()`

- replace `get_config`/`update_config` with `get`, `set`

- add a path syntax that can be used to refer to specific config options
  without firt getting the entire configuration dict

- update usages of `get_config` and `update_config` to use `get` and `set`
2018-05-17 14:10:30 -07:00
Todd Gamblin
c9ea957563 config: create internal config scope for commands to use. 2018-05-17 14:10:30 -07:00
Todd Gamblin
eee502cc3b init: Move file cache initialization out of __init__.py 2018-05-17 14:10:30 -07:00
Todd Gamblin
d3a1ccd2fa config: rework config system into a class instead of globals
- Current configuration code forces the config system to be initialized
  at module scope, so configs are parsed on every Spack run, essentially
  before anything else.

- We need more control over configuration init order, so move the config
  scopes into a class and reduce global state in config.py
2018-05-17 14:10:30 -07:00
Todd Gamblin
6e398c7f08 init: move editor from spack/__init__.py to spack.util.editor 2018-05-17 14:10:30 -07:00
Todd Gamblin
a4d276fbe4 init: factor paths out of spack/__init__.py and into spack.paths module 2018-05-17 14:10:30 -07:00
Todd Gamblin
74aee60f7d init: simplify import ordering in __init__.py 2018-05-17 14:10:30 -07:00
Zack Galbreath
f7d080b7fb Move report writers to separate classes 2018-05-15 05:43:07 -07:00
Zack Galbreath
49e37a5ecf Add test case for CDash reporting 2018-05-15 05:43:07 -07:00
Zack Galbreath
218331552c upload reports to CDash 2018-05-15 05:43:07 -07:00
Zack Galbreath
847c1216d0 Generate CDash reports for build/install step 2018-05-15 05:43:07 -07:00
Zack Galbreath
ae0ba373b8 CDash report for concretization errors
Capture any concretization errors and record them in a CTest Update.xml file.
2018-05-15 05:43:07 -07:00
Zack Galbreath
d7581697a5 CDash report for configure step
Generate CTest XML file containing configure output
2018-05-15 05:43:07 -07:00
Zack Galbreath
f35d5bbf2b Capture source line number in CTest log parser 2018-05-15 05:43:07 -07:00
Zack Galbreath
d174d6618f Fix typo in comment 2018-05-15 05:43:07 -07:00
Zack Galbreath
072f907831 Add CDash as a valid reporting format 2018-05-15 05:43:07 -07:00
Zack Galbreath
e4e8c72fa1 Initialize report generator before parsing specs
This will allow us to generate reports for concretization errors
2018-05-15 05:43:07 -07:00
Zack Galbreath
de01d70ae4 Allow --overwrite and --log-format to be used together
Restructure the logic of the spack install command to allow these two
command-line arguments to be used at the same time.
2018-05-15 05:43:07 -07:00
Zack Galbreath
6c5dbdd9cd Format-agnostic data structure for reports
Replace the JUnit-specific terms 'testsuite' and 'testcase' with
'spec' and 'package', respectively.
2018-05-15 05:43:07 -07:00
Massimiliano Culpo
b4859e10e2 Packages can tune the list of files to be archived at the end of install (#7760)
Fixes #2781

This PR introduces a new attribute for packages called
`archive_files`, which designates files that should be saved from
a package build (e.g. the config.log generated during autotools
builds).

The attribute contains a list of glob expressions; Any file that
matches will be archived in the `<prefix>/.spack/archived-files`
directory. Errors that occur when archiving files are collected and
reported in a file named `<prefix>/.spack/archived-files/errors.txt`.

`AutotoolsPackage` and `CMakePackage` provide a sensible default
override for this attribute.
2018-05-08 17:40:53 -07:00