Commit graph

2818 commits

Author SHA1 Message Date
Todd Gamblin
6b2c49648a init: move symbols in spack to spack.pkgkit
- Spack packages were originally expected to call `from spack import *`
  themselves, but it has become difficult to manage imports in the
  Spack core.

- the top-level namespace polluted by package symbols, and it's not
  possible to avoid circular dependencies and unnecessary module loads in
  the core, given all the stuff the packages need.

- This makes the top-level `spack` package essentially empty, save for a
  version tuple and a version string, and `from spack import *` is now
  essentially a no-op.

- The common routines and directives that packages need are now in
  `spack.pkgkit`, and the import system forces packages to automatically
  include this so that old packages that call `from spack import *`
  will continue to work without modification.

- Since `from spack import *` is no longer required, we could consider
  removing ``from spack import *`` from packages in the future and
  shifting to ``from spack.pkgkit import *``, but we can wait a while to
  do this.
2018-05-18 16:06:21 -07:00
Todd Gamblin
54201e3c02
locks: add configuration and command-line options to enable/disable locks (#7692)
- spack.util.lock behaves the same as llnl.util.lock, but Lock._lock and
  Lock._unlock do nothing.

- can be disabled with a control variable.

- configuration options can enable/disable locking:
  - `locks` option in spack configuration controls whether Spack will use filesystem locks or not.
  - `-l` and `-L` command-line options can force-disable or force-enable locking.

- Spack will check for group- and world-writability before disabling
  locks, and it will not allow a group- or world-writable instance to
  have locks disabled.

- update documentation
2018-05-18 14:41:03 -07:00
Todd Gamblin
5ef30e0a51 make check_for_compiler_existence an instance variable 2018-05-17 14:10:30 -07:00
Todd Gamblin
e9a3e3bfbb use os.path.join instead of join_path in core
- Spack core has long used llnl.util.filesystem.join_path, but
  os.path.join is pretty much the same thing, and is more efficient.

- Use os.path.join in the core Spack code from now on.
2018-05-17 14:10:30 -07:00
Todd Gamblin
f202198777 Convert lazy singleton functions to Singleton object
- simplify the singleton pattern across the codebase
- reduce lines of code needed for crufty initialization
- reduce functions that need to mess with a global

- Singletons whose semantics changed:
  - spack.store.store() -> spack.store
  - spack.repo.path() -> spack.repo.path
  - spack.config.config() -> spack.config.config
  - spack.caches.fetch_cache() -> spack.caches.fetch_cache
  - spack.caches.misc_cache() -> spack.caches.misc_cache
2018-05-17 14:10:30 -07:00
Todd Gamblin
3493f7e793 init: make spack.cmd.all_commands lazy
- `spack.cmd.all_commands` does a directory listing on
  `lib/spack/spack/cmd`, regardless of whether it is needed

- make this lazy so that the directory listing won't happen unless it's
  necessary.
2018-05-17 14:10:30 -07:00
Todd Gamblin
1fe5dbf338 init: only imoprt jsonschema if needed
- It turns out that jsonschema is one of the more expensive imports.

- move imports of jsonschema into functions to avoid the performance hits
  for calls that don't need config.
2018-05-17 14:10:30 -07:00
Todd Gamblin
6f2cac9565 init: initialize spack.store lazily
- spack.store was previously initialized at the spack.store module level,
  but this means the store has to be initialized on every spack call.

- this moves the state in spack.store to a singleton so that the store is
  only initialized when needed.
2018-05-17 14:10:30 -07:00
Todd Gamblin
c615d2be06 init: move spack.repo global variable into its own module
- spack.repository module is now spack.repo

- `spack.repo` is now `spack.repo.path()` and loaded lazily

- Added `spack.repo.get()` and `spack.repo.all_package_names()` as
  convenience functions to simplify the new lazy interface.

- updated tests and code
2018-05-17 14:10:30 -07:00
Todd Gamblin
2a09b627c9 tests: rename builtin_mock to "mock_packages" for clarity.
- rename `builtin_mock` and `refresh_builtin_mock` to the more clear
  `mock_packages` and `mutable_mock_packages`
2018-05-17 14:10:30 -07:00
Todd Gamblin
d1903f3bf3 init: convert spack.debug global variable to config option 2018-05-17 14:10:30 -07:00
Todd Gamblin
47dc96224d init: remove package_testing global
- refactor the way test dependencies are passed to the concretizer
- remove global state
- update tests
2018-05-17 14:10:30 -07:00
Todd Gamblin
bc9f5f084f init: remove dependency on spack.version
- no longer require `spack_version` to be a Version (it isn't used that
  way anyway)

- use a simple tuple `spack_version_info` with major, minor, patch
  versions

- generate `spack_version` from the tuple
2018-05-17 14:10:30 -07:00
Todd Gamblin
678639749c init: remove binary_cache_retrieved_specs global variable
- remove variable from spack/__init__.py
- clean up imports and some code structure in binary_distribution.py
2018-05-17 14:10:30 -07:00
Todd Gamblin
3e6d85c404 init: remove spack.concretizer global variable 2018-05-17 14:10:30 -07:00
Todd Gamblin
d4709673b5 init: remove build_jobs global variable 2018-05-17 14:10:30 -07:00
Todd Gamblin
2ab7b7a5a0 init: convert spack.dirty global to config option 2018-05-17 14:10:30 -07:00
Todd Gamblin
278933e824 init: remove template_dirs global
- remove template_dirs global variable from __init__.py

- also remove update_template_dirs fixture, which had no effect on test
  correctness
2018-05-17 14:10:30 -07:00
Todd Gamblin
77bd2dd706 init: replace global spack.do_checksum with config option 2018-05-17 14:10:30 -07:00
Todd Gamblin
73ab0e5dd7 init: replace spack.insecure global with spack.config.get 2018-05-17 14:10:30 -07:00
Todd Gamblin
863ccd5e94 config: simplify configuration API to just get and set
- replace `spack.config.get_configuration()` with `spack.config.config()`

- replace `get_config`/`update_config` with `get`, `set`

- add a path syntax that can be used to refer to specific config options
  without firt getting the entire configuration dict

- update usages of `get_config` and `update_config` to use `get` and `set`
2018-05-17 14:10:30 -07:00
Todd Gamblin
c9ea957563 config: create internal config scope for commands to use. 2018-05-17 14:10:30 -07:00
Todd Gamblin
eee502cc3b init: Move file cache initialization out of __init__.py 2018-05-17 14:10:30 -07:00
Todd Gamblin
d3a1ccd2fa config: rework config system into a class instead of globals
- Current configuration code forces the config system to be initialized
  at module scope, so configs are parsed on every Spack run, essentially
  before anything else.

- We need more control over configuration init order, so move the config
  scopes into a class and reduce global state in config.py
2018-05-17 14:10:30 -07:00
Todd Gamblin
6e398c7f08 init: move editor from spack/__init__.py to spack.util.editor 2018-05-17 14:10:30 -07:00
Todd Gamblin
a4d276fbe4 init: factor paths out of spack/__init__.py and into spack.paths module 2018-05-17 14:10:30 -07:00
Todd Gamblin
74aee60f7d init: simplify import ordering in __init__.py 2018-05-17 14:10:30 -07:00
Zack Galbreath
f7d080b7fb Move report writers to separate classes 2018-05-15 05:43:07 -07:00
Zack Galbreath
49e37a5ecf Add test case for CDash reporting 2018-05-15 05:43:07 -07:00
Zack Galbreath
218331552c upload reports to CDash 2018-05-15 05:43:07 -07:00
Zack Galbreath
847c1216d0 Generate CDash reports for build/install step 2018-05-15 05:43:07 -07:00
Zack Galbreath
ae0ba373b8 CDash report for concretization errors
Capture any concretization errors and record them in a CTest Update.xml file.
2018-05-15 05:43:07 -07:00
Zack Galbreath
d7581697a5 CDash report for configure step
Generate CTest XML file containing configure output
2018-05-15 05:43:07 -07:00
Zack Galbreath
f35d5bbf2b Capture source line number in CTest log parser 2018-05-15 05:43:07 -07:00
Zack Galbreath
d174d6618f Fix typo in comment 2018-05-15 05:43:07 -07:00
Zack Galbreath
072f907831 Add CDash as a valid reporting format 2018-05-15 05:43:07 -07:00
Zack Galbreath
e4e8c72fa1 Initialize report generator before parsing specs
This will allow us to generate reports for concretization errors
2018-05-15 05:43:07 -07:00
Zack Galbreath
de01d70ae4 Allow --overwrite and --log-format to be used together
Restructure the logic of the spack install command to allow these two
command-line arguments to be used at the same time.
2018-05-15 05:43:07 -07:00
Zack Galbreath
6c5dbdd9cd Format-agnostic data structure for reports
Replace the JUnit-specific terms 'testsuite' and 'testcase' with
'spec' and 'package', respectively.
2018-05-15 05:43:07 -07:00
Massimiliano Culpo
b4859e10e2 Packages can tune the list of files to be archived at the end of install (#7760)
Fixes #2781

This PR introduces a new attribute for packages called
`archive_files`, which designates files that should be saved from
a package build (e.g. the config.log generated during autotools
builds).

The attribute contains a list of glob expressions; Any file that
matches will be archived in the `<prefix>/.spack/archived-files`
directory. Errors that occur when archiving files are collected and
reported in a file named `<prefix>/.spack/archived-files/errors.txt`.

`AutotoolsPackage` and `CMakePackage` provide a sensible default
override for this attribute.
2018-05-08 17:40:53 -07:00
Massimiliano Culpo
ff3205d21e
Specs add a space before dependencies (#7942)
fixes #7941

Modified string representation of Specs to add a space before deps
Unit-tests have been modified accordingly
Added a test for regression on #7941
2018-05-07 09:05:50 +02:00
健美猫
c451644f12 Support *.TAR.gz extension. (#7973) 2018-05-06 09:34:16 -05:00
Adam J. Stewart
3a29cc3f5c
Fix coloring of URLs containing @ symbol (#7799)
* Fix coloring of URLs containing @ symbol

* Fix coloring of descriptions containing @ symbol
2018-05-05 17:27:40 -05:00
Adam J. Stewart
5af4d4f0ac
Set default provider for every virtual package (#7784)
* Set default provider for every virtual package

* Add unit test to ensure that every virtual package has a default provider explicitly set
2018-05-05 17:26:47 -05:00
scheibelp
d17372290d
Fix initialization of defaults in yaml schema validation (#7959)
Fixes #7924

Spack's yaml schema validator was initializing all instances of
unspecified properties with the same object, so that updating the
property for one entry was updating it for others (e.g. updating
versions for one package would update it for other packages).

This updates the schema validator to instantiate defaults with
separate object instances and adds a test to confirm this behavior
(and also confirms #7924 without this change).
2018-05-01 17:23:27 -07:00
Adam J. Stewart
c1febf8d9c Document how to run unit tests with print statements (#7860) 2018-04-28 17:59:33 +02:00
Chris Green
6cc79f8142 Allow len(Version) to return number of split components. (#7918) 2018-04-26 15:38:43 -07:00
serbanmaerean
963c103a67 XL Fortran compiler >= 16.0 is synced with C compiler version (#7887)
* Use reported version of IBM XL Fortran compiler for compiler versions >= 16.0.

Starting with the April 2018 release, the IBM XL C and Fortran compilers report the same version, 16.0.  Consequently, there is no need to downgrade the Fortran compiler version to match that of the C compiler.
2018-04-26 14:50:14 -07:00
Sergey Kosukhin
0f35f00b6e Use GitLab's API endpoint for fetching a git snapshot. (#7881)
* Use GitLab's API endpoint for fetching a git snapshot.

* More GitLab packages use the API.

* find_list_url for GitLab's API URLs.

* Flake8

* Url for 'hacckernels'.

* Check GitLab API regexps before the non-API ones.
2018-04-26 16:40:11 -05:00
健美猫
9dad069e89 config_args does NOT apply to CMakePackage. (#7917) 2018-04-26 13:03:45 -05:00