Commit graph

2924 commits

Author SHA1 Message Date
Todd Gamblin
0e81f6cba5 refactor: clean up and fix the cc test
- Add tests to ensure that RPATHs are not added in cc mode, which can
  cause some builds to fail.

- Change cc.py to use pytest style

- Instead of writing out all the flags, break the flags down into
  variables so that it's easy to read what each test is supposed to
  check. This should make cc.py more maintainable in the future.
2018-08-08 01:51:51 -07:00
Todd Gamblin
c8fb9b5479 bugfix: cc should not add -L or -Wl,-rpath in compile-only mode
- Adding -L and -Wl,-rpath to compile-only command lines ("cc mode" or
  "-c") causes clang (if not also other compilers) to emit warnings that
  confuse configure systems.

- Clang will print warnings about unused command-line arguments.

- This fix ensures that -L and -Wl,-rpath are not added if the compile
  line is just building an object file with -c

- This also cleans up the cc script in several places.
2018-08-08 01:51:51 -07:00
Gregory Becker
683c7fbf3b Restore cc: package search paths come before dependency paths (#4692)
Spack currently prepends include paths, library paths, and rpaths to the
compile line.  This causes problems when a header or library in the package
has the same name as one exported by one of its dependencies.  The
*dependency's* header will be preferred over the package's, which is not
what most builds expect.  This also breaks some of our production codes.

This restores the original cc behavior (from *very* early Spack) of parsing
compiler arguments out by type (`-L`, `-I`, `-Wl,-rpath`) and reconstituting
the full command at the end.

`<includes> <other_args> <library dirs> <rpaths>`

This differs from the original behavior in one significant way, though: it
*appends* the library arguments so that dependency libraries do not shadow
those in the build.

This is safe because semantics aren't affected by *interleaving* `-I`, `-L`,
and `-Wl,-rpath` arguments with others, only with each other (so the order of
two `-L` args affects the search path, but we search for all libraries on the
command line using the same search path).

We preserve the following:
1. Any system directory in the paths will be listed last.
2. The root package's include/library/RPATH flags come before flags of the
   same type for any dependency.
3. Order will be preserved within flags passed by the build (except system
   paths, which are moved to be last)
4. Flags for dependencies will appear between the root flags and the system
   flags, and the flags for any dependency will come before those for *its*
   dependencies (this is for completeness -- we already guarantee this in
   `build_environment.py`)
2018-08-08 01:51:51 -07:00
cedricchevalier19
3301e21f06 Fix performance issue when compiling. (#8828)
* Fix performance issue when compiling.

Spack was doing active wait when compiling, spoiling one core.
My fix consists in not setting any timeout for select, instead of
the previous 0 second.

* Fix comments about select.select timeout
2018-08-07 09:13:07 -07:00
Paul Chelarescu
6ef2eb8f7e libstdc++ detection uses os.path.realpath instead of os.readlink (#8865) 2018-08-06 22:30:29 -07:00
Adam J. Stewart
5f10d9239b Remove default variant bug from list of known issues (#8900) 2018-08-06 16:56:45 -04:00
George Hartzell
4e985051c0 Docstring typo: builing -> building (#8896) 2018-08-06 13:36:57 -04:00
Todd Gamblin
5cc2309330 revert cd9691de5 (#4692) while we work on a fix. 2018-08-04 20:31:55 -07:00
Todd Gamblin
b5071312c4 spack spec: no extra newline with --yaml; error with no specs
- `spack spec` now returns an error if given no specs

- removed superfluous trailing newline from `spack spec --yaml` output
  (only one newline now)
2018-08-03 15:18:34 -05:00
Adam J. Stewart
4f76e80925 Add spack arch --operating-system and --target flags 2018-08-03 11:32:08 -05:00
Todd Gamblin
43c77ac3cb libdwarf: remove use of hide_files()
- This was a nasty workaround due to the way our compiler wrappers used
  to work.  We don't want to have to modify our elfutils installation to
  install libdwarf.

- Since cd9691de5, we no longer need this because the package will always
  come before dependencies in our include order.
2018-08-02 19:00:52 -07:00
becker33
cd9691de53 cc: package search paths come before dependency paths (#4692)
Spack currently prepends include paths, library paths, and rpaths to the compile line.  This causes problems when a header or library in the package has the same name as one exported by one of its dependencies.  The *dependency's* header will be preferred over the package's, which is not what most builds expect.  This also breaks some of our production codes.

This restores the original cc behavior (from *very* early Spack) of parsing compiler arguments out by type (`-L`, `-I`, `-Wl,-rpath`) and reconstituting the full command at the end.

`<includes> <other_args> <library dirs> <rpaths>`

This differs from the original behavior in one significant way, though: it *appends* the library arguments so that dependency libraries do not shadow those in the build. 

This is safe because semantics aren't affected by *interleaving* `-I`, `-L`, and `-Wl,-rpath` arguments with others, only with each other (so the order fo two `-L` args affects the search path, but we search for all libraries on the command line using the same search path).

We preserve the following:
1. Any system directory in the paths will be listed last.
2. The root package's include/library/RPATH flags come before flags of the same type for any dependency.
3. Order will be preserved within flags passed by the build (except system paths, which are moved to be last)
4. Flags for dependencies will appear between the root flags and the system flags, and the flags for any dependency will come before those for *its* dependencies (this is for completeness -- we already guarantee this in `build_environment.py`)
2018-08-01 15:48:00 -07:00
Holly
38062a8abc docs: minor text changes (#8858) 2018-08-01 13:59:51 -07:00
Stephen Herbein
de60e9d582 lmod: fix use of custom separator in prepend_path etc. (#8737)
fixes #8736
2018-08-01 12:58:54 +02:00
Stephen Herbein
cdbc545793 docs: update spack view argument synatx (#8839)
The `--dependencies` and `--exclude` flags are now arguments to the `spack view`
comand rather than the `spack view symlink` command.
2018-07-31 10:41:41 -07:00
Todd Gamblin
2c45c3c5b3
spec: Spec.tree() merges deptypes when only covering nodes (#8821)
- previously, output could be confusing when deptypes were only shown for
  one dependent when a node had *multiple* dependents

- also fix default coverage of `Spec.tree()`: it previously defaulted to
  cover only build and link dependencies, but this is a holdover from
  when those were the only types.
2018-07-29 11:54:20 -07:00
Massimiliano Culpo
1fd9574353 Fixed a typo in the docs (the docker image to pull was wrong) (#8814) 2018-07-27 09:56:01 -05:00
Adam J. Stewart
7902f3eb49 Add docs on new top-level VCS attributes 2018-07-25 23:10:10 -07:00
Todd Gamblin
5f00056490 tests: convert mirror test to pytest with free functions. 2018-07-25 23:10:10 -07:00
Todd Gamblin
020c34e136 tests: add checks and tests for consistent version() arguments
- Previously, Spack didn't check the arguments you put in version()
  directives.

- So, you could do something like this, where there are arguments for a
  URL fetcher AND for a git fetcher:

  version('1.0', md5='abc123', git='https://foo.bar', commit='feda2343')

- Now, we check the arguments before constructing a fetcher, to ensure
  that each package has *only* arguments for a single type of fetcher.

- Also added `test_package_version_consistency()` to the `package_sanity`
  test, so that all builtin packages are required to have valid
  `version()` directives.
2018-07-25 23:10:10 -07:00
Todd Gamblin
6f7eaecfa0 tests: correct name of check_db to check_repo
- this test is checking the package *repository*, not the database.
2018-07-25 23:10:10 -07:00
Todd Gamblin
773cfe088f core: differentiate package-level fetch URLs by args to version()
- packagers can specify two top-level fetch URLs if one is `url`
  - e.g., `url` and `git` or `url` and `svn`

- allow only one VCS fetcher so we can differentiate between URL and VCS.

- also clean up fetcher logic and class structure
2018-07-25 23:10:10 -07:00
Todd Gamblin
04aec9d6f8 core: add check for conflicting top-level fetch attributes in packages
- ensure that packages can't have more than one of git, hg, svn, or url
2018-07-25 23:10:10 -07:00
Todd Gamblin
6837da2ad7 style: clean up exception definitions in package.py 2018-07-25 23:10:10 -07:00
Todd Gamblin
498d8cf04b core: fixes and tests for handling of fetcher attributes in packages
- Packages can remove the top-level `url` attribute and still work

- These are now legal:
  - Packages with *only* version-specific URLs (even with gaps)

  - Packages with a top-level git/hg/svn attribute and `version`
    directives for that.

- If a package has both a top-level hg/git/svn attribute AND a top-level
  url attribute, the url attribute takes precedence.
2018-07-25 23:10:10 -07:00
Adam J. Stewart
5792a805fc core: quick fix for packages without URLs
Some packages do not have a `url` and are instead downloaded via `git`,
`hg`, or `svn`. Some packages like `spectrum-mpi` cannot be downloaded at
all, and are placeholder packages for system installations. Previously,
`__init__()` in `PackageBase` crashed if a package did not have a `url`
attribute defined.

I hacked this section of code out, but I have no idea what the
repercussions of that are.
2018-07-25 23:10:10 -07:00
Adam J. Stewart
13c52066a5 Fix order of regexes reported by spack url summary 2018-07-24 21:02:28 -07:00
Todd Gamblin
656e935e50 core: make spack.util.crypto initialization less expensive.
- This hard-codes the hash lengths rather than computing them on import.

- Also cleans up the code in `spack.util.crypto` to make it easier to
  understand.
2018-07-24 18:35:02 -07:00
Todd Gamblin
674eb00e53
bugfix: spack module -m error message (#8794)
Fix this output error:

```
$ spack -m module loads mpileaks
==> Error: `spack module loads -m t -m c -m l ...` has been moved. Try this instead:

    $ spack module t loads mpileaks
    $ spack module c loads mpileaks
    $ spack module l loads mpileaks
```
2018-07-24 16:12:47 -07:00
Massimiliano Culpo
6bb2256043 Die with a meaningful error if a deprecated command is used
In case a deprecated form of the module command is used, the program
will exit non-zero and print an informative error message suggesting
which command should be used instead.
2018-07-24 11:27:17 -07:00
Massimiliano Culpo
35cef16974 Grouped all the module commands under spack module
As requested in the review all the commands meant to manage module
files have been grouped under the `spack module` command.

Unit tests have been refactored to match the new command structure.
2018-07-24 11:27:17 -07:00
Massimiliano Culpo
7cbe1aedb7 module file tutorial: rebuilt docker image and updated the script 2018-07-24 11:27:17 -07:00
Massimiliano Culpo
576f0c98d2 Added --dependencies to the help text of spack load
fixes #6321
2018-07-24 11:27:17 -07:00
Massimiliano Culpo
443d702971 spack load exits with 1 if module does not exist or is not installed
fixes #2215
fixes #2570
fixes #6676
fixes #7281
closes #3827

This PR reverts the use of `spack module loads` in favor of
`spack module find` when loading module files via Spack. After this PR
`spack load` will accept a single spec at a time, and will be able
to interpret correctly the `--dependencies` option.
2018-07-24 11:27:17 -07:00
Massimiliano Culpo
7a49ba56b6 Added blacklisting of implicit modules in docs + regression tests
fixes #4400

The feature requested in #4400 was already part of the module file
configuration, but it was neither tested nor documented. This
commit takes care of adding a few lines in the documentation and a
regression test.
2018-07-24 11:27:17 -07:00
Massimiliano Culpo
ff83003eaf Renamed 'patch_configuration' fixture to 'module_configuration'
This just because the fixture has been moved one level above the one
it was originally defined. In this more general context there's more
than one configuration file that could be patched for tests.
2018-07-24 11:27:17 -07:00
Massimiliano Culpo
008f171a7e Added a unit test for 'spack lmod setdefault' 2018-07-24 11:27:17 -07:00
Massimiliano Culpo
e81c0c3e2c Updated references to spack module in the documentation. 2018-07-24 11:27:17 -07:00
Massimiliano Culpo
6ab57571c2 Added 'setdefault' subcommand to 'spack lmod'
Added a subcommand that sets the default module to be loaded, if
multiple installations of the same package are present.
2018-07-24 11:27:17 -07:00
Massimiliano Culpo
0457c1fbee Split 'spack module' into multiple commands
'spack module' has been split into multiple commands, each one tied to a
specific module type. This permits the specialization of the new
commands with features that are module type specific (e.g. set the
default module file in lmod when multiple versions of the same package
are installed at the same time).
2018-07-24 11:27:17 -07:00
Todd Gamblin
fdcaf5c4c8 bugfix: fix spack spec --yaml
- repo membership test was broken by the refactor of spack/__init__.py

- refactor singleton so that 'spec in repo' works again for `spack.repo.path`

- fix spec command and add basic tests for `spack spec` and `spack spec --yaml`
2018-07-24 11:25:34 -07:00
Todd Gamblin
d4c4effb5e docs: update old docs for spack.package.PackageBase
- There was a lot of documentation in `PackageBase` dating back to the
  very first versions of Spack.

- It was repetitive and out of date, and the docs at spack.readthedocs.io
  are better.

- Remove the outdated specifics, and leave the minimal useful set of
  developer docs in `package.py`.
2018-07-23 11:57:52 -07:00
Todd Gamblin
bd3ffc7b76 core: use sha256 instead of md5 for spack checksum and spack create
- This changes `get_checksums_for_versions` to generate code that uses an
  explicit `sha256` argument instead if the bare `md5` hash we used to
  generate.

- also use a generic digest parameter for the `version` directive, rather
  than a specific `md5` parameter.
2018-07-23 11:57:52 -07:00
Todd Gamblin
a1d6909864 bugfix: stage test no longer modifies ~/.spack/config.yaml
- two stage tests weren't properly using the mutable_config fixture.
- this caused running `spack test` to modify the user's config.yaml
2018-07-21 17:12:36 -07:00
Todd Gamblin
b2f3b07ec0 docs: doucment spack config blame 2018-07-21 14:04:18 -07:00
Todd Gamblin
331da779f8 docs: updates to spack config get docs for command-line scopes 2018-07-21 14:04:18 -07:00
Elizabeth Fischer
46f87cb9f2 docs: command-line scopes
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
2018-07-21 14:04:18 -07:00
Elizabeth Fischer
52fbbdf5a1 config: allow user to add configuration scopes on the command line.
- Add command-line scope option to Spack

- Rework structure of main to allow configuration system to raise
  errors more naturally

Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
2018-07-21 14:04:18 -07:00
Todd Gamblin
2b0d944341 locks: fix bug when creating lockfiles in the current directory.
- Fixes a bug in `llnl.util.lock`

- Locks in the current directory would fail because the parent directory
  was the empty string.

- Fix this and return '.' for the parent of locks in the current
  directory.
2018-07-21 10:39:47 -07:00
Todd Gamblin
20e4038a72 style: make core comply with pep8-naming 2018-07-19 00:41:36 -07:00