hlrs-software-stack/spack
7
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Add API Docs for lib/spack/llnl (#3982)

Browse source 
* Add API Docs for lib/spack/llnl
* Clean up after previous builds
* Better fix for purging API docs
This commit is contained in:
Adam J. Stewart 2017-04-26 00:24:02 -05:00 committed by Todd Gamblin
parent 11dae722c2
commit eaa50d3b7c
10 changed files with 128 additions and 99 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
lib/spack/docs/.gitignore vendored
View file

@ -1,5 +1,5 @@
package_list.rst
command_index.rst
spack*.rst
modules.rst
llnl*.rst
_build

7
lib/spack/docs/Makefile
View file

@ -9,7 +9,7 @@ PAPER =
BUILDDIR = _build
export PYTHONPATH := ../../spack:$(PYTHONPATH)
APIDOC_FILES = spack*.rst
APIDOC_FILES = spack*.rst llnl*.rst
# Internal variables.
PAPEROPT_a4 = -D latex_paper_size=a4
@ -58,7 +58,8 @@ upload:
git push -f github gh-pages
apidoc:
sphinx-apidoc -T -o . $(PYTHONPATH)/spack
sphinx-apidoc -f -T -o . ../spack
sphinx-apidoc -f -T -o . ../llnl
help:
@echo "Please use \`make <target>' where <target> is one of"
@ -83,7 +84,7 @@ help:
@echo " doctest to run all doctests embedded in the documentation (if enabled)"
clean:
-rm -f package_list.rst command_index.rst modules.rst
-rm -f package_list.rst command_index.rst
-rm -rf $(BUILDDIR)/* $(APIDOC_FILES)
html:

17
lib/spack/docs/conf.py
View file

@ -95,10 +95,21 @@
for cmd in sorted(command_names):
index.write(' * :ref:`%s`\n' % cmd)
#
# Run sphinx-apidoc
sphinx_apidoc(['-T', '-o', '.', '../spack'])
os.remove('modules.rst')
#
# Remove any previous API docs
# Read the Docs doesn't clean up after previous builds
# Without this, the API Docs will never actually update
#
apidoc_args = [
'sphinx_apidoc', # The first arugment is ignored
'--force', # Overwrite existing files
'--no-toc', # Don't create a table of contents file
'--output-dir=.', # Directory to place all output
]
sphinx_apidoc(apidoc_args + ['../spack'])
sphinx_apidoc(apidoc_args + ['../llnl'])
#
# Exclude everything in spack.__all__ from indexing. All of these

3
lib/spack/docs/index.rst
View file

@ -73,7 +73,8 @@ or refer to the full manual below.
contribution_guide
packaging_guide
developer_guide
API Docs <spack>
Spack API Docs <spack>
LLNL API Docs <llnl>
==================
Indices and tables

55
lib/spack/llnl/util/lang.py
View file

@ -43,7 +43,7 @@ def index_by(objects, *funcs):
"""Create a hierarchy of dictionaries by splitting the supplied
set of objects on unique values of the supplied functions.
Values are used as keys. For example, suppose you have four
objects with attributes that look like this:
objects with attributes that look like this::
a = Spec(name="boost", compiler="gcc", arch="bgqos_0")
b = Spec(name="mrnet", compiler="intel", arch="chaos_5_x86_64_ib")
@ -55,15 +55,15 @@ def index_by(objects, *funcs):
lambda s: s.compiler)
index2 = index_by(list_of_specs, lambda s: s.compiler)
``index1'' now has two levels of dicts, with lists at the
leaves, like this:
``index1`` now has two levels of dicts, with lists at the
leaves, like this::
{ 'bgqos_0' : { 'gcc' : [a], 'xlc' : [c] },
'chaos_5_x86_64_ib' : { 'intel' : [b, d] }
}
And ``index2'' is a single level dictionary of lists that looks
like this:
And ``index2`` is a single level dictionary of lists that looks
like this::
{ 'gcc' : [a],
'intel' : [b,d],
@ -72,12 +72,12 @@ def index_by(objects, *funcs):
If any elemnts in funcs is a string, it is treated as the name
of an attribute, and acts like getattr(object, name). So
shorthand for the above two indexes would be:
shorthand for the above two indexes would be::
index1 = index_by(list_of_specs, 'arch', 'compiler')
index2 = index_by(list_of_specs, 'compiler')
You can also index by tuples by passing tuples:
You can also index by tuples by passing tuples::
index1 = index_by(list_of_specs, ('arch', 'compiler'))
@ -204,7 +204,7 @@ def clear(self):
def list_modules(directory, **kwargs):
"""Lists all of the modules, excluding __init__.py, in a
"""Lists all of the modules, excluding ``__init__.py``, in a
particular directory. Listed packages have no particular
order."""
list_directories = kwargs.setdefault('directories', True)
@ -226,14 +226,16 @@ def list_modules(directory, **kwargs):
def key_ordering(cls):
"""Decorates a class with extra methods that implement rich comparison
operations and __hash__. The decorator assumes that the class
implements a function called _cmp_key(). The rich comparison operations
will compare objects using this key, and the __hash__ function will
return the hash of this key.
operations and ``__hash__``. The decorator assumes that the class
implements a function called ``_cmp_key()``. The rich comparison
operations will compare objects using this key, and the ``__hash__``
function will return the hash of this key.
If a class already has __eq__, __ne__, __lt__, __le__, __gt__, or __ge__
defined, this decorator will overwrite them. If the class does not
have a _cmp_key method, then this will raise a TypeError.
If a class already has ``__eq__``, ``__ne__``, ``__lt__``, ``__le__``,
``__gt__``, or ``__ge__`` defined, this decorator will overwrite them.
Raises:
TypeError: If the class does not have a ``_cmp_key`` method
"""
def setter(name, value):
value.__name__ = name
@ -322,14 +324,14 @@ def match_predicate(*args):
"""Utility function for making string matching predicates.
Each arg can be a:
- regex
- list or tuple of regexes
- predicate that takes a string.
* regex
* list or tuple of regexes
* predicate that takes a string.
This returns a predicate that is true if:
- any arg regex matches
- any regex in a list or tuple of regexes matches.
- any predicate in args matches.
* any arg regex matches
* any regex in a list or tuple of regexes matches.
* any predicate in args matches.
"""
def match(string):
for arg in args:
@ -374,11 +376,12 @@ def __init__(self, message):
def duplicate_stream(original):
"""Duplicates a stream at the os level.
:param stream original: original stream to be duplicated. Must have a
`fileno` callable attribute.
Args:
original (stream): original stream to be duplicated. Must have a
``fileno`` callable attribute.
:return: duplicate of the original stream
:rtype: file like object
Returns:
file like object: duplicate of the original stream
"""
return os.fdopen(os.dup(original.fileno()))
@ -388,7 +391,7 @@ class ObjectWrapper(object):
while staying undercover.
This class is modeled after the stackoverflow answer:
- http://stackoverflow.com/a/1445289/771663
* http://stackoverflow.com/a/1445289/771663
"""
def __init__(self, wrapped_object):
wrapped_cls = type(wrapped_object)

20
lib/spack/llnl/util/lock.py
View file

@ -45,7 +45,7 @@
class Lock(object):
"""This is an implementation of a filesystem lock using Python's lockf.
In Python, `lockf` actually calls `fcntl`, so this should work with
In Python, ``lockf`` actually calls ``fcntl``, so this should work with
any filesystem implementation that supports locking through the fcntl
calls. This includes distributed filesystems like Lustre (when flock
is enabled) and recent NFS versions.
@ -60,7 +60,7 @@ def __init__(self, path, start=0, length=0):
This exposes a subset of fcntl locking functionality. It does
not currently expose the ``whence`` parameter -- ``whence`` is
always os.SEEK_SET and ``start`` is always evaluated from the
always ``os.SEEK_SET`` and ``start`` is always evaluated from the
beginning of the file.
"""
self.path = path
@ -80,7 +80,7 @@ def _lock(self, op, timeout=_default_timeout):
"""This takes a lock using POSIX locks (``fnctl.lockf``).
The lock is implemented as a spin lock using a nonblocking call
to lockf().
to ``lockf()``.
On acquiring an exclusive lock, the lock writes this process's
pid and host to the lock file, in case the holding process needs
@ -276,14 +276,14 @@ class LockTransaction(object):
This class can trigger actions when the lock is acquired for the
first time and released for the last.
If the acquire_fn returns a value, it is used as the return value for
__enter__, allowing it to be passed as the `as` argument of a `with`
statement.
If the ``acquire_fn`` returns a value, it is used as the return value for
``__enter__``, allowing it to be passed as the ``as`` argument of a
``with`` statement.
If acquire_fn returns a context manager, *its* `__enter__` function will be
called in `__enter__` after acquire_fn, and its `__exit__` funciton will be
called before `release_fn` in `__exit__`, allowing you to nest a context
manager to be used along with the lock.
If ``acquire_fn`` returns a context manager, *its* ``__enter__`` function
will be called in ``__enter__`` after ``acquire_fn``, and its ``__exit__``
funciton will be called before ``release_fn`` in ``__exit__``, allowing you
to nest a context manager to be used along with the lock.
Timeout for lock is customizable.

7
lib/spack/llnl/util/tty/__init__.py
View file

@ -213,9 +213,10 @@ def get_yes_or_no(prompt, **kwargs):
def hline(label=None, **kwargs):
"""Draw a labeled horizontal line.
Options:
char Char to draw the line with. Default '-'
max_width Maximum width of the line. Default is 64 chars.
Keyword Arguments:
char (str): Char to draw the line with. Default '-'
max_width (int): Maximum width of the line. Default is 64 chars.
"""
char = kwargs.pop('char', '-')
max_width = kwargs.pop('max_width', 64)

36
lib/spack/llnl/util/tty/colify.py
View file

@ -23,7 +23,7 @@
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
##############################################################################
"""
Routines for printing columnar output. See colify() for more information.
Routines for printing columnar output. See ``colify()`` for more information.
"""
from __future__ import division
@ -124,26 +124,22 @@ def colify(elts, **options):
uniform-width and variable-width (tighter) columns.
If elts is not a list of strings, each element is first conveted
using str().
using ``str()``.
Keyword arguments:
output=<stream> A file object to write to. Default is sys.stdout.
indent=<int> Optionally indent all columns by some number of spaces.
padding=<int> Spaces between columns. Default is 2.
width=<int> Width of the output. Default is 80 if tty not detected.
cols=<int> Force number of columns. Default is to size to terminal,
or single-column if no tty
tty=<bool> Whether to attempt to write to a tty. Default is to
autodetect a tty. Set to False to force
single-column output.
method=<string> Method to use to fit columns. Options are variable or
Keyword Arguments:
output (stream): A file object to write to. Default is ``sys.stdout``
indent (int): Optionally indent all columns by some number of spaces
padding (int): Spaces between columns. Default is 2
width (int): Width of the output. Default is 80 if tty not detected
cols (int): Force number of columns. Default is to size to
terminal, or single-column if no tty
tty (bool): Whether to attempt to write to a tty. Default is to
autodetect a tty. Set to False to force single-column
output
method (str): Method to use to fit columns. Options are variable or
uniform. Variable-width columns are tighter, uniform
columns are all the same width and fit less data on
the screen.
the screen
"""
# Get keyword arguments or set defaults
cols = options.pop("cols", 0)
@ -220,7 +216,7 @@ def colify(elts, **options):
def colify_table(table, **options):
"""Version of colify() for data expressed in rows, (list of lists).
"""Version of ``colify()`` for data expressed in rows, (list of lists).
Same as regular colify but takes a list of lists, where each
sub-list must be the same length, and each is interpreted as a
@ -247,7 +243,7 @@ def transpose():
def colified(elts, **options):
"""Invokes the colify() function but returns the result as a string
"""Invokes the ``colify()`` function but returns the result as a string
instead of writing it to an output string."""
sio = StringIO()
options['output'] = sio

68
lib/spack/llnl/util/tty/color.py
View file

@ -23,39 +23,45 @@
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
##############################################################################
"""
This file implements an expression syntax, similar to printf, for adding
This file implements an expression syntax, similar to ``printf``, for adding
ANSI colors to text.
See colorize(), cwrite(), and cprint() for routines that can generate
colored output.
See ``colorize()``, ``cwrite()``, and ``cprint()`` for routines that can
generate colored output.
colorize will take a string and replace all color expressions with
ANSI control codes. If the isatty keyword arg is set to False, then
``colorize`` will take a string and replace all color expressions with
ANSI control codes. If the ``isatty`` keyword arg is set to False, then
the color expressions will be converted to null strings, and the
returned string will have no color.
cwrite and cprint are equivalent to write() and print() calls in
python, but they colorize their output. If the stream argument is
not supplied, they write to sys.stdout.
``cwrite`` and ``cprint`` are equivalent to ``write()`` and ``print()``
calls in python, but they colorize their output. If the ``stream`` argument is
not supplied, they write to ``sys.stdout``.
Here are some example color expressions:
@r Turn on red coloring
@R Turn on bright red coloring
@*{foo} Bold foo, but don't change text color
@_{bar} Underline bar, but don't change text color
@*b Turn on bold, blue text
@_B Turn on bright blue text with an underline
@. Revert to plain formatting
@*g{green} Print out 'green' in bold, green text, then reset to plain.
@*ggreen@. Print out 'green' in bold, green text, then reset to plain.
========== ============================================================
Expression Meaning
========== ============================================================
@r Turn on red coloring
@R Turn on bright red coloring
@*{foo} Bold foo, but don't change text color
@_{bar} Underline bar, but don't change text color
@*b Turn on bold, blue text
@_B Turn on bright blue text with an underline
@. Revert to plain formatting
@*g{green} Print out 'green' in bold, green text, then reset to plain.
@*ggreen@. Print out 'green' in bold, green text, then reset to plain.
========== ============================================================
The syntax consists of:
color-expr = '@' [style] color-code '{' text '}' | '@.' | '@@'
style = '*' | '_'
color-code = [krgybmcwKRGYBMCW]
text = .*
========== =================================================
color-expr '@' [style] color-code '{' text '}' | '@.' | '@@'
style '*' | '_'
color-code [krgybmcwKRGYBMCW]
text .*
========== =================================================
'@' indicates the start of a color expression. It can be followed
by an optional * or _ that indicates whether the font should be bold or
@ -82,6 +88,7 @@ class ColorParseError(Exception):
def __init__(self, message):
super(ColorParseError, self).__init__(message)
# Text styles for ansi codes
styles = {'*': '1', # bold
'_': '4', # underline
@ -118,8 +125,8 @@ def escape(self, s):
return ''
def __call__(self, match):
"""Convert a match object generated by color_re into an ansi color code
This can be used as a handler in re.sub.
"""Convert a match object generated by ``color_re`` into an ansi
color code. This can be used as a handler in ``re.sub``.
"""
style, color, text = match.groups()
m = match.group(0)
@ -147,10 +154,17 @@ def __call__(self, match):
def colorize(string, **kwargs):
"""Take a string and replace all color expressions with ANSI control
codes. Return the resulting string.
If color=False is supplied, output will be plain text without
control codes, for output to non-console devices.
"""Replace all color expressions in a string with ANSI control codes.
Args:
string (str): The string to replace
Returns:
str: The filtered string
Keyword Arguments:
color (bool): If False, output will be plain text without control
codes, for output to non-console devices.
"""
color = kwargs.get('color', True)
return re.sub(color_re, match_to_ansi(color), string)

8
lib/spack/llnl/util/tty/log.py
View file

@ -51,7 +51,7 @@ class _SkipWithBlock():
class keyboard_input(object):
"""Disable canonical input and echo on a stream within a with block.
Use this with sys.stdin for keyboard input, e.g.:
Use this with ``sys.stdin`` for keyboard input, e.g.::
with keyboard_input(sys.stdin):
r, w, x = select.select([sys.stdin], [], [])
@ -103,14 +103,16 @@ def __exit__(self, exc_type, exception, traceback):
class log_output(object):
"""Spawns a daemon that reads from a pipe and writes to a file
Usage:
Usage::
# Spawns the daemon
with log_output('logfile.txt', 'w') as log_redirection:
# do things ... output is not redirected
with log_redirection:
# do things ... output will be logged
or:
or::
with log_output('logfile.txt', echo=True) as log_redirection:
# do things ... output is not redirected
with log_redirection: