hlrs-software-stack/spack
7
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Windows: Create installer and environment

Browse source 
* Add 'make-installer' command for Windows

* Add '--bat' arg to env activate, env deactivate and unload commands

* An equivalent script to setup-env on linux: spack_cmd.bat. This script
has a wrapper to evaluate cd, load/unload, env activate/deactivate.(#21734)

* Add spacktivate and config editor (#22049)

* spack_cmd: will find python and spack on its own. It preferentially
tries to use python on your PATH (#22414)

* Ignore Windows python installer if found (#23134)

* Bundle git in windows installer (#23597)

* Add Windows section to Getting Started document
(#23131), (#23295), (#24240)

Co-authored-by: Stephen Crowell <stephen.crowell@kitware.com>
Co-authored-by: lou.lawrence@kitware.com <lou.lawrence@kitware.com>
Co-authored-by: Betsy McPhail <betsy.mcphail@kitware.com>
Co-authored-by: Jared Popelar <jpopelar@txcorp.com>
Co-authored-by: Ben Cowan <benc@txcorp.com>

Update Installer CI

Co-authored-by: John Parent <john.parent@kitware.com>
This commit is contained in:
lou.lawrence@kitware.com 2021-01-08 15:48:07 -05:00 committed by Peter Scheibel
parent a7101db39d
commit 012758c179
20 changed files with 1002 additions and 12 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
.gitignore vendored
View file

@ -269,6 +269,11 @@ local.properties
# CDT-specific (C/C++ Development Tooling)
.cproject
# VSCode files
.vs
.vscode
.devcontainer
# CDT- autotools
.autotools

218
lib/spack/docs/getting_started.rst
View file

@ -1516,3 +1516,221 @@ To ensure that Spack does not autodetect the Cray programming
environment, unset the environment variable ``MODULEPATH``. This
will cause Spack to treat a linux container on a Cray system as a base
linux distro.
.. _windows_support:
----------------
Spack On Windows
----------------
Windows support for Spack is currently under development. While this work is
still in an early stage, it is currently possible to set up Spack and
perform a few operations on Windows. This section will guide
you through the steps needed to install Spack and start running it on a
fresh Windows machine.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Step 1: Install prerequisites
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
To use Spack on Windows, you will need the following packages:
* Microsoft Visual Studio
* Python
* Git
"""""""""""""""""""""""
Microsoft Visual Studio
"""""""""""""""""""""""
Microsoft Visual Studio provides the Windows C/C++ compiler that is
currently supported by Spack.
We require several specific components to be included in the Visual Studio
installation. One is the C/C++ toolset, which can be selected as "Desktop
development with C++" or "C++ build tools," depending on installation type
(Professional, Build Tools, etc.) The other required component is
"C++ CMake tools for Windows," which can be selected from among the optional
packages. This provides CMake and Ninja for use during Spack configuration.
If you already have Visual Studio installed, you can make sure these
components are installed by rerunning the installer. Next to your
installation, select "Modify" and look at the "Installation details" pane on the right.
""""""
Python
""""""
As Spack is a Python-based package, an installation of Python will be needed
to run it. Python 3 can be downloaded and installed from the Windows Store,
and will be automatically added to your ``PATH`` in this case.
"""
Git
"""
A bash console and GUI can be downloaded from https://git-scm.com/downloads.
When given the option of adjusting your ``PATH``, choose the ``Git from the
command line and also from 3rd-party software`` option. This will automatically
update your ``PATH`` variable to include the ``git`` command.
If you are unfamiliar with Git, there are a myriad of resources online to help
guide you through checking out repositories and switching development
branches.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Step 2: Install and setup Spack
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
We are now ready to get the Spack environment set up on our machine. We
begin by creating a top-level directory to do our work in: we will call
it ``spack_install`` in this tutorial. Inside this directory, use Git to
clone the Spack repo, hosted at https://github.com/spack/spack.git.
The files and scripts used for Windows installation are on the
``features/windows-support`` branch; ``cd`` into the repo and use
``git checkout`` to switch to it. Then navigate to
``lib\spack\spack\cmd\installer`` and copy the ``scripts`` directory and
``spack_cmd.bat`` up to the top-level ``spack_install`` directory. In a
Windows console, you can do both of these things by executing the following
commands from the ``spack_install`` level:
.. code-block:: console
xcopy lib\spack\spack\cmd\installer\scripts\ scripts\
xcopy lib\spack\spack\cmd\installer\spack_cmd.bat .
Your file structure should look like this after following the above
steps:
.. code-block:: console
spack_install
|--------spack
|--------scripts
|--------spack_cmd.bat
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Step 3: Run and configure Spack
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
To use Spack, run ``spack_cmd.bat`` (you may need to Run as Administrator).
This will provide a Windows command prompt with an environment properly set
up with Spack and its prerequisites. If you receive a warning message that
Python is not in your ``PATH`` (which may happen if you installed Python
from the website and not the Windows Store), add the location of the Python
executable to your ``PATH`` now.
To configure Spack, first run the following command inside the Spack console:
.. code-block:: console
spack compiler find
This creates a ``.spack`` directory in our home directory, along with a
``windows`` subdirectory containing a ``compilers.yaml`` file. On a fresh
Windows install, the only compiler that should be found is your installation
of Microsoft Visual Studio.
We need to provide the ``config.yaml`` configuration by ourselves. This goes
in the ``.spack\windows`` directory in your home directory. Open your text
editor of choice and enter the following lines for ``config.yaml``:
.. code-block:: yaml
config:
locks: false
install_tree:
root: $spack\opt\spack
projections:
all: '${ARCHITECTURE}\${COMPILERNAME}-${COMPILERVER}\${PACKAGE}-${VERSION}-${HASH}'
build_stage:
- ~/.spack/stage
(These settings are identical to those in the default ``config.yaml``
provided with your Spack checkout, except with forward slashes replaced by
backslashes for Windows compatibility.) It is important that all indentions
in .yaml files are done with spaces and not tabs, so take care when editing
one by hand.
For the ``packages.yaml`` file, there are two options. The first
and easiest choice is to use Spack to find installation on your system. In
the Spack terminal, run the following commands:
.. code-block:: console
spack external find cmake
spack external find ninja
The ``spack external find <name>`` will find executables on your system
with the same name given. The command will store the items found in
``packages.yaml`` in the ``.spack\`` directory.
Assuming the Spack found CMake and Ninja executables in the previous
step, continue to Step 4. If no executables were found, we will need to
direct spack towards the CMake and Ninja installations we set up with
Visual Studio. Therefore, your ``packages.yaml`` file will look something
like this, with possibly slight variants in the paths to CMake and Ninja:
.. code-block:: yaml
packages:
cmake:
externals:
- spec: cmake@3.19
prefix: 'c:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake'
buildable: False
ninja:
externals:
- spec: ninja@1.8.2
prefix: 'c:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\Common7\IDE\CommonExtensions\Microsoft\CMake\Ninja'
buildable: False
It is important to note that the version of your Ninja and CMake could
be different than what is shown here. If there is a difference, make sure
to use your version instead of the version listed above. Similiarly, if
you use a different version of Visual Studio ("Community" for example),
make sure the Professional part of the location is changed to your version.
The ``packages.yaml`` file should be placed inside either the ``.spack``
directory or the ``.spack\windows`` directory.
You can also use an separate installation of CMake if you have one and prefer
to use it. If you don't have a path to Ninja analogous to the above, then
you can obtain it by running the Visual Studio Installer and following the
instructions at the start of this section.
^^^^^^^^^^^^^^^^^
Step 4: Use Spack
^^^^^^^^^^^^^^^^^
Once the configuration is complete, it is time to give the installation a
test. Install a basic package through the Spack console via:
.. code-block:: console
spack install cpuinfo
"""""""""""""""""""""""""""
Windows Compatible Packages
"""""""""""""""""""""""""""
Many Spack packages are not currently compatible with Windows, due to Unix
dependencies or incompatible build tools like autoconf. Here are several
packages known to work on Windows:
* abseil-cpp
* cpuinfo
* glm
^^^^^^^^^^^^^^
For developers
^^^^^^^^^^^^^^
The intent is to provide a Windows installer that will automatically set up
Python, Git, and Spack, instead of requiring the user to do so manually.
Instructions for creating the installer are at
https://github.com/spack/spack/blob/features/windows-support/lib/spack/spack/cmd/installer/README.md.

4
lib/spack/spack/cmd/common/__init__.py
View file

@ -35,6 +35,9 @@ def shell_init_instructions(cmd, equivalent):
color.colorize("@*c{For fish:}"),
" source %s/setup-env.fish" % spack.paths.share_path,
"",
color.colorize("@*c{For Windows batch:}"),
" source %s/spack_cmd.bat" % spack.paths.share_path,
"",
"Or, if you do not want to use shell support, run " + (
"one of these" if shell_specific else "this") + " instead:",
"",
@ -45,6 +48,7 @@ def shell_init_instructions(cmd, equivalent):
equivalent.format(sh_arg="--sh ") + " # bash/zsh/sh",
equivalent.format(sh_arg="--csh ") + " # csh/tcsh",
equivalent.format(sh_arg="--fish") + " # fish",
equivalent.format(sh_arg="--bat ") + " # batch"
]
else:
msg += [" " + equivalent]

6
lib/spack/spack/cmd/env.py
View file

@ -60,6 +60,9 @@ def env_activate_setup_parser(subparser):
shells.add_argument(
'--fish', action='store_const', dest='shell', const='fish',
help="print fish commands to activate the environment")
shells.add_argument(
'--bat', action='store_const', dest='shell', const='bat',
help="print bat commands to activate the environment")
view_options = subparser.add_mutually_exclusive_group()
view_options.add_argument(
@ -173,6 +176,9 @@ def env_deactivate_setup_parser(subparser):
shells.add_argument(
'--fish', action='store_const', dest='shell', const='fish',
help="print fish commands to activate the environment")
shells.add_argument(
'--bat', action='store_const', dest='shell', const='bat',
help="print bat commands to activate the environment")
def env_deactivate(args):

116
lib/spack/spack/cmd/installer/CMakeLists.txt Normal file
View file

@ -0,0 +1,116 @@
cmake_minimum_required (VERSION 3.13)
project(spack_installer NONE)
set(PYTHON_VERSION "3.9.0" CACHE STRING "Version of Python to build.")
set(PY_DOWNLOAD_LINK "https://www.paraview.org/files/dependencies")
set(PY_FILENAME "Python-${PYTHON_VERSION}-win64.tar.xz")
set(PYTHON_DIR "Python-${PYTHON_VERSION}")
if (SPACK_VERSION)
set(SPACK_DL "https://github.com/spack/spack/releases/download/v${SPACK_VERSION}")
set(SPACK_FILENAME "spack-${SPACK_VERSION}.tar.gz")
set(SPACK_DIR "spack-${SPACK_VERSION}")
# SPACK DOWLOAD AND EXTRACTION-----------------------------------
file(DOWNLOAD "${SPACK_DL}/${SPACK_FILENAME}"
"${CMAKE_CURRENT_BINARY_DIR}/${SPACK_FILENAME}"
STATUS download_status
)
list(GET download_status 0 res)
if(res)
list(GET download_status 1 err)
message(FATAL_ERROR "Failed to download ${SPACK_FILENAME} ${err}")
endif()
message(STATUS "Successfully downloaded ${SPACK_FILENAME}")
execute_process(COMMAND ${CMAKE_COMMAND} -E tar xfz
"${CMAKE_CURRENT_BINARY_DIR}/${SPACK_FILENAME}"
WORKING_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}"
RESULT_VARIABLE res)
if(NOT res EQUAL 0)
message(FATAL_ERROR "Extraction of ${SPACK_FILENAME} failed.")
endif()
message(STATUS "Extracted ${SPACK_DIR}")
SET(SPACK_SOURCE "${CMAKE_CURRENT_BINARY_DIR}/${SPACK_DIR}")
elseif(SPACK_SOURCE)
get_filename_component(SPACK_DIR ${SPACK_SOURCE} NAME)
else()
message(FATAL_ERROR "Must specify SPACK_VERSION or SPACK_SOURCE")
endif()
# GIT DOWNLOAD----------------------------------------------------
set(GIT_FILENAME "Git-2.31.1-64-bit.exe")
file(DOWNLOAD "https://github.com/git-for-windows/git/releases/download/v2.31.1.windows.1/Git-2.31.1-64-bit.exe"
"${CMAKE_CURRENT_BINARY_DIR}/${GIT_FILENAME}"
STATUS download_status
EXPECTED_HASH "SHA256=c43611eb73ad1f17f5c8cc82ae51c3041a2e7279e0197ccf5f739e9129ce426e"
)
list(GET download_status 0 res)
if(res)
list(GET download_status 1 err)
message(FATAL_ERROR "Failed to download ${GIT_FILENAME} ${err}")
endif()
message(STATUS "Successfully downloaded ${GIT_FILENAME}")
# PYTHON DOWLOAD AND EXTRACTION-----------------------------------
file(DOWNLOAD "${PY_DOWNLOAD_LINK}/${PY_FILENAME}"
"${CMAKE_CURRENT_BINARY_DIR}/${PY_FILENAME}"
STATUS download_status
EXPECTED_HASH "SHA256=f6aeebc6d1ff77418678ed5612b64ce61be6bc9ef3ab9c291ac557abb1783420"
)
list(GET download_status 0 res)
if(res)
list(GET download_status 1 err)
message(FATAL_ERROR "Failed to download ${PY_FILENAME} ${err}")
endif()
message(STATUS "Successfully downloaded ${PY_FILENAME}")
execute_process(COMMAND ${CMAKE_COMMAND} -E tar xfz
"${CMAKE_CURRENT_BINARY_DIR}/${PY_FILENAME}"
WORKING_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}"
RESULT_VARIABLE res)
if(NOT res EQUAL 0)
message(FATAL_ERROR "Extraction of ${PY_FILENAME} failed.")
endif()
message(STATUS "Extracted ${PY_FILENAME}.")
# license must be a .txt or .rtf file
configure_file("${SPACK_LICENSE}" "${CMAKE_CURRENT_BINARY_DIR}/LICENSE.rtf" COPYONLY)
#INSTALLATION COMMANDS---------------------------------------------------
install(DIRECTORY "${SPACK_SOURCE}/"
DESTINATION "${SPACK_DIR}")
install(DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}/Python-${PYTHON_VERSION}-win64/"
DESTINATION "${PYTHON_DIR}")
install(DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/scripts/" DESTINATION "scripts")
#install(FILES "${CMAKE_CURRENT_SOURCE_DIR}/spack_cmd.bat/" DESTINATION ".")
# CPACK Installer Instructions
set(CPACK_PACKAGE_NAME "Spack")
set(CPACK_PACKAGE_VENDOR "Lawrence Livermore National Laboratories")
set(CPACK_PACKAGE_VERSION "0.16.0")
set(CPACK_PACKAGE_DESCRIPTION "A flexible package manager designed to support multiple versions, configurations, platforms, and compilers.")
set(CPACK_PACKAGE_HOMEPAGE_URL "https://spack.io")
set(CPACK_PACKAGE_FILE_NAME "${CPACK_PACKAGE_NAME}")
set(CPACK_PACKAGE_ICON "${SPACK_LOGO}")
set(CPACK_RESOURCE_FILE_README "${CMAKE_CURRENT_SOURCE_DIR}/README.md")
set(CPACK_RESOURCE_FILE_LICENSE "${CMAKE_CURRENT_BINARY_DIR}/LICENSE.rtf")
#set(CPACK_RESOURCE_FILE_WELCOME "${CMAKE_CURRENT_SOURCE_DIR}/NOTICE")
# WIX options (the default)
set(CPACK_GENERATOR "WIX")
set(CPACK_WIX_PRODUCT_ICON "${SPACK_LOGO}")
set(CPACK_WIX_UI_BANNER "${CMAKE_CURRENT_SOURCE_DIR}/banner493x58.bmp")
set(CPACK_WIX_PATCH_FILE "${CMAKE_CURRENT_SOURCE_DIR}/patch.xml")
set(CPACK_WIX_UPGRADE_GUID "D2C703E4-721D-44EC-8016-BCB96BB64E0B")
set(CPACK_WIX_SKIP_PROGRAM_FOLDER TRUE)
# Set full path to icon, shortcut in spack.wxs
set(SPACK_SHORTCUT "${CMAKE_CURRENT_SOURCE_DIR}/spack_cmd.bat")
configure_file("spack.wxs.in" "${CMAKE_CURRENT_BINARY_DIR}/spack.wxs")
configure_file("bundle.wxs.in" "${CMAKE_CURRENT_BINARY_DIR}/bundle.wxs")
set(CPACK_WIX_EXTRA_SOURCES "${CMAKE_CURRENT_BINARY_DIR}/spack.wxs")
include(CPack)

85
lib/spack/spack/cmd/installer/README.md Normal file
View file

@ -0,0 +1,85 @@
This README is a guide for creating a Spack installer for Windows using the
``make-installer`` command. The installer is an executable file that users
can run to install Spack like any other Windows binary.
Before proceeding, follow the setup instructions in Steps 1 and 2 of
[Getting Started on Windows](https://spack.readthedocs.io/en/latest/getting_started.html#windows_support).
# Step 1: Install prerequisites
The only additional prerequisite for making the installer is Wix. Wix is a
utility used for .msi creation and can be downloaded and installed at
https://wixtoolset.org/releases/. The Visual Studio extensions are not
necessary.
# Step 2: Make the installer
To use Spack, run ``spack_cmd.bat``. This will provide a Windows command
prompt with an environment properly set up with Spack and its prerequisites.
Ensure that Python and CMake are on your PATH. If needed, you may add the
CMake executable provided by Visual Studio to your path, which will look
something like:
``C:\Program Files (x86)\Microsoft Visual Studio\<year>\<distribution>\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake``
**IMPORTANT**: If you use Tab to complete any part of this path, the console
will automatically add quotation marks to the start and the end since it will
see the spaces and want to parse the whole of it as a string. This is
incorrect for our purposes so before submitting the command, ensure that the
quotes are removed. You will encounter configuration errors if you fail to do
this.
There are two ways to create the installer using Spack's ``make-installer``
command. The recommended method is to build the installer using a local
checkout of Spack source (release or development), using the
`-s` flag to specify the directory where the local checkout is. For
example, if the local checkout is in a directory called ``spack-develop``
and want to generate an installer with the source there, you can use:
``spack make-installer -s spack-develop tmp``
Both the Spack source directory (e.g. ``spack-develop``) and installer
destination directory (e.g. ``tmp``) may be an absolute path or relative to
the current working directory. The entire contents of the specified
directory will be included in the installer (e.g. .git files or local
changes).
Alternatively, if you would like to create an installer from a release version
of Spack, say, 0.16.0, and store it in ``tmp``, you can use the following
command:
``spack make-installer -v 0.16.0 tmp``
**IMPORTANT**: Windows features are not currently supported in Spack's
official release branches, so an installer created using this method will
*not* run on Windows.
Regardless of your method, a file called ``Spack.exe`` will be created
inside the destination directory. This executable bundles the Spack installer
(``Spack.msi`` also located in destination directory) and the git installer.
# Step 3: Run the installer
After accepting the terms of service, select where on your computer you would
like Spack installed, and after a few minutes Spack, Python and git will be
installed and ready for use.
**IMPORTANT**: To avoid permissions issues, it is recommended to select an
install location other than ``C:\Program Files``.
**IMPORTANT**: There is a specific option that must be chosen when letting Git
install. When given the option of adjusting your ``PATH``, choose the
``Git from the command line and also from 3rd-party software`` option. This will
automatically update your ``PATH`` variable to include the ``git`` command.
Certain Spack commands expect ``git`` to be part of the ``PATH``. If this step
is not performed properly, certain Spack comands will not work.
If your Spack installation needs to be modified, repaired, or uninstalled,
you can do any of these things by rerunning ``Spack.exe``.
Running the installer creates a shortcut on your desktop that, when
launched, will run ``spack_cmd.bat`` and launch a console with its initial
directory being wherever Spack was installed on your computer. If Python is
found on your PATH, that will be used. If not, the Python included with the
installer will be used when running Spack.

BIN
lib/spack/spack/cmd/installer/banner493x58.bmp Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 29 KiB

23
lib/spack/spack/cmd/installer/bundle.wxs.in Normal file
View file

@ -0,0 +1,23 @@
<?xml version="1.0"?>
<Wix xmlns="http://schemas.microsoft.com/wix/2006/wi"
xmlns:bal="http://schemas.microsoft.com/wix/BalExtension">
<Bundle Version="1.0.0.0" UpgradeCode="63C4E213-0297-4CFE-BB7B-7A77EB68E966"
IconSourceFile="@CPACK_WIX_PRODUCT_ICON@"
Name="Spack Package Manager"
Manufacturer="Lawrence Livermore National Laboratory">
<BootstrapperApplicationRef Id="WixStandardBootstrapperApplication.RtfLicense">
<bal:WixStandardBootstrapperApplication LicenseFile="@CPACK_RESOURCE_FILE_LICENSE@"/>
</BootstrapperApplicationRef>
<Chain>
<MsiPackage
SourceFile="Spack.msi"
DisplayInternalUI="yes"/>
<ExePackage
SourceFile="Git-2.31.1-64-bit.exe"
DetectCondition="ExeDetectedVariable"
InstallCommand="@SPACK_GIT_VERBOSITY@ /SUPPRESSMSGBOXES"
RepairCommand="/VERYSILENT"
UninstallCommand="/VERYSILENT" />
</Chain>
</Bundle>
</Wix>

10
lib/spack/spack/cmd/installer/patch.xml Normal file
View file

@ -0,0 +1,10 @@
<?xml version="1.0" encoding="utf-8"?>
<CPackWiXPatch>
<CPackWiXFragment Id ="#PRODUCT">
<Property Id="DISABLEADVTSHORTCUTS" Value="1"/>
</CPackWiXFragment>
<CPackWiXFragment Id ="#PRODUCTFEATURE">
<ComponentGroupRef Id="ProductComponents" />
<ComponentRef Id="ProgramMenuDir"/>
</CPackWiXFragment>
</CPackWiXPatch>

20
lib/spack/spack/cmd/installer/scripts/haspywin.py Normal file
View file

@ -0,0 +1,20 @@
# Copyright 2013-2021 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
import subprocess
import sys
def getpywin():
try:
import win32con # noqa
except ImportError:
subprocess.check_call(
[sys.executable, "-m", "pip", "-q", "install", "--upgrade", "pip"])
subprocess.check_call(
[sys.executable, "-m", "pip", "-q", "install", "pywin32"])
if __name__ == '__main__':
getpywin()

223
lib/spack/spack/cmd/installer/scripts/spack.bat Normal file
View file

@ -0,0 +1,223 @@
:: Copyright 2013-2021 Lawrence Livermore National Security, LLC and other
:: Spack Project Developers. See the top-level COPYRIGHT file for details.
::
:: SPDX-License-Identifier: (Apache-2.0 OR MIT)
::#######################################################################
::
:: This file is part of Spack and sets up the spack environment for batch,
:: This includes environment modules and lmod support,
:: and it also puts spack in your path. The script also checks that at least
:: module support exists, and provides suggestions if it doesn't. Source
:: it like this:
::
:: . /path/to/spack/install/spack_cmd.bat
::
@echo off
set spack=%SPACK_ROOT%\bin\spack
::#######################################################################
:: This is a wrapper around the spack command that forwards calls to
:: 'spack load' and 'spack unload' to shell functions. This in turn
:: allows them to be used to invoke environment modules functions.
::
:: 'spack load' is smarter than just 'load' because it converts its
:: arguments into a unique Spack spec that is then passed to module
:: commands. This allows the user to use packages without knowing all
:: their installation details.
::
:: e.g., rather than requiring a full spec for libelf, the user can type:
::
:: spack load libelf
::
:: This will first find the available libelf module file and use a
:: matching one. If there are two versions of libelf, the user would
:: need to be more specific, e.g.:
::
:: spack load libelf@0.8.13
::
:: This is very similar to how regular spack commands work and it
:: avoids the need to come up with a user-friendly naming scheme for
:: spack module files.
::#######################################################################
:_sp_shell_wrapper
set "_sp_flags="
set "_sp_args="
set "_sp_subcommand="
setlocal enabledelayedexpansion
:: commands have the form '[flags] [subcommand] [args]'
:: flags will always start with '-', e.g. --help or -V
:: subcommands will never start with '-'
:: everything after the subcommand is an arg
for %%x in (%*) do (
set t="%%~x"
if "!t:~0,1!" == "-" (
if defined _sp_subcommand (
:: We already have a subcommand, processing args now
set "_sp_args=!_sp_args! !t!"
) else (
set "_sp_flags=!_sp_flags! !t!"
shift
)
) else if not defined _sp_subcommand (
set "_sp_subcommand=!t!"
shift
) else (
set "_sp_args=!_sp_args! !t!"
shift
)
)
:: --help, -h and -V flags don't require further output parsing.
:: If we encounter, execute and exit
if defined _sp_flags (
if NOT "%_sp_flags%"=="%_sp_flags:-h=%" (
python "%spack%" %_sp_flags%
exit /B 0
) else if NOT "%_sp_flags%"=="%_sp_flags:--help=%" (
python "%spack%" %_sp_flags%
exit /B 0
) else if NOT "%_sp_flags%"=="%_sp_flags:-V=%" (
python "%spack%" %_sp_flags%
exit /B 0
)
)
:: pass parsed variables outside of local scope. Need to do
:: this because delayedexpansion can only be set by setlocal
echo %_sp_flags%>flags
echo %_sp_args%>args
echo %_sp_subcommand%>subcmd
endlocal
set /p _sp_subcommand=<subcmd
set /p _sp_flags=<flags
set /p _sp_args=<args
set str_subcommand=%_sp_subcommand:"='%
set str_flags=%_sp_flags:"='%
set str_args=%_sp_args:"='%
if "%str_subcommand%"=="ECHO is off." (set "_sp_subcommand=")
if "%str_flags%"=="ECHO is off." (set "_sp_flags=")
if "%str_args%"=="ECHO is off." (set "_sp_args=")
del subcmd
del flags
del args
:: Filter out some commands. For any others, just run the command.
if "%_sp_subcommand%" == "cd" (
goto :case_cd
) else if "%_sp_subcommand%" == "env" (
goto :case_env
) else if "%_sp_subcommand%" == "load" (
goto :case_load
) else if "%_sp_subcommand%" == "unload" (
goto :case_load
) else (
goto :default_case
)
::#######################################################################
:case_cd
:: Check for --help or -h
:: TODO: This is not exactly the same as setup-env.
:: In setup-env, '--help' or '-h' must follow the cd
:: Here, they may be anywhere in the args
if defined _sp_args (
if NOT "%_sp_args%"=="%_sp_args:--help=%" (
python "%spack%" cd -h
goto :end_switch
) else if NOT "%_sp_args%"=="%_sp_args:-h=%" (
python "%spack%" cd -h
goto :end_switch
)
)
for /F "tokens=* USEBACKQ" %%F in (
`python "%spack%" location %_sp_args%`) do (
set "LOC=%%F"
)
for %%Z in ("%LOC%") do if EXIST %%~sZ\NUL (cd /d "%LOC%")
goto :end_switch
:case_env
:: If no args or args contain --bat or -h/--help: just execute.
if NOT defined _sp_args (
goto :default_case
)else if NOT "%_sp_args%"=="%_sp_args:--help=%" (
goto :default_case
) else if NOT "%_sp_args%"=="%_sp_args: -h=%" (
goto :default_case
) else if NOT "%_sp_args%"=="%_sp_args:--bat=%" (
goto :default_case
) else if NOT "%_sp_args%"=="%_sp_args:deactivate=%" (
for /f "tokens=* USEBACKQ" %%I in (
`call python "%spack%" %_sp_flags% env deactivate --bat %_sp_args:deactivate=%`
) do %%I
) else if NOT "%_sp_args%"=="%_sp_args:activate=%" (
for /f "tokens=* USEBACKQ" %%I in (
`call python "%spack%" %_sp_flags% env activate --bat %_sp_args:activate=%`
) do %%I
) else (
goto :default_case
)
goto :end_switch
:case_load
:: If args contain --sh, --csh, or -h/--help: just execute.
if defined _sp_args (
if NOT "%_sp_args%"=="%_sp_args:--help=%" (
goto :default_case
) else if NOT "%_sp_args%"=="%_sp_args: -h=%" (
goto :default_case
) else if NOT "%_sp_args%"=="%_sp_args:--bat=%" (
goto :default_case
)
)
for /f "tokens=* USEBACKQ" %%I in (
`python "%spack%" %_sp_flags% %_sp_subcommand% --bat %_sp_args%`) do %%I
)
goto :end_switch
:case_unload
goto :case_load
:default_case
python "%spack%" %_sp_flags% %_sp_subcommand% %_sp_args%
goto :end_switch
:end_switch
exit /B 0
::########################################################################
:: Prepends directories to path, if they exist.
:: pathadd /path/to/dir # add to PATH
:: or pathadd OTHERPATH /path/to/dir # add to OTHERPATH
::########################################################################
:_spack_pathadd
set "_pa_varname=PATH"
set "_pa_new_path=%~1"
if NOT "%~2" == "" (
set "_pa_varname=%~1"
set "_pa_new_path=%~2"
)
set "_pa_oldvalue=%_pa_varname%"
for %%Z in ("%_pa_new_path%") do if EXIST %%~sZ\NUL (
if defined %_pa_oldvalue% (
set "_pa_varname=%_pa_new_path%:%_pa_oldvalue%"
) else (
set "_pa_varname=%_pa_new_path%"
)
)
exit /b 0
:: set module system roots
:_sp_multi_pathadd
for %%I in (%~2) do (
for %%Z in (%_sp_compatible_sys_types%) do (
:pathadd "%~1" "%%I\%%Z"
)
)
exit /B 0

48
lib/spack/spack/cmd/installer/spack.wxs.in Normal file
View file

@ -0,0 +1,48 @@
<?xml version="1.0" encoding="UTF-8"?>
<Wix xmlns="http://schemas.microsoft.com/wix/2006/wi">
<Fragment>
<DirectoryRef Id="TARGETDIR">
<Directory Id="DesktopFolder" Name="Desktop" />
<Directory Id="ProgramMenuFolder" Name="Programs">
<Directory Id="ApplicationProgramsFolder" Name="Spack">
<Component Id="ProgramMenuDir" Guid="*">
<RemoveFolder Id="ProgramMenuDir" On="uninstall"/>
<RegistryValue Root="HKMU" Key="Software\LLNL\Spack"
Type="integer" Value="1" Name="installed" KeyPath="yes" />
</Component>
</Directory>
</Directory>
</DirectoryRef>
</Fragment>
<Fragment>
<Icon Id="icon.ico" SourceFile="@CPACK_WIX_PRODUCT_ICON@"/>
<Property Id="ARPPRODUCTICON" Value="icon.ico" />
</Fragment>
<Fragment>
<!-- Add the shortcut to installer package -->
<ComponentGroup Id="ProductComponents" Directory="INSTALL_ROOT">
<Component Id="ApplicationShortcut" Guid="@CPACK_WIX_UPGRADE_GUID@">
<File Source="@SPACK_SHORTCUT@" KeyPath="yes">
<Shortcut Id="SpackStartMenuShortcut"
Directory="DesktopFolder"
Advertise="yes"
Name="Spack Package Manager"
Description="Spack package manager"
WorkingDirectory="INSTALL_ROOT"
Icon="icon1.ico">
<Icon Id="icon1.ico" SourceFile="@CPACK_WIX_PRODUCT_ICON@" />
</Shortcut>
<Shortcut Id="startMenuShotcut"
Directory="ApplicationProgramsFolder"
Name="Spack Package Manager"
WorkingDirectory="INSTALL_ROOT"
Icon="icon2.ico"
IconIndex="0"
Advertise="yes">
<Icon Id="icon2.ico" SourceFile="@CPACK_WIX_PRODUCT_ICON@" />
</Shortcut>
</File>
</Component>
</ComponentGroup>
</Fragment>
</Wix>

63
lib/spack/spack/cmd/installer/spack_cmd.bat Normal file
View file

@ -0,0 +1,63 @@
@ECHO OFF
setlocal EnableDelayedExpansion
:: (c) 2021 Lawrence Livermore National Laboratory
:: To use this file independently of Spack's installer, please copy this file, and the
:: 'scripts' directory, to be adjacent to your spack directory. You must have python on
:: your path for Spack to locate it.
:: source_dir -------- spack
:: |--- scripts
:: |--- spack_cmd.bat
pushd %~dp0
set spackinstdir=%CD%
popd
:: Check if Python is on the PATH
(for /f "delims=" %%F in ('where python.exe') do (set python_pf_ver=%%F) ) 2> NUL
if not defined python_pf_ver (
:: If not, look for Python from the Spack installer
:get_builtin
(for /f "tokens=*" %%g in ('dir /b /a:d "!spackinstdir!\Python*"') do (
set python_ver=%%g)) 2> NUL
if not defined python_ver (
echo Python was not found on your system.
echo Please install Python or add Python to your PATH.
) else (
set py_path=!spackinstdir!\!python_ver!
set py_exe=!py_path!\python.exe
)
goto :exitpoint
) else (
:: Python is already on the path
set py_exe=!python_pf_ver!
(for /F "tokens=* USEBACKQ" %%F in (
`!py_exe! --version`) do (set "output=%%F")) 2>NUL
if not "!output:Microsoft Store=!"=="!output!" goto :get_builtin
goto :exitpoint
)
:exitpoint
for /f "tokens=*" %%g in ('dir /b /a:d "%spackinstdir%\spack*"') do (set spack_ver=%%g)
set "SPACK_ROOT=%spackinstdir%\%spack_ver%"
set "PATH=%spackinstdir%\scripts\;%PATH%"
if defined py_path (
set "PATH=%py_path%;%PATH%"
)
if defined py_exe (
"%py_exe%" "%spackinstdir%\scripts\haspywin.py"
"%py_exe%" "%SPACK_ROOT%\bin\spack" external find python >NUL
)
set "EDITOR=notepad"
DOSKEY spacktivate=spack env activate $*
@echo **********************************************************************
@echo ** Spack Package Manager
@echo **********************************************************************
%comspec% /k

3
lib/spack/spack/cmd/load.py
View file

@ -33,6 +33,9 @@ def setup_parser(subparser):
shells.add_argument(
'--fish', action='store_const', dest='shell', const='fish',
help="print fish commands to load the package")
shells.add_argument(
'--bat', action='store_const', dest='shell', const='bat',
help="print bat commands to load the package")
subparser.add_argument(
'--first',

137
lib/spack/spack/cmd/make_installer.py Normal file
View file

@ -0,0 +1,137 @@
# Copyright 2013-2021 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
import os
import posixpath
import subprocess
import sys
import spack.paths
import spack.util.executable
from spack.spec import Spec
description = "generate Windows installer"
section = "admin"
level = "long"
def txt_to_rtf(file_path):
rtf_header = """{{\\rtf1\\ansi\\deff0\\nouicompat
{{\\fonttbl{{\\f0\\fnil\\fcharset0 Courier New;}}}}
{{\\colortbl ;\\red0\\green0\\blue255;}}
{{\\*\\generator Riched20 10.0.19041}}\\viewkind4\\uc1
\\f0\\fs22\\lang1033
{}
}}
"""
def line_to_rtf(str):
return str.replace("\n", "\\par")
contents = ""
with open(file_path, "r+") as f:
for line in f.readlines():
contents += line_to_rtf(line)
return rtf_header.format(contents)
def setup_parser(subparser):
spack_source_group = subparser.add_mutually_exclusive_group(required=True)
spack_source_group.add_argument(
'-v', '--spack_version', default="",
help='download given spack version e.g. 0.16.0')
spack_source_group.add_argument(
'-s', '--spack_source', default="",
help='full path to spack source')
subparser.add_argument(
'-g', '--git-installer-verbosity', default="",
choices=set(['SILENT', 'VERYSILENT']),
help="Level of verbosity provided by bundled Git Installer.\
Default is fully verbose",
required=False, action='store', dest="git_verbosity"
)
subparser.add_argument(
'output_dir', help="output directory")
def make_installer(parser, args):
"""
Use CMake to generate WIX installer in newly created build directory
"""
if sys.platform == 'win32':
output_dir = args.output_dir
cmake_spec = Spec('cmake')
cmake_spec.concretize()
cmake_path = os.path.join(cmake_spec.prefix, "bin", "cmake.exe")
cpack_path = os.path.join(cmake_spec.prefix, "bin", "cpack.exe")
spack_source = args.spack_source
git_verbosity = ""
if args.git_verbosity:
git_verbosity = "/" + args.git_verbosity
if spack_source:
if not os.path.exists(spack_source):
print("%s does not exist" % spack_source)
return
else:
if not os.path.isabs(spack_source):
spack_source = posixpath.abspath(spack_source)
spack_source = spack_source.replace('\\', '/')
spack_version = args.spack_version
here = os.path.dirname(os.path.abspath(__file__))
source_dir = os.path.join(here, "installer")
posix_root = spack.paths.spack_root.replace('\\', '/')
spack_license = posixpath.join(posix_root, "LICENSE-APACHE")
rtf_spack_license = txt_to_rtf(spack_license)
spack_license = posixpath.join(source_dir, "LICENSE.rtf")
with open(spack_license, 'w') as rtf_license:
written = rtf_license.write(rtf_spack_license)
if written == 0:
raise RuntimeError("Failed to generate properly formatted license file")
spack_logo = posixpath.join(posix_root,
"share/spack/logo/favicon.ico")
try:
subprocess.check_call(
('"%s" -S "%s" -B "%s" -DSPACK_VERSION=%s '
'-DSPACK_SOURCE="%s" -DSPACK_LICENSE="%s" '
'-DSPACK_LOGO="%s" -DSPACK_GIT_VERBOSITY="%s"')
% (cmake_path, source_dir, output_dir, spack_version, spack_source,
spack_license, spack_logo, git_verbosity),
shell=True)
except subprocess.CalledProcessError:
print("Failed to generate installer")
return subprocess.CalledProcessError.returncode
try:
subprocess.check_call(
'"%s" --config "%s/CPackConfig.cmake" -B "%s/"'
% (cpack_path, output_dir, output_dir),
shell=True)
except subprocess.CalledProcessError:
print("Failed to generate installer")
return subprocess.CalledProcessError.returncode
try:
subprocess.check_call(
'"%s/bin/candle.exe" -ext WixBalExtension "%s/bundle.wxs"'
' -out "%s/bundle.wixobj"'
% (os.environ.get('WIX'), output_dir, output_dir), shell=True)
except subprocess.CalledProcessError:
print("Failed to generate installer chain")
return subprocess.CalledProcessError.returncode
try:
subprocess.check_call(
'"%s/bin/light.exe" -sw1134 -ext WixBalExtension "%s/bundle.wixobj"'
' -out "%s/Spack.exe"'
% (os.environ.get('WIX'), output_dir, output_dir), shell=True)
except subprocess.CalledProcessError:
print("Failed to generate installer chain")
return subprocess.CalledProcessError.returncode
print("Successfully generated Spack.exe in %s" % (output_dir))
else:
print('The make-installer command is currently only supported on Windows.')

3
lib/spack/spack/cmd/unload.py
View file

@ -32,6 +32,9 @@ def setup_parser(subparser):
shells.add_argument(
'--fish', action='store_const', dest='shell', const='fish',
help="print fish commands to load the package")
shells.add_argument(
'--bat', action='store_const', dest='shell', const='bat',
help="print bat commands to load the package")
subparser.add_argument('-a', '--all', action='store_true',
help='unload all loaded Spack packages.')

10
lib/spack/spack/environment/shell.py
View file

@ -37,6 +37,11 @@ def activate_header(env, shell, prompt=None):
# solution to the PS1 variable) here. This is a bit fiddly, and easy to
# screw up => spend time reasearching a solution. Feedback welcome.
#
elif shell == 'bat':
# TODO: Color
cmds += 'set "SPACK_ENV=%s"\n' % env.path
# TODO: despacktivate
# TODO: prompt
else:
if 'color' in os.getenv('TERM', '') and prompt:
prompt = colorize('@G{%s}' % prompt, color=True)
@ -69,6 +74,11 @@ def deactivate_header(shell):
#
# NOTE: Not changing fish_prompt (above) => no need to restore it here.
#
elif shell == 'bat':
# TODO: Color
cmds += 'set "SPACK_ENV="\n'
# TODO: despacktivate
# TODO: prompt
else:
cmds += 'if [ ! -z ${SPACK_ENV+x} ]; then\n'
cmds += 'unset SPACK_ENV; export SPACK_ENV;\n'

2
lib/spack/spack/util/editor.py
View file

@ -21,7 +21,7 @@
from spack.util.executable import which_string
#: editors to try if VISUAL and EDITOR are not set
_default_editors = ['vim', 'vi', 'emacs', 'nano']
_default_editors = ['vim', 'vi', 'emacs', 'nano', 'notepad']
def _find_exe_from_env_var(var):

27
lib/spack/spack/util/environment.py
View file

@ -36,7 +36,8 @@
_shell_set_strings = {
'sh': 'export {0}={1};\n',
'csh': 'setenv {0} {1};\n',
'fish': 'set -gx {0} {1};\n'
'fish': 'set -gx {0} {1};\n',
'bat': 'set "{0}={1}"\n'
}
@ -44,6 +45,7 @@
'sh': 'unset {0};\n',
'csh': 'unsetenv {0};\n',
'fish': 'set -e {0};\n',
'bat': 'set "{0}="\n'
}
@ -83,7 +85,7 @@ def prune_duplicate_paths(paths):
def get_path(name):
path = os.environ.get(name, "").strip()
if path:
return path.split(":")
return path.split(os.pathsep)
else:
return []
@ -96,7 +98,7 @@ def env_flag(name):
def path_set(var_name, directories):
path_str = ":".join(str(dir) for dir in directories)
path_str = os.pathsep.join(str(dir) for dir in directories)
os.environ[var_name] = path_str
@ -104,7 +106,7 @@ def path_put_first(var_name, directories):
"""Puts the provided directories first in the path, adding them
if they're not already there.
"""
path = os.environ.get(var_name, "").split(':')
path = os.environ.get(var_name, "").split(os.pathsep)
for dir in directories:
if dir in path:
@ -214,7 +216,7 @@ class NameModifier(object):
def __init__(self, name, **kwargs):
self.name = name
self.separator = kwargs.get('separator', ':')
self.separator = kwargs.get('separator', os.pathsep)
self.args = {'name': name, 'separator': self.separator}
self.args.update(kwargs)
@ -234,7 +236,7 @@ class NameValueModifier(object):
def __init__(self, name, value, **kwargs):
self.name = name
self.value = value
self.separator = kwargs.get('separator', ':')
self.separator = kwargs.get('separator', os.pathsep)
self.args = {'name': name, 'value': value, 'separator': self.separator}
self.args.update(kwargs)
@ -634,8 +636,13 @@ def shell_modifications(self, shell='sh', explicit=False, env=None):
if new is None:
cmds += _shell_unset_strings[shell].format(name)
else:
cmds += _shell_set_strings[shell].format(
name, cmd_quote(new_env[name]))
if sys.platform != "win32":
cmd = _shell_set_strings[shell].format(
name, cmd_quote(new_env[name]))
else:
cmd = _shell_set_strings[shell].format(
name, new_env[name])
cmds += cmd
return cmds
@staticmethod
@ -815,13 +822,13 @@ def return_separator_if_any(*args):
return env
def concatenate_paths(paths, separator=':'):
def concatenate_paths(paths, separator=os.pathsep):
"""Concatenates an iterable of paths into a string of paths separated by
separator, defaulting to colon.
Args:
paths: iterable of paths
separator: the separator to use, default ':'
separator: the separator to use, default ';' windows, ':' otherwise
Returns:
string

11
share/spack/spack-completion.bash
View file

@ -337,7 +337,7 @@ _spack() {
then
SPACK_COMPREPLY="-h --help -H --all-help --color -c --config -C --config-scope -d --debug --show-cores --timestamp --pdb -e --env -D --env-dir -E --no-env --use-env-repo -k --insecure -l --enable-locks -L --disable-locks -m --mock -b --bootstrap -p --profile --sorted-profile --lines -v --verbose --stacktrace -V --version --print-shell-vars"
else
SPACK_COMPREPLY="activate add analyze arch audit blame bootstrap build-env buildcache cd checksum ci clean clone commands compiler compilers concretize config containerize create deactivate debug dependencies dependents deprecate dev-build develop diff docs edit env extensions external fetch find gc gpg graph help info install license list load location log-parse maintainers mark mirror module monitor patch pkg providers pydoc python reindex remove rm repo resource restage solve spec stage style tags test test-env tutorial undevelop uninstall unit-test unload url verify versions view"
SPACK_COMPREPLY="activate add analyze arch audit blame bootstrap build-env buildcache cd checksum ci clean clone commands compiler compilers concretize config containerize create deactivate debug dependencies dependents deprecate dev-build develop diff docs edit env extensions external fetch find gc gpg graph help info install license list load location log-parse maintainers make-installer mark mirror module monitor patch pkg providers pydoc python reindex remove rm repo resource restage solve spec stage style tags test test-env tutorial undevelop uninstall unit-test unload url verify versions view"
fi
}
@ -1238,6 +1238,15 @@ _spack_maintainers() {
fi
}
_spack_make_installer() {
if $list_options
then
SPACK_COMPREPLY="-h --help -v --spack_version -s --spack_source"
else
SPACK_COMPREPLY=""
fi
}
_spack_mark() {
if $list_options
then