From e686f1500e2397897c2e4f07f987873f71dc121d Mon Sep 17 00:00:00 2001 From: Scott Wittenburg Date: Thu, 18 Jun 2020 16:28:28 -0600 Subject: [PATCH] Update pipeline documentation to describe user-provided scripts --- lib/spack/docs/pipelines.rst | 90 +++++++++++++++++++++++++++++------- 1 file changed, 73 insertions(+), 17 deletions(-) diff --git a/lib/spack/docs/pipelines.rst b/lib/spack/docs/pipelines.rst index 342024d94c..8559702b6a 100644 --- a/lib/spack/docs/pipelines.rst +++ b/lib/spack/docs/pipelines.rst @@ -122,6 +122,10 @@ pipeline jobs. Concretizes the specs in the active environment, stages them (as described in :ref:`staging_algorithm`), and writes the resulting ``.gitlab-ci.yml`` to disk. +This sub-command takes two arguments, but the most useful is ``--output-file``, +which should be an absolute path (including file name) to the generated +pipeline, if the default (``./.gitlab-ci.yml``) is not desired. + .. _cmd-spack-ci-rebuild: ^^^^^^^^^^^^^^^^^^^^ @@ -132,6 +136,10 @@ This sub-command is responsible for ensuring a single spec from the release environment is up to date on the remote mirror configured in the environment, and as such, corresponds to a single job in the ``.gitlab-ci.yml`` file. +Rather than taking command-line arguments, this sub-command expects information +to be communicated via environment variables, which will typically come via the +``.gitlab-ci.yml`` job as ``variables``. + ------------------------------------ A pipeline-enabled spack environment ------------------------------------ @@ -245,7 +253,21 @@ runners known to the gitlab instance. For Docker executor type runners, the as well as an ``entrypoint`` to override whatever the default for that image is). For other types of runners the ``variables`` key will be useful to pass any information on to the runner that it needs to do its work (e.g. scheduler -parameters, etc.). +parameters, etc.). Any ``variables`` provided here will be added, verbatim, to +each job, unless the value takes on a special form. If the value has the form +``$env:``, then that variable will be inserted with ``var_value`` +first sampled from the environment during job generation. This can be useful +for propagating generation-time variables which normally would not be available +child build jobs. + +The ``runner-attributes`` section also allows users to supply custom ``script``, +``before_script``, and ``after_script`` sections to be applied to every job +scheduled on that runner. This allows users to do any custom preparation or +cleanup tasks that fit their particular workflow, as well as completely +customize the rebuilding of a spec if they so choose. We will never generate +a ``before_script`` or ``after_script`` for jobs, but if you do not provide +a custom ``script``, we will generate one for you that invokes +``spack ci rebuild``. .. _staging_algorithm: @@ -368,13 +390,15 @@ Using a custom spack in your pipeline If your runners will not have a version of spack ready to invoke, or if for some other reason you want to use a custom version of spack to run your pipelines, -this can be accomplished fairly simply. First, create CI environment variables -containing the url and branch/tag you want to clone (calling them, for example, -``SPACK_REPO`` and ``SPACK_REF``), use them to clone spack in your pre-ci -``before_script``, and finally pass those same values along to the workload -generation process via the ``spack-repo`` and ``spack-ref`` cli args. Here's -the ``generate-pipeline`` job from the top of this document, updated to clone -a custom spack and make sure the generated rebuild jobs will clone it too: +this section provides an example of how you could take advantage of +user-provided pipeline scripts to accomplish this fairly simply. First, you +could use the GitLab user interface to create CI environment variables +containing the url and branch/tag/SHA you want to clone (calling them, for +example, ``SPACK_REPO`` and ``SPACK_REF``), then use those in a custom shell +script invoked both from your pipeline generation job, as well in your rebuild +jobs. Here's the ``generate-pipeline`` job from the top of this document, +updated to invoke a custom shell script that will clone and source a custom +spack: .. code-block:: yaml @@ -382,12 +406,10 @@ a custom spack and make sure the generated rebuild jobs will clone it too: tags: - before_script: - - git clone ${SPACK_REPO} --branch ${SPACK_REF} - - . ./spack/share/spack/setup-env.sh + - ./cloneSpack.sh script: - spack env activate . - spack ci generate - --spack-repo ${SPACK_REPO} --spack-ref ${SPACK_REF} --output-file "${CI_PROJECT_DIR}/jobs_scratch_dir/pipeline.yml" after_script: - rm -rf ./spack @@ -395,13 +417,47 @@ a custom spack and make sure the generated rebuild jobs will clone it too: paths: - "${CI_PROJECT_DIR}/jobs_scratch_dir/pipeline.yml" +And that script could contain: -If the ``spack ci generate`` command receives those extra command line arguments, -then it adds similar ``before_script`` and ``after_script`` sections for each of -the ``spack ci rebuild`` jobs it generates (cloning and sourcing a custom -spack in the ``before_script`` and removing it again in the ``after_script``). -This gives you control over the version of spack used when the rebuild jobs -are actually run on the gitlab runner. +.. code-block:: bash + + #!/bin/bash + + git clone ${SPACK_REPO} + pushd ./spack + git checkout ${SPACK_REF} + popd + + . "./spack/share/spack/setup-env.sh" + + spack --version + +Finally, you would also want your generated rebuild jobs to clone that version +of spack, so you would update your ``spack.yaml`` from above as follows: + +.. code-block:: yaml + + spack: + ... + gitlab-ci: + mappings: + - match: + - os=ubuntu18.04 + runner-attributes: + tags: + - spack-kube + image: spack/ubuntu-bionic + before_script: + - ./cloneSpack.sh + script: + - spack -d ci rebuild + after_script: + - rm -rf ./spack + +Now all the generated rebuild jobs will use the same shell script to clone +spack before running their actual workload. Here we have also provided a +custom ``script`` because we want to run ``spack ci rebuild`` in debug mode +to get more information when builds fail. .. _ci_environment_variables: