Update dashboard

README.md

@@ -7,3 +7,9 @@ Anything related to SPMT.
 Guides:
   - [Best Practice -- Storage on Hawk](guides/Best_practice--Storage_on_Hawk.md)
   - [Best Practice -- Connecting Hawk to the world](guides/Best_practice--Connecting_Hawk_to_the_world.md)
+
+
+Scripts and environment snippets:
+  - [ws_quickcheck](envs/ws_quickcheck)   Add this to your `.bashrc` for quick summary of you workspaces on login 
+  - [borg_daily](scripts/borg_daily)  Script for daily backups with borg
+

scripts/borg_daily/README.md

@@ -0,0 +1,19 @@
+# borg_daily
+
+This is a script to do borg backups on the SPMT backup server. Old backups are thinned out (pruned) progressively.
+
+Either manually run the script (more or less) daily or automatically through crontab or similar etc.
+
+Borg backup needs the passphrase / key of your repository. In the script, there is two options to provide it:
+1. set variable `BORG_PASSPHRASE` verbatim
+2. provide a command in `BORG_PASSCOMMAND` to generate / load the passphrase programmatically
+
+I prefer the second option. However, providing the passphrase programmatically safely is non-trivial, see [here](https://borgbackup.readthedocs.io/en/stable/faq.html#how-can-i-specify-the-encryption-passphrase-programmatically).
+
+I have chosen to do this quite simple: select a password and store an obfuscated version of it in `$HOME/.borg_passphrase`. The provided command just undoes the obfuscation. So, before using the backup script do the following on your command line
+```bash
+echo "MY SUPER SECRET PASSWORD ... ACTUALLY NOT SO SECRET AS IT END UP IN YOUR HISTORY" | \
+   base64 > $HOME/.borg_passphrase
+```
+
+Note, that it is also possible to store the password in your GNOME/macOS Keyring, or other password managers, or even on your USB crypto token. See the link above.

scripts/borg_daily/borg_daily.sh

@@ -0,0 +1,71 @@
+#!/bin/sh
+
+# Account on spmt-backup
+SPMT_USER=hpcjgrac
+
+# Which directories should be backed up
+# INCLUDE_BACKUP_DIRS=($HOME/delme $HOME/bin)    # note: these are bsh arrays, keep the braces and escape spaces if necessary  
+INCLUDE_BACKUP_DIRS=($HOME)
+# and which should not 
+# EXCLUDE_BACKUP_DIRS=($HOME/delme)
+EXCLUDE_BACKUP_DIRS=($HOME/Library/Thunderbird/Profiles/ $HOME/delme $HOME/Downloads/ $HOME/Sync $HOME/work/ChEESE_audits $HOME/work/spack)
+
+# Retention policies, how many daily, weekly, monthly backups to keep.
+KEEP_DAILY=7
+KEEP_WEEKLY=4
+KEEP_MONTHLY=12
+KEEP_YEARLY=-1       # never delete yearly backups
+
+# Setup file with borg repo passphrase: `echo "MY SUPER PASSPHRASE" | base64 > $HOME/.borg_passphrase`
+export BORG_PASSCOMMAND="base64 -d $HOME/.borg_passphrase"
+# or use BORG_PASSPHRASE below.
+
+# See the section "Passphrase notes" for more infos.
+# export BORG_PASSPHRASE='XYZl0ngandsecurepa_55_phrasea&&123'
+
+
+#### Nothing else to be done below! Change at your onw risk.
+
+export BORG_REPO=ssh://spmt-backup/storage/backup/repos/${SPMT_USER}
+
+# some helpers and error handling:
+info() { printf "\n%s %s\n\n" "$( date )" "$*" >&2; }
+trap 'echo $( date ) Backup interrupted >&2; exit 2' INT TERM
+
+info "Starting backup"
+
+EXCLUDE_FROM=$(mktemp tmp.borg_dayly.XXXXX)
+#EXCLUDE_FROM=tmp.borg_dayly
+trap 'echo "Removing temporary files"; rm ${EXCLUDE_FROM}' EXIT
+for d in "${EXCLUDE_BACKUP_DIRS[@]}"; do
+  echo ${d} >> ${EXCLUDE_FROM}
+done
+
+
+# Backup the most important directories into an archive named after
+# the machine this script is currently running on
+
+borg create                         \
+    --verbose                       \
+    --filter AME                    \
+    --list                          \
+    --stats                         \
+    --show-rc                       \
+    --compression lz4               \
+    --exclude-caches                \
+    --exclude-from ${EXCLUDE_FROM}  \
+    ::'{hostname}-{now}'            \
+    ${INCLUDE_BACKUP_DIRS}
+
+
+# Prune old backups and compact repo
+
+borg prune                         \
+    --stats --verbose --list       \
+    --keep-daily=${KEEP_DAILY}     \
+    --keep-weekly=${KEEP_WEEKLY}   \
+    --keep-monthly=${KEEP_MONTHLY} \
+    --keep-yearly=${KEEP_YEARLY}   
+
+borg compact                       \
+    --verbose