96 lines
5 KiB
Text
96 lines
5 KiB
Text
# Best Practice -- Connecting Hawk to the world
|
|
|
|
Hawk is surrounded by a strict outgoing firewall. Users can not open arbitrary connections to hosts on the internet. Connections can only be established through a proxy host. This document describes how to set up a proxy, and how to use this proxy in a number of common use cases.
|
|
|
|
Common use cases include:
|
|
- working on remote repositories with git
|
|
- installing Python packages from PyPi, etc
|
|
- access to package registries for languages such as Golang, Rust, etc.
|
|
|
|
|
|
When following this guide, you will have to execute actions or command on Hawk, but also on your local workstation or laptop. For clarity, examples below will use prompts `local>` and `hawk>`, respectively.
|
|
|
|
|
|
## Setting up a proxy with ssh
|
|
|
|
Please note, that there is numerous ways to set up proxies. However, HLRS only supports the method describe in the following.
|
|
|
|
Ssh allows to create reverse tunnels from Hawk back to your local machine. In addition, it can run a proxy, more specifically a SOCKS5 proxy, on the local end of this tunnel. SOCKS5 is a widely supported standard which allows many applications to use such proxies to connect to remote hosts.
|
|
|
|
Note however, that the tunnel is open on the remote side. Every user on Hawk will be able to use the tunnel and thus have a network connection to your local network. We strongly suggest to keep the tunnel open only when necessary.
|
|
|
|
Tunnels are associated to _ports_. In the following, you will have to specify a port which is referred by the environment variable `$MY_PROXY_PORT`. You may choose the port numbers freely in the suggested range 10000 to 60000. It may happen, that a port is currently used by another user ... just choose a different one.
|
|
|
|
Ok, let's do it.
|
|
|
|
The following will open a regular ssh connection to Hawk. In addition, the argument `-D $MY_PROXY_PORT` will establish a reverse dynamic port forwarding tunnel which opens at port `$MY_PROXY_PORT` on the Hawk side and connects to a SOCKS5 proxy on your local side.
|
|
```bash
|
|
local> MY_PROXY_PORT=XXXX # choose a port number between 10000 and 60000
|
|
local> ssh -D $MY_PROXY_PORT hawk.hww.hlrs.de
|
|
```
|
|
|
|
The tunnel will remain open as long as the connection to Hawk is open. The tunnel will close as soon as your connection closes. Please note, that we do recommend to _not use_ command line arguments to keep the connection open in the background as you may loose track of open reverse tunnels.
|
|
|
|
Once you proxy is established, you may proceed to use it for various tasks.
|
|
|
|
|
|
## HTTPS/HTTP proxy variables
|
|
|
|
Requirements:
|
|
- set up a proxy with ssh as describe in [this section](#setting-up-a-proxy-with-ssh)
|
|
|
|
Most applications which support https or http protocols can use proxies. Most of them will take the proxy from the environment variables `$https_proxy` and `$http_proxy`. Sometimes, you will have to specify the proxy through command line arguments.
|
|
|
|
Example usage:
|
|
```bash
|
|
# set up proxy with ssh and continue in the terminal which just opened
|
|
hawk> MY_PROXY_PORT=XXXX # choose same port as above
|
|
|
|
hawk> curl ifconfig.me # timeout, no connection to internet
|
|
|
|
hawk> curl --proxy socks5://localhost:$MY_PROXY_PORT # specify proxy through command line argument
|
|
|
|
hawk> export https_proxy=socks5://localhost:$MY_PROXY_PORT # set proxy environment variables
|
|
hawk> export http_proxy=$https_proxy
|
|
hawk> curl ifconfig.me
|
|
|
|
hawk> # close terminal to close reverse tunnel when done
|
|
```
|
|
|
|
Most of the other use cases will build on the usage of these environment variables. Therefore we recommend to set them in you `.bashrc`. ==**TODO: do we recommend this?**==
|
|
|
|
## Installing Python packages with pip
|
|
|
|
Requirements:
|
|
- set up a proxy with ssh as describe in [this section](#setting-up-a-proxy-with-ssh)
|
|
- set https proxy variables as describe in [this section](#httpshttp-proxy-variables)
|
|
|
|
Python's package manager `pip` will honour proxy settings through environment variables or command line arguments. However, it will need the package `pysocks` to do so. Unfortunately, `pysocks` is not a part of the core of Python and you may end up without it (for instance if you create a Python environment without including the system site-packages). Note, that you might have to add the command line argument `--user` to pip commands below.
|
|
|
|
```bash
|
|
# set up proxy with ssh and continue in the terminal which just opened
|
|
hawk> MY_PROXY_PORT=XXXX # choose same port as above
|
|
|
|
hawk> python3 -m pip install six # timeout, no connection to internet
|
|
|
|
hawk> python3 -m pip --proxy socks5://localhost:$MY_PROXY_PORT
|
|
|
|
# set https/http proxy environment variables as describe above, then
|
|
hawk> python3 -m pip install six
|
|
```
|
|
|
|
If you get error messages related to missing SOCKS5 support in pip, do
|
|
```bash
|
|
# set up proxy with ssh and continue in the terminal which just opened
|
|
# set https/http proxy environment variables as describe above, then
|
|
|
|
hawk> python3 -m pip install /sw/general/x86_64/development/python/share/PySocks-1.7.1-py3-none-any.whl
|
|
```
|
|
|
|
## Using proxy with git
|
|
|
|
Two protocols: https and ssh
|
|
|
|
Push works with https protocol as well, see below.
|
|
|
|
### git https protocol
|