Tunnels¶
Tunnels are the lowest-level API, used for invoking commands on an individual host or container. For a higher-level API that allows invoking commands in parallel across a range of hosts, see Groups.
An established tunnel can be used to invoke commands and receive results.
Tunnel reference¶
All tunnels support the following methods:
-
class
chopsticks.tunnel.
BaseTunnel
[source]¶ -
call
(callable, *args, **kwargs)[source]¶ Call the given callable on the remote host.
The callable must return a value that can be serialised as JSON, but there is no such restriction on the parameters.
-
close
()[source]¶ Disconnect the tunnel.
Note that this will terminate the remote process and any state will be lost. This does not destroy the Tunnel object, which can be reconnected with
connect()
.
-
fetch
(remote_path, local_path=None)[source]¶ Fetch one file from the remote host.
If local_path is given, it is the local path to write to. Otherwise, a temporary filename will be used.
This operation supports arbitarily large files (file data is streamed, not buffered in memory).
The return value is a dict containing:
local_path
- the local path written toremote_path
- the absolute remote pathsize
- the number of bytes receivedsha1sum
- a sha1 checksum of the file data
-
put
(local_path, remote_path=None, mode=420)[source]¶ Copy a file to the remote host.
If remote_path is given, it is the remote path to write to. Otherwise, a temporary filename will be used.
mode gives is the permission bits of the file to create, or 0o644 if unspecified.
This operation supports arbitarily large files (file data is streamed, not buffered in memory).
The return value is a dict containing:
remote_path
- the absolute remote pathsize
- the number of bytes receivedsha1sum
- a sha1 checksum of the file data
-
SSH¶
-
class
chopsticks.tunnel.
SSHTunnel
(host, user=None, sudo=False)[source]¶ A tunnel that connects to a remote host over SSH.
Parameters: - host – The hostname to connect to, as would be specified on an
ssh
command line. - user – The username to connect as.
- sudo – If true, use
sudo
on the remote end in order to run as theroot
user. Use this when you cansudo
to root but notssh
directly as the root user.
- host – The hostname to connect to, as would be specified on an
Docker¶
-
class
chopsticks.tunnel.
Docker
(name, image='python:2.7', rm=True)[source]¶ A tunnel connected to a throwaway Docker container.
Parameters: - name – The name of the Docker instance to create.
- image – The Docker image to launch. By default, download and run an official Docker Python image corresponding to the running Python version. Official images are curated by Docker.
- rm – If true, destroy the container when the tunnel is closed.
Subprocess¶
Sudo¶
-
class
chopsticks.tunnel.
Sudo
(user='root', name=None)[source]¶ A tunnel to a process on the same host, launched with sudo.
The Sudo
tunnel does not deal with password dialogues etc. In order
for this to work you must configure sudo
not to need a password. You can
do this with these lines in /etc/sudoers
:
Cmnd_Alias PYTHON_CMDS = /usr/bin/python, /usr/bin/python2, /usr/bin/python3
%somegroup ALL=NOPASSWD: PYTHON_CMDS
This would allow users in the group somegroup
to be able
to run the system Python interpreters using sudo, without
passwords.
Warning
Naturally, as Chopsticks is a framework for executing arbitrary code, this allows executing arbitrary code as root. Only make this change if you are happy with relaxing security in this way.
Writing new tunnels¶
It is possible to write a new tunnel driver for any system that allows you to
execute a python
binary with direct relay of stdin
and stdout
pipes. To do this, simply subclass chopsticks.group.SubprocessTunnel
. Note
that all tunnel instances must have a host
attibute which is used as the
key for the result in the GroupResult
dictionary when executing tasks
in a Group
.
So, strictly, these requirements apply:
- The tunnel setup machinery should not write to
stdout
- else you will have to identify and consume this output. - The tunnel setup machinery should not read from
stdin
- else you will have to feed the required input. - Both
stdin
andstdout
must be binary-safe pipes.
The tunnel machinery may write to stderr
; this output will be presented to
the user.
Recursively tunnelling¶
Chopsticks can be imported and used on the remote side of a tunnel. This situation is called recursive tunnelling, and it has its uses. For example:
- You could create an
SSHTunnel
to a remote host and thenSudo
to execute certain actions as root. - You could maintain a group of
SSHTunnels
to physical hosts, that each construct a pool ofDocker
tunnels - for an instant cluster.
Recursion could be dangerous. For example, consider this function:
def recursive():
with Local() as tun:
tun.call(recursive)
This would effectively fork-bomb your host! To avoid this pitfall, Chopsticks has a built-in depth limit of 2. You can override this limit by setting
chopsticks.DEPTH_LIMIT = 3
Caution
Do not write
chopsticks.DEPTH_LIMIT += 1
This will undo the limiting!