Tunnels

Tunnels are the lowest-level API, used for invoking commands on an individual host or container. For a higher-level API that allows invoking commands in parallel across a range of hosts, see Groups.

An established tunnel can be used to invoke commands and receive results.

SSH

class chopsticks.tunnel.SSHTunnel(host, user=None)[source]

A tunnel that connects to a remote host over SSH.

call(callable, *args, **kwargs)

Call the given callable on the remote host.

The callable must return a value that can be serialised as JSON, but there is no such restriction on the parameters.

fetch(remote_path, local_path=None)

Fetch one file from the remote host.

If local_path is given, it is the local path to write to. Otherwise, a temporary filename will be used.

This operation supports arbitarily large files (file data is streamed, not buffered in memory).

The return value is a dict containing:

  • local_path - the local path written to
  • remote_path - the absolute remote path
  • size - the number of bytes received
  • sha1sum - a sha1 checksum of the file data
put(local_path, remote_path=None, mode=420)

Copy a file to the remote host.

If remote_path is given, it is the remote path to write to. Otherwise, a temporary filename will be used.

mode gives is the permission bits of the file to create, or 0o644 if unspecified.

This operation supports arbitarily large files (file data is streamed, not buffered in memory).

The return value is a dict containing:

  • remote_path - the absolute remote path
  • size - the number of bytes received
  • sha1sum - a sha1 checksum of the file data
chopsticks.tunnel.Tunnel

alias of SSHTunnel

Docker

class chopsticks.tunnel.Docker(name, image='python:2.7', rm=True)[source]

A tunnel connected to a throwaway Docker container.

Subprocess

class chopsticks.tunnel.Local(name='localhost')[source]

A tunnel to a subprocess on the same host.

Writing new tunnels

It is possible to write a new tunnel driver for any system that allows you to execute a python binary with direct relay of stdin and stdout pipes. To do this, simply subclass chopsticks.group.SubprocessTunnel. Note that all tunnel instances must have a host attibute which is used as the key for the result in the GroupResult dictionary when executing tasks in a Group.

So, strictly, these requirements apply:

  • The tunnel setup machinery should not write to stdout - else you will have to identify and consume this output.
  • The tunnel setup machinery should not read from stdin - else you will have to feed the required input.
  • Both stdin and stdout must be binary-safe pipes.

The tunnel machinery may write to stderr; this output will be presented to the user.