Groups¶
Groups can be used to perform a remote operation in parallel across a number of hosts, and collect the results.
Group API¶
-
class
chopsticks.group.
Group
(hosts)[source]¶ A group of hosts, for performing operations in parallel.
-
__init__
(hosts)[source]¶ Construct a group from a list of tunnels or hosts.
hosts may contain hostnames - in which case the connections will be made via SSH using the default settings. Alternatively, it may contain tunnel instances.
-
call
(callable, *args, **kwargs)[source]¶ Call the given callable on all hosts in the group.
The given callable and parameters must be pickleable.
However, the callable’s return value has a tighter restriction: it must be serialisable as JSON, in order to ensure the orchestration host cannot be compromised through pickle attacks.
The return value is a
GroupResult
.
-
fetch
(remote_path, local_path=None)[source]¶ Fetch files from all remote hosts.
If local_path is given, it is a local path template, into which the tunnel’s
host
name will be substituted usingstr.format()
. Hostnames generated in this way must be unique.For example:
group.fetch('/etc/passwd', local_path='passwd-{host}')
If local_path is not given, a temporary file will be used for each host.
Return a
GroupResult
of dicts, each containing:local_path
- the local path written toremote_path
- the absolute remote pathsize
- the number of bytes receivedsha1sum
- a sha1 checksum of the file data
-
filter
(predicate, exclude=False)[source]¶ Return a Group of the tunnels for which predicate returns True.
predicate must be a no-argument callable that can be pickled.
If exclude is True, then return a Group that only contains tunnels for which predicate returns False.
Raise RemoteException if any hosts could not be connected or fail to evaluate the predicate.
-
put
(local_path, remote_path=None, mode=420)[source]¶ Copy a file to all remote hosts.
If remote_path is given, it is the remote path to write to. Otherwise, a temporary filename will be used (which will be different on each host).
mode gives the permission bits of the files to create, or 0o644 if unspecified.
This operation supports arbitarily large files (file data is streamed, not buffered in memory).
Return a
GroupResult
of dicts, each containing:remote_path
- the absolute remote pathsize
- the number of bytes receivedsha1sum
- a sha1 checksum of the file data
-
Results¶
-
class
chopsticks.group.
GroupResult
[source]¶ The results of a
Group.call()
operation.GroupResult behaves as a dictionary of results, keyed by hostname, although failures from individual hosts are represented as
ErrorResult
objects.Methods are provided to easily process successes and failures separately.
-
class
chopsticks.group.
ErrorResult
(msg, tb=None)[source]¶ Indicates an error returned by the remote host.
Because tracebacks or error types cannot be represented across hosts this will simply consist of a message.
Error results provide the following attributes:
-
msg
¶ A human-readable error message.
-
tb
¶ The traceback from the remote host as a string, or
None
if unavailable.
-
Set operations¶
Groups also behave like sets over tunnels. Tunnels are compared by name for this purpose (in general, tunnels need unique names due to the way results are returned from group methods).
For example:
webservers = Group(['web1', 'web2'])
celery_workers = Group(['worker1', 'worker2', 'worker3'])
(webservers + celery_workers).call(install_virtualenv)
For this purpose, individual tunnels act as a group containing just one tunnel:
>>> dck1 = Docker('docker1')
>>> dck2 = Docker('docker2')
>>> dck1 + dck2
Group([Docker('docker1'), Docker('docker2')])
Examples¶
For example, this code:
from chopsticks.facts import ip
from chopsticks.group import Group
group = Group([
'web1.example.com',
'web2.example.com',
'web3.example.com',
])
for host, addr in group.call(ip).items():
print('%s ip:' % host, addr)
might output:
web1.example.com ip: 196.168.10.5
web3.example.com ip: 196.168.10.7
web2.example.com ip: 196.168.10.6
You could also construct a group from existing tunnels - or mix and match:
all_hosts = Group([
'web1.example.com',
Docker('example'),
Local('worker')
])