worker_plugins

class pyrosetta.distributed.cluster.worker_plugins.SocketLoggerPlugin(logging_level: str, maxsize: int = 128)

Bases: WorkerPlugin

Install a MultiSocketHandler logging handler on a dask worker logger.

__init__(logging_level: str, maxsize: int = 128) → None

setup(worker: Worker) → None

Setup dask worker plugin.

teardown(worker: Worker) → None

Teardown dask worker plugin.

idempotent: bool = False

transition(key: Key, start: WorkerTaskStateState, finish: WorkerTaskStateState, **kwargs: Any) → None

Throughout the lifecycle of a task (see Worker State), Workers are instructed by the scheduler to compute certain tasks, resulting in transitions in the state of each task. The Worker owning the task is then notified of this state transition.

Whenever a task changes its state, this method will be called.

Warning

This is an advanced feature and the transition mechanism and details of task states are subject to change without deprecation cycle.

key : start :

Start state of the transition. One of waiting, ready, executing, long-running, memory, error.

finish :

Final state of the transition.

kwargs :

More options passed when transitioning

class pyrosetta.distributed.cluster.worker_plugins.TaskSecurityPlugin(instance_id: str, prk: MaskedBytes, max_nonce: int)

Bases: WorkerPlugin, NonceCache

Install a secure NonceCache instance with replay protection on a dask worker.

__init__(instance_id: str, prk: MaskedBytes, max_nonce: int) → None

setup(worker: Worker) → None

Run when the plugin is attached to a worker. This happens when the plugin is registered and attached to existing workers, or when a worker is created after the plugin has been registered.

teardown(worker: Worker) → None

Run when the worker to which the plugin is attached is closed, or when the plugin is removed.

_cache_nonce(sealed: bytes) → None

Run Hash-based Message Authentication Code (HMAC) verification and cache nonces for replay protection without data decompression.

static _get_state(self) → Dict[str, Any]

A method used to override the default NonceCache.__getstate__() method that sets the pseudo-random key value to None in the returned state.

_is_protocol = False

static _on_worker() → bool

Test if the nonce cache is on a dask worker.

idempotent: bool = False

transition(key: Key, start: WorkerTaskStateState, finish: WorkerTaskStateState, **kwargs: Any) → None

Throughout the lifecycle of a task (see Worker State), Workers are instructed by the scheduler to compute certain tasks, resulting in transitions in the state of each task. The Worker owning the task is then notified of this state transition.

Whenever a task changes its state, this method will be called.

Warning

This is an advanced feature and the transition mechanism and details of task states are subject to change without deprecation cycle.

key : start :

Start state of the transition. One of waiting, ready, executing, long-running, memory, error.

finish :

Final state of the transition.

kwargs :

More options passed when transitioning