parallelization_layer

Parallelization Layer's Protocol classes.

This module contains base Protocol classes for OM's Parallelization Layer.

OmParallelizationProtocol

Bases: Protocol

See documentation of the __init__ function.

__init__(*, data_retrieval_layer, processing_layer, monitor_parameters)

Protocol for OM's Parallelization classes.

Parallelization classes orchestrate OM's processing and collecting nodes, and take care of the communication between them.

• When OM start, a Parallelization class instance initializes several processing nodes, plus a single collecting node. The class then associates an instance of a Data Retrieval class (see OmDataRetrievalProtocol;) # noqa: E501 and an instance of a Processing class (see OmProcessingProtocol][om.protocols.processing_layer.OmProcessingProtocol]) with each node.

• Each processing node retrieves an event from a data event source by calling the relevant Data Retrieval class methods. It then invokes the appropriate Processing class methods on the event. Finally, it transfers the processed data to the collecting node. The node then retrieves another event, and the cycle continues until there are no more data events or OM shuts down.

• Every time it receives data from a processing node, the collecting node invokes the relevant Processing class methods to aggregate the received data.

• When all events from the source have been processed, all nodes perform some final clean-up tasks by calling the appropriate methods of the Processing class. All nodes then shut down.

This Protocol class describes the interface that every Parallelization class in OM must implement.

Parameters:

Name	Type	Description	Default
data_retrieval_layer	OmDataRetrievalProtocol	A class instance defining how data and data events are retrieved and handled.	required
processing_layer	OmProcessingProtocol	A class instance defining how retrieved data is processed.	required
monitor_parameters	MonitorParameters	An object storing OM's configuration parameters.	required

start()

Starts OM.

This function begins operations on the processing and collecting nodes.

When this function is called on a processing node, the processing node starts retrieving data events and processing them. When instead this function is called on the collecting node, the node starts receiving data from the processing nodes and aggregating it.

shutdown(*, msg='Reason not provided.')

Shuts down OM.

This function stops the processing and collecting nodes.

When this function is called on a processing node, the processing node communicates to the collecting node that it is shutting down, then shuts down. When instead this function is called on the collecting node, the collecting node tells every processing node to shut down, waits for all the nodes to confirm that they have stopped operating, then shuts itself down.

Parameters:

Name	Type	Description	Default
msg	str	Reason for shutting down. Defaults to "Reason not provided".	'Reason not provided.'