Skip to content

data_event_handlers_files

Handling of file-based data events.

This module contains Data Event Handler classes that manipulate file-based events.

PilatusFilesEventHandler

Bases: OmDataEventHandlerProtocol

See documentation of the __init__ function.

__init__(*, source, data_sources, monitor_parameters)

Data Event Handler for Pilatus single-frame files.

This class handles data events originating from single-frame CBF files written by a Pilatus detector.

This class implements the interface described by its base Protocol class. Please see the documentation of that class for additional information about the interface.

  • For this Event Handler, a data event corresponds to the content of an individual single-frame CBF file.

  • The source string required by this Data Event Handler is the path to a file containing a list of CBF files to process, one per line, with their absolute or relative path.

Parameters:

Name Type Description Default
source str

A string describing the data event source.

required
data_sources Dict[str, OmDataSourceProtocol]

A dictionary containing a set of Data Source class instances.

  • Each dictionary key must define the name of a data source.

  • The corresponding dictionary value must store the instance of the Data Source class # noqa: E501 that describes the source.

required
monitor_parameters MonitorParameters

An object storing OM's configuration parameters.

required

initialize_event_handling_on_collecting_node(*, node_rank, node_pool_size)

Initializes Pilatus single-frame file event handling on the collecting node.

Please see the documentation of the base Protocol class for additional information about this method.

There is usually no need to initialize Pilatus file-based event handling on the collecting node, so this function actually does nothing.

Parameters:

Name Type Description Default
node_rank int

The OM rank of the current node int the OM node pool. The rank is an integer that unambiguously identifies the node in the pool.

required
node_pool_size int

The total number of nodes in the OM pool, including all the processing nodes and the collecting node.

required

initialize_event_handling_on_processing_node(*, node_rank, node_pool_size)

Initializes Pilatus single-frame file event handling on the processing nodes.

Please see the documentation of the base Protocol class for additional information about this method.

Parameters:

Name Type Description Default
node_rank int

The OM rank of the current node int the OM node pool. The rank is an integer that unambiguously identifies the node in the pool.

required
node_pool_size int

The total number of nodes in the OM pool, including all the processing nodes and the collecting node.

required

event_generator(*, node_rank, node_pool_size)

Retrieves Pilatus single-frame file events.

Please see the documentation of the base Protocol class for additional information about this method.

This function retrieves data events on the processing nodes. Each retrieved event corresponds to the content of an individual single-frame CBF file. The function tries to distribute the events as evenly as possible across all the processing nodes, with each node ideally processing the same number of events. If the total number of events cannot be split evenly, the last last node will process fewer events than the others.

Parameters:

Name Type Description Default
node_rank int

The OM rank of the current node int the OM node pool. The rank is an integer that unambiguously identifies the node in the pool.

required
node_pool_size int

The total number of nodes in the OM pool, including all the processing nodes and the collecting node.

required

open_event(*, event)

Opens a Pilatus single-frame file event.

Please see the documentation of the base Protocol class for additional information about this method.

This function opens the CBF file associated with the data event and stores its content in the provided event dictionary, as the value corresponding to the data key.

Parameters:

Name Type Description Default
event Dict[str, Any]

A dictionary storing the event data.

required

close_event(*, event)

Closes a Pilatus single-frame file event.

Please see the documentation of the base Protocol class for additional information about this method.

CBF files don't need to be closed, so this function does nothing.

Parameters:

Name Type Description Default
event Dict[str, Any]

A dictionary storing the event data.

required

extract_data(*, event)

Extracts data from a Pilatus single-frame file event.

Please see the documentation of the base Protocol class for additional information about this method.

Parameters:

Name Type Description Default
event Dict[str, Any]

A dictionary storing the event data.

required

Returns:

Type Description
Dict[str, Any]

A dictionary storing the extracted data.

  • Each dictionary key identifies a Data Source in the event for which data has been retrieved.

  • The corresponding dictionary value stores the data extracted from the Data Source for the event being processed.

Raises:

Type Description
OmDataExtractionError

Raised when data cannot be extracted from the event.

initialize_event_data_retrieval()

Initializes data data retrieval from Pilatus single-frame files.

This function initializes the retrieval of single standalone data events from Pilatus single-frame data files.

Please see the documentation of the base Protocol class for additional information about this method.

retrieve_event_data(event_id)

Retrieves all data related to the requested event.

Please see the documentation of the base Protocol class for additional information about this method.

This function retrieves all data related to the event specified by the provided identifier. A Pilatus single-frame file event identifier corresponds to the relative or absolute path to a file containing the event data.

Parameters:

Name Type Description Default
event_id str

A string that uniquely identifies a data event.

required

Returns:

Type Description
Dict[str, Any]

All data related to the requested event.

Jungfrau1MFilesDataEventHandler

Bases: OmDataEventHandlerProtocol

See documentation of the __init__ function.

__init__(*, source, data_sources, monitor_parameters)

Data Event Handler for Jungfrau 1M files.

This class handles data events retrieved from files written by a Jungfrau 1M detector in HDF5 format.

This class implements the interface described by its base Protocol class. Please see the documentation of that class for additional information about the interface.

  • For this Event Handler, a data event corresponds to all the information associated with an individual frame stored in an HDF5 file.

  • The source string required by this Data Event Handler is the path to a file containing a list of master HDF5 files to process, one per line, with their absolute or relative path. Each file can store more than one detector data frame, and each frame in the file is processed as a separate event.

Parameters:

Name Type Description Default
source str

A string describing the data event source.

required
data_sources Dict[str, OmDataSourceProtocol]

A dictionary containing a set of Data Source class instances.

  • Each dictionary key must define the name of a data source.

  • The corresponding dictionary value must store the instance of the Data Source class # noqa: E501 that describes the source.

required
monitor_parameters MonitorParameters

An object storing OM's configuration parameters.

required

initialize_event_handling_on_collecting_node(*, node_rank, node_pool_size)

Initializes Jungfrau 1M file event handling on the collecting node.

Please see the documentation of the base Protocol class for additional information about this method.

There is usually no need to initialize Jungfrau 1M file-based event handling on the collecting node, so this function actually does nothing.

Parameters:

Name Type Description Default
node_rank int

The OM rank of the current node int the OM node pool. The rank is an integer that unambiguously identifies the node in the pool.

required
node_pool_size int

The total number of nodes in the OM pool, including all the processing nodes and the collecting node.

required

initialize_event_handling_on_processing_node(*, node_rank, node_pool_size)

Initializes Jungfrau 1M file event handling on the processing nodes.

Please see the documentation of the base Protocol class for additional information about this method.

Parameters:

Name Type Description Default
node_rank int

The OM rank of the current node int the OM node pool. The rank is an integer that unambiguously identifies the node in the pool.

required
node_pool_size int

The total number of nodes in the OM pool, including all the processing nodes and the collecting node.

required

event_generator(*, node_rank, node_pool_size)

Retrieves Jungfrau 1M file events.

Please see the documentation of the base Protocol class for additional information about this method.

This function retrieves data events on the processing nodes. Each retrieved event corresponds to a single single detector data frame with all its associated data. The function tries to distribute the events as evenly as possible across all the processing nodes, with each node ideally processing the same number of events. If the total number of events cannot be split evenly, the last last node processes fewer events than the others.

Parameters:

Name Type Description Default
node_rank int

The OM rank of the current node int the OM node pool. The rank is an integer that unambiguously identifies the node in the pool.

required
node_pool_size int

The total number of nodes in the OM pool, including all the processing nodes and the collecting node.

required

open_event(*, event)

Opens a Jungfrau 1M file event.

Please see the documentation of the base Protocol class for additional information about this method.

Since each detector frame in each HDF5 file is considered a separate event, the event_generator method, which distributes the frames across the processing nodes, takes care of opening and closing the files. This function therefore does nothing.

Parameters:

Name Type Description Default
event Dict[str, Any]

A dictionary storing the event data.

required

close_event(*, event)

Closes a Jungfrau 1M file event.

Please see the documentation of the base Protocol class for additional information about this method.

Since each detector frame in each HDF5 file is considered a separate event, the event_generator method, which distributes the frames across the processing nodes, takes care of opening and closing the files. This function therefore does nothing.

Parameters:

Name Type Description Default
event Dict[str, Any]

A dictionary storing the event data.

required

extract_data(*, event)

Extracts data from a Jungfrau 1M file event.

Please see the documentation of the base Protocol class for additional information about this method.

Parameters:

Name Type Description Default
event Dict[str, Any]

A dictionary storing the event data.

required

Returns:

Type Description
Dict[str, Any]

A dictionary storing the extracted data.

  • Each dictionary key identifies a Data Source in the event for which data has been retrieved.

  • The corresponding dictionary value stores the data extracted from the Data Source for the event being processed.

Raises:

Type Description
OmDataExtractionError

Raised when data cannot be extracted from the event.

initialize_event_data_retrieval()

Initializes event data retrieval from Jungfrau 1M HDF5 files.

This function initializes the retrieval of single standalone data events from Jungfrau 1M HDF5 files.

Please see the documentation of the base Protocol class for additional information about this method.

retrieve_event_data(event_id)

Retrieves all data related to the requested event.

Please see the documentation of the base Protocol class for additional information about this method.

This function retrieves all data related to the event specified by the provided identifier. A Jungfrau 1M unique event identifier is a string consisting of the absolute or relative path to the master HDF5 file attached to the event, and the index of the event within the file, separated by the '//' symbol.

Parameters:

Name Type Description Default
event_id str

A string that uniquely identifies a data event.

required

Returns:

Type Description
Dict[str, Any]

All data related to the requested event.

EigerFilesDataEventHandler

Bases: OmDataEventHandlerProtocol

See documentation of the __init__ function.

__init__(*, source, data_sources, monitor_parameters)

Data Event Handler for Eiger files.

This Data Event Handler deals with events originating from files written by an Eiger detector in HDF5 format.

This class implements the interface described by its base Protocol class. Please see the documentation of that class for additional information about the interface.

  • For this Event Handler, a data event corresponds to all the information associated with an individual frame stored in an HDF5 file.

  • The source string required by this Data Event Handler is the path to a file containing a list of HDF5 files to process, one per line, with their absolute or relative path. Each file can store more than one detector data frame, and each frame in the file is processed as a separate event.

Parameters:

Name Type Description Default
source str

A string describing the data event source.

required
data_sources Dict[str, OmDataSourceProtocol]

A dictionary containing a set of Data Source class instances.

  • Each dictionary key must define the name of a data source.

  • The corresponding dictionary value must store the instance of the Data Source class # noqa: E501 that describes the source.

required
monitor_parameters MonitorParameters

An object storing OM's configuration parameters.

required

initialize_event_handling_on_collecting_node(*, node_rank, node_pool_size)

Initializes Eiger file event handling on the collecting node.

Please see the documentation of the base Protocol class for additional information about this method.

There is usually no need to initialize Eiger's file-based data handling on the collecting node, so this function actually does nothing.

Parameters:

Name Type Description Default
node_rank int

The OM rank of the current node int the OM node pool. The rank is an integer that unambiguously identifies the node in the pool.

required
node_pool_size int

The total number of nodes in the OM pool, including all the processing nodes and the collecting node.

required

initialize_event_handling_on_processing_node(*, node_rank, node_pool_size)

Initializes Eiger file event handling on the processing nodes.

Please see the documentation of the base Protocol class for additional information about this method.

Parameters:

Name Type Description Default
node_rank int

The OM rank of the current node int the OM node pool. The rank is an integer that unambiguously identifies the node in the pool.

required
node_pool_size int

The total number of nodes in the OM pool, including all the processing nodes and the collecting node.

required

event_generator(*, node_rank, node_pool_size)

Retrieves Eiger file events.

Please see the documentation of the base Protocol class for additional information about this method.

This function retrieves data events on the processing nodes. Each retrieved event corresponds to a single detector data frame with all its associated data. The function tries to distribute the events as evenly as possible across all the processing nodes, with each node ideally processing the same number of events. If the total number of events cannot be split evenly, the last node processes fewer events than the others.

Parameters:

Name Type Description Default
node_rank int

The OM rank of the current node int the OM node pool. The rank is an integer that unambiguously identifies the node in the pool.

required
node_pool_size int

The total number of nodes in the OM pool, including all the processing nodes and the collecting node.

required

open_event(*, event)

Opens a Eiger file event.

Please see the documentation of the base Protocol class for additional information about this method.

Since each detector frame in each HDF5 file is considered a separate event, the event_generator method, which distributes the frames across the processing nodes, takes care of opening and closing the files. This function therefore does nothing.

Parameters:

Name Type Description Default
event Dict[str, Any]

A dictionary storing the event data.

required

close_event(*, event)

Closes a Eiger file event.

Please see the documentation of the base Protocol class for additional information about this method.

Since each detector frame in each HDF5 file is considered a separate event, the event_generator method, which distributes the frames across the processing nodes, takes care of opening and closing the files. This function therefore does nothing.

Parameters:

Name Type Description Default
event Dict[str, Any]

A dictionary storing the event data.

required

extract_data(*, event)

Extracts data from an Eiger file event.

Please see the documentation of the base Protocol class for additional information about this method.

Parameters:

Name Type Description Default
event Dict[str, Any]

A dictionary storing the event data.

required

Returns:

Type Description
Dict[str, Any]

A dictionary storing the extracted data.

  • Each dictionary key identifies a Data Source in the event for which data has been retrieved.

  • The corresponding dictionary value stores the data extracted from the Data Source for the event being processed.

Raises:

Type Description
OmDataExtractionError

Raised when data cannot be extracted from the event.

initialize_event_data_retrieval()

Initializes event data retrieval from Eiger files.

This function initializes the retrieval of single standalone data events from Eiger files.

Please see the documentation of the base Protocol class for additional information about this method.

retrieve_event_data(event_id)

Retrieves all data related to the requested event.

Please see the documentation of the base Protocol class for additional information about this method.

This function retrieves all data related to the event specified by the provided identifier. An Eiger event identifier is a string consisting of the absolute or relative path to the HDF5 data file attached to the event, and the index of the event within the file, separated by '//' symbol.

Parameters:

Name Type Description Default
event_id str

A string that uniquely identifies a data event.

required

Returns:

Type Description
Dict[str, Any]

All data related to the requested event.

RayonixMccdFilesEventHandler

Bases: OmDataEventHandlerProtocol

See documentation of the __init__ function.

__init__(*, source, data_sources, monitor_parameters)

Data Event Handler for Rayonix MX340-HS single-frame files.

This class handles data events originating from single-frame mccd files written by a Rayonix MX340-HS detector.

This class implements the interface described by its base Protocol class. Please see the documentation of that class for additional information about the interface.

  • For this Event Handler, a data event corresponds to the content of an individual single-frame mccd file.

  • The source string required by this Data Event Handler is the path to a file containing a list of mccd files to process, one per line, with their absolute or relative path.

Parameters:

Name Type Description Default
source str

A string describing the data event source.

required
data_sources Dict[str, OmDataSourceProtocol]

A dictionary containing a set of Data Source class instances.

  • Each dictionary key must define the name of a data source.

  • The corresponding dictionary value must store the instance of the Data Source class # noqa: E501 that describes the source.

required
monitor_parameters MonitorParameters

An object storing OM's configuration parameters.

required

initialize_event_handling_on_collecting_node(*, node_rank, node_pool_size)

Initializes Rayonix MX340-HS file-based event handling on the collecting node.

Please see the documentation of the base Protocol class for additional information about this method.

There is usually no need to initialize Rayonix MX340-HS file-based data event handling on the collecting node, so this function actually does nothing.

Parameters:

Name Type Description Default
node_rank int

The rank, in the OM pool, of the processing node calling the function.

required
node_pool_size int

The total number of nodes in the OM pool, including all the processing nodes and the collecting node.

required

initialize_event_handling_on_processing_node(*, node_rank, node_pool_size)

Initializes Rayonix MX340-HS file-based event handling on the processing nodes.

Please see the documentation of the base Protocol class for additional information about this method.

Parameters:

Name Type Description Default
node_rank int

The OM rank of the current node int the OM node pool. The rank is an integer that unambiguously identifies the node in the pool.

required
node_pool_size int

The total number of nodes in the OM pool, including all the processing nodes and the collecting node.

required

event_generator(*, node_rank, node_pool_size)

Retrieves Rayonix MX340-HS single-frame file events.

Please see the documentation of the base Protocol class for additional information about this method.

This function retrieves data events on the processing nodes. Each retrieved event corresponds to the content of an individual single-frame mccd file. The function tries to distribute the events as evenly as possible across all the processing nodes, with each node ideally processing the same number of events. If the total number of events cannot be split evenly, the last last node processes fewer events than the others.

Parameters:

Name Type Description Default
node_rank int

The OM rank of the current node int the OM node pool. The rank is an integer that unambiguously identifies the node in the pool.

required
node_pool_size int

The total number of nodes in the OM pool, including all the processing nodes and the collecting node.

required

open_event(*, event)

Opens a Rayonix MX340-HS single-frame file event.

Please see the documentation of the base Protocol class for additional information about this method.

Since detector_data is the only event data which is retrieved from the mccd files, the Data Source that handles this type of data takes care of opening and closing the files. This function therefore does nothing.

Parameters:

Name Type Description Default
event Dict[str, Any]

A dictionary storing the event data.

required

close_event(*, event)

Closes a Rayonix MX340-HS single-frame file event.

Please see the documentation of the base Protocol class for additional information about this method.

Since detector_data is the only event data which is retrieved from the mccd files, the Data Source that handles this type of data takes care of opening and closing the files. This function therefore does nothing.

Parameters:

Name Type Description Default
event Dict[str, Any]

A dictionary storing the event data.

required

extract_data(*, event)

Extracts data from a Rayonix MX340-HS single-frame file event.

Please see the documentation of the base Protocol class for additional information about this method.

Parameters:

Name Type Description Default
event Dict[str, Any]

A dictionary storing the event data.

required

Returns:

Type Description
Dict[str, Any]

A dictionary storing the extracted data.

  • Each dictionary key identifies a Data Source in the event for which data has been retrieved.

  • The corresponding dictionary value stores the data extracted from the Data Source for the event being processed.

Raises:

Type Description
OmDataExtractionError

Raised when data cannot be extracted from the event.

initialize_event_data_retrieval()

Initializes event data retrievals from Rayonix MX340-HS single-frame files.

This function initializes the retrieval of single standalone data events from Rayonix MX340-HS single-frame files.

Please see the documentation of the base Protocol class for additional information about this method.

retrieve_event_data(event_id)

Retrieves all data related to the requested event.

Please see the documentation of the base Protocol class for additional information about this method.

This function retrieves all data related to the event specified by the provided identifier. A Rayonix MX340-HS event identifier is the full absolute or relative path to the mccd file associated with the event.

Parameters:

Name Type Description Default
event_id str

A string that uniquely identifies a data event.

required

Returns:

Type Description
Dict[str, Any]

All data related to the requested detector event.

Lambda1M5FilesDataEventHandler

Bases: OmDataEventHandlerProtocol

See documentation of the __init__ function.

__init__(*, source, data_sources, monitor_parameters)

Data Event Handler for Lambda 1.5M files.

This Data Event Handler deals with events originating from files written by a Lambda 1.5M detector in HDF5 format.

This class implements the interface described by its base Protocol class. Please see the documentation of that class for additional information about the interface.

  • For this Event Handler, a data event corresponds to all the information associated with an individual frame stored in two separate HDF5 files written by two detector modules.

  • The source string required by this Data Event Handler is the path to a file containing a list of HDF5 files written by the first detector module ("_m01.nxs"), one per line, with their absolute or relative path. Each file can store more than one detector data frame, and each frame in the file is processed as a separate event.

Parameters:

Name Type Description Default
source str

A string describing the data event source.

required
data_sources Dict[str, OmDataSourceProtocol]

A dictionary containing a set of Data Source class instances.

  • Each dictionary key must define the name of a data source.

  • The corresponding dictionary value must store the instance of the Data Source class # noqa: E501 that describes the source.

required
monitor_parameters MonitorParameters

An object storing OM's configuration parameters.

required

initialize_event_handling_on_collecting_node(*, node_rank, node_pool_size)

Initializes Lambda 1.5M file event handling on the collecting node.

Please see the documentation of the base Protocol class for additional information about this method.

There is usually no need to initialize Lambda 1.5M file-based event handling on the collecting node, so this function actually does nothing.

Parameters:

Name Type Description Default
node_rank int

The OM rank of the current node int the OM node pool. The rank is an integer that unambiguously identifies the node in the pool.

required
node_pool_size int

The total number of nodes in the OM pool, including all the processing nodes and the collecting node.

required

initialize_event_handling_on_processing_node(*, node_rank, node_pool_size)

Initializes Lambda 1.5M file event handling on the processing nodes.

Please see the documentation of the base Protocol class for additional information about this method.

Parameters:

Name Type Description Default
node_rank int

The OM rank of the current node int the OM node pool. The rank is an integer that unambiguously identifies the node in the pool.

required
node_pool_size int

The total number of nodes in the OM pool, including all the processing nodes and the collecting node.

required

event_generator(*, node_rank, node_pool_size)

Retrieves Lambda 1.5M file events.

This method overrides the corresponding method of the base class: please also refer to the documentation of that class for more information.

This function retrieves data events on the processing nodes. Each retrieved event corresponds to a single detector frame with all its associated data. The function tries to distribute the events as evenly as possible across all the processing nodes, with each node ideally processing the same number of events. If the total number of events cannot be split evenly, the last last node processes fewer events than the others.

Parameters:

Name Type Description Default
node_rank int

The OM rank of the current node int the OM node pool. The rank is an integer that unambiguously identifies the node in the pool.

required
node_pool_size int

The total number of nodes in the OM pool, including all the processing nodes and the collecting node.

required

open_event(*, event)

Opens a Lambda 1.5M file event.

Please see the documentation of the base Protocol class for additional information about this method.

Since each detector frame in each HDF5 file is considered a separate event, the event_generator method, which distributes the frames across the processing nodes, takes care of opening and closing the files. This function therefore does nothing.

Parameters:

Name Type Description Default
event Dict[str, Any]

A dictionary storing the event data.

required

close_event(*, event)

Closes a Lambda 1.5M file event.

Please see the documentation of the base Protocol class for additional information about this method.

Since each detector frame in each HDF5 file is considered a separate event, the event_generator method, which distributes the frames across the processing nodes, takes care of opening and closing the files. This function therefore does nothing.

Parameters:

Name Type Description Default
event Dict[str, Any]

A dictionary storing the event data.

required

extract_data(*, event)

Extracts data from a Lambda 1.5M file event.

Please see the documentation of the base Protocol class for additional information about this method.

Parameters:

Name Type Description Default
event Dict[str, Any]

A dictionary storing the event data.

required

Returns:

Type Description
Dict[str, Any]

A dictionary storing the extracted data.

  • Each dictionary key identifies a Data Source in the event for which data has been retrieved.

  • The corresponding dictionary value stores the data extracted from the Data Source for the event being processed.

Raises:

Type Description
OmDataExtractionError

Raised when data cannot be extracted from the event.

initialize_event_data_retrieval()

Initializes event data retrievals from Lambda 1.5M HDF5 files.

This function initializes the retrieval of single standalone events from Lambda 1.5M HDF5 files.

Please see the documentation of the base Protocol class for additional information about this method.

retrieve_event_data(event_id)

Retrieves all data related to the requested event.

Please see the documentation of the base Protocol class for additional information about this method.

This function retrieves all data related to the event specified by the provided identifier. A Lambda 1.5M unique event identifier is a string consisting of two parts. The first part is the absolute or relative path to the HDF5 file storing the first panel of the detector data frame attached to the event ("*_m01.nxs"), while the second part is the index of the event within the file. The two parts are separated by the '//' symbol.

Parameters:

Name Type Description Default
event_id str

A string that uniquely identifies a data event.

required

Returns:

Type Description
Dict[str, Any]

All data related to the requested event.