Skip to content

cheetah

TypeFrameListData

Bases: NamedTuple

Cheetah frame list data.

This named tuple is used to store the detector frame data which is later written to the frames.txt file.

Attributes:

Name Type Description
timestamp numpy.float64

The timestamp of the frame.
event_id Union[str, None]

The event ID of the frame.
frame_is_hit int

A flag indicating whether the frame is a labelled as a hit.
filename str

The name of the file containing the frame.
index_in_file int

The index of the frame in the file.
num_peaks int

The number of peaks in the frame.
average_intensity numpy.float64

The average intensity of the peaks in the frame.

TypeClassSumData

Bases: TypedDict

Cheetah data class frame sum data.

A dictionary storing the number of detector frames belonging to a specific data class, their sum, and the virtual powder plot generated from the Bragg peaks detected in them.

Attributes:

Name Type Description
num_frames int

The number of detector frames belonging to the data class.
sum_frames NDArray[numpy.float_]

The sum of the detector frames belonging to the class.
peak_powder NDArray[numpy.float_]

The virtual powder plot for the data class.

CheetahStatusFileWriter

See documentation for the __init__ function.

__init__(*, parameters)

Cheetah status file writer.

This class stores information about the current state of data processing in Cheetah.

Parameters:

Name Type Description Default
parameters Dict[str, Any]

A set of OM configuration parameters collected together in a parameter group. The parameter group must contain the following entries:

  • directory_for_processed_data: A relative or absolute path to the directory where the output files are to be written.
 required

update_status(*, status='', num_frames=0, num_hits=0)

Writes a status file.

This function writes the information about the current state of data processing to a status file. The Cheetah GUI inspect this file to get information about Cheetah's current state.

Parameters:

Name Type Description Default
status str

A string describing the current status of the data processing in Cheetah.

 ''
num_frames int

The number of detector frames processed so far by Cheetah.

 0
num_hits int

The number of hits found so far by Cheetah.

 0

CheetahListFilesWriter

See documentation for the __init__ function.

__init__(*, cheetah_parameters)

Cheetah list files writer.

This class manages the information that gets written to the "frames.txt", "cleaned.txt", "events.lst", "hits.lst" and "peaks.txt" files required by the Cheetah GUI.

"frames.txt" contains a list of all the detector frames processed by Cheetah, with information about the frame timestamp, event ID, whether the frame is a hit, the name of the file containing the frame, the index of the frame in the file, the number of peaks detected in the frame, and the average intensity of the peaks in the frame.

"cleaned.txt" contains a list of all the detector frames that have been identified as hits by Cheetah, with the same information as "frames.txt".

"events.lst" contains a list of all the event IDs of the detector frames processed by Cheetah.

"hits.lst" contains a list of all the event IDs of the detector frames that have been identified as hits by Cheetah.

"peaks.txt" contains a list of all the Bragg peaks detected by Cheetah, with information about the event ID of the frame to which the peak belongs, the number of peaks in the frame, the fast-scan and slow-scan coordinates of the peak, the peak intensity, the number of pixels in the peak, the maximum pixel intensity in the peak, and the signal-to-noise ratio of the peak.

Parameters:

Name Type Description Default
cheetah_parameters Dict[str, Any]

An object storing Cheetah's configuration parameters.

 required

add_frame(*, frame_data, peak_list)

Adds a frame to the list files.

This function adds information related to a single detector data frame to the list files.

Parameters:

Name Type Description Default
frame_data TypeFrameListData

Information about the frame that must be added to the list files.

 required
peak_list TypePeakList

The list of peaks detected in the frame being added to the files.

 required

flush_files()

Flushes the list files.

This function flushes the list files to disk, writing on storage media the information still stored only in memory.

sort_frames_and_close_files()

Performs final operations on the list files.

This functions performs some operations on the list files just before closing them: it sorts the frames according to their event identifier and it writes the sorted data to the frames.txt, cleaned.txt and events.lst files. The function then closes all the list files.

CheetahClassSumsAccumulator

See documentation for the __init__ function.

__init__(*, cheetah_parameters, num_classes=2)

Cheetah data class sum accumulator.

This class accumulates information about the sum and virtual powder plot of all detector frames belonging to a specific data class. After data frame information has been added to the accumulator, the sum and virtual powder plot for the data class can be retrieved from the accumulator, either after a predefined number of frames has been added to the accumulator, or at will.

Parameters:

Name Type Description Default
cheetah_parameters Dict[str, Any]

A set of OM configuration parameters collected together in a parameter group. The parameter group must contain the following entries:

class_sums_sending_interval: The number of detector frames that the accumulator can receive before returning the sum and virtual powder plot information and resetting the accumulator.

 required
num_classes int

The total number of data classes currently managed by Cheetah.

 2

add_frame(*, class_number, frame_data, peak_list)

Adds a detector frame to the accumulator.

This function adds information about a detector data frame to the accumulator.

Parameters:

Name Type Description Default
class_number int

The data class number to which the frame being added belongs.

 required
frame_data Union[NDArray[numpy.float_], NDArray[numpy.int_]]

Information about the detector data frame that must be added to the accumulator.

 required
peak_list TypePeakList

The list of peaks detected in the frame being added to the accumulator.

 required

get_sums_for_sending(disregard_counter=False)

Retrieves sum and virtual powder plot data from the accumulator.

This function returns the data stored in the accumulator if the predefined number of frames has been added to the accumulator, or if the disregard_counter argument is True. Otherwise, returns None.

Parameters:

Name Type Description Default
disregard_counter bool

If True, the sending counter is disregarded and the class sums are returned.

 False

Returns:

Type Description
Union[None, List[TypeClassSumData]]

The sum and virtual powder plot stored by the accumulator, or None.

CheetahClassSumsCollector

See documentation for the __init__ function.

__init__(*, cheetah_parameters, num_classes)

Cheetah data class sum collector.

This class collects accumulated data class information retrieved from the processing nodes, and stores the cumulative total information associated with the data class. The information collected by this class can optionally be written to an HDF5 file.

Parameters:

Name Type Description Default
cheetah_parameters Dict[str, Any]

A set of OM configuration parameters collected together in a parameter group. The parameter group must contain the following entries:

  • write_class_sums: Whether the information stored by the collector should be written to disk regularly.

  • class_sums_update_interval: If the information stored by the collector must be written to disk (see the write_class_sums parameter), this parameter determines how many times the collector can be updated before the accumulated data is written to a file After the HDF5 has been written, the update count is reset.

 required
num_classes int

The total number of data classes currently managed by Cheetah.

 required

add_sums(*, class_sums)

Adds information to the collectors

Adds class sums information, retrieved from the processing nodes, to the collector. If the predefined number of updates has been reached, the function triggers the writing of the collector's data to an HDF5 file.

Parameters:

Name Type Description Default
class_sums List[TypeClassSumData]

The information to be added to the collector.

 required

save_sums()

Saves the collector's data to an HDF5 file.

This function saves the collector's accumulated data to an HDF5 file. It is called automatically by the collector when required, but can also be called manually.

HDF5Writer

See documentation of the __init__ function.

__init__(*, node_rank, cheetah_parameters)

HDF5 file writer for Cheetah.

This class creates HDF5 data files to store the information processed by Cheetah. For each data event, this class saves into an HDF5 file a processed detector data frame, the list of Bragg peaks detected in the frame, and some additional information (timestamp, beam energy, detector distance, pump laser state).

Parameters:

Name Type Description Default
cheetah_parameters Dict[str, Any]

A set of OM configuration parameters collected together in a parameter group. The parameter group must contain the following entries:

directory_for_processed_data: A relative or absolute path to the directory where the output files are written.

compression: The compression filter to be applied to the data in the output file.

hdf5_fields: A dictionary storing information about the internal HDF5 path where each data entry must be written.

* The keys in the dictionary must store the names of data entries
  to write.

* The corresponding dictionary values must contain the internal
  HDF5 paths where the entries must be written.

processed_filename_prefix: A string that is prepended to the name of the output files. Optional. If the value of this entry is None, the string 'processed_' will be used as prefix. Defaults to None.

processed_filename_extension: An extension string that id appended to the name of the output files. Optional. If the value of this entry is None, the string 'h5' is be used as extension. Defaults to None.

compression_opts: The compression level to be used, if data compression is applied to the output files. The information in this entry only applies if the corresponding compression entry is not None, otherwise, it is ignored. Optional. If the value of this entry is None, the compression level is set to 4. Defaults to None.

compression_shuffle: Whether the shuffle filter is applied. If the value of this entry is True, the filter is applied to the data being written, otherwise it is not. Defaults to None.

max_num_peaks: The maximum number of detected Bragg peaks that are written in the HDF5 file for each event. Optional. If the value of this entry is None, only the first 1024 peaks detected in each frame are written to the output file. Defaults to None.

 required
node_rank int

The rank of the OM node that writes the data in the output files.

 required

write_frame(*, processed_data)

Writes data into an HDF5 data file.

This function writes the provided data into an HDF5 data file, assuming that all the data belongs to the same processed data event.

Parameters:

Name Type Description Default
processed_data Dict[str, Any]

A dictionary containing the data to write into the HDF5 file.

 required

close()

Closes the file currently being written.

This function closes the HDF5 file that the class is currently writing.

get_current_filename()

Retrieves the path to the file currently being written.

This function retrieves the full path to the file that the class is currently writing.

Returns:

Type Description
pathlib.Path

The path to the file currently being written.

get_num_written_frames()

Retrieves the number of data events already written to the current file.

This function retrieves the number of data events that the class has already saved into the file that is currently writing.

Returns:

Type Description
int

The number of data events already written in the current file.

SumHDF5Writer

See documentation of the __init__ function.

__init__(*, powder_class, cheetah_parameters)

HDF5 writer for sum of frames.

This class creates HDF5 data files to store the aggregate information collected by Cheetah. the function saves into HDF5 files sums of detector data frames, together with virtual powder patterns created using the Bragg peaks detected in them.

Parameters:

Name Type Description Default
powder_class int

A unique identifier for the sum of frames and virtual powder pattern being saved.

 required
cheetah_parameters Dict[str, Any]

A dictionary containing the Cheetah parameters.

 required

write_sums(*, data)

Writes aggregated detector frame data into an HDF5 file.

This function writes the provided aggregated data into an HDF5 data file.

Parameters:

Name Type Description Default
data TypeClassSumData

A dictionary containing the aggregated data to write into the HDF5 file.

 required