Receiving Data

Data can be received from the hardware by configuring one or more data pipes before starting measurements. An application can open as many data pipes as required for operation. All of them can have their own parameters, settings and types. The only limit is posed by the machine resources such as memory and processor power. The processing of data pipes is not parallelized. Therefore the amount of CPU cycles required to process one data unit (such as TDC or DLD events, camera blobs, camera raw images) is growing with number of data pipes opened.

Data pipes can be created by calling sc_pipe_open2() and read by calling sc_pipe_read2(). An exception is the USER_CALLBACKS pipe. This pipe does not support reading. During creation of this pipe, the application developer provides a set of callback functions, and the library invokes these functions at the moment where data is produced, or when certain events happen.

sc_pipe_close2() is used to close pipes and free associated resources.

Common configuration parameters and concepts

Before documenting the various pipe types, a few commonly used concepts and configuration parameters are discussed:

TDC events and DLD events

TDCs for applications without a detector (stand-alone TDCs) measure an electronic pulse (Stop pulse) occurring on one of their Stop channel inputs and generate a TDC event for each detected pulse. In the simplest case, this TDC event is only characterized by the index of the Stop channel and the digitized time value that reports the time difference between the Stop pulse and the Start pulse. Furthermore, the TDC event can carry the information of the Start counter at the time the event was registered. The Start counter keeps track of the number of Start pulses. For advanced use cases, additional information can be associated with every TDC event.

TDCs for delay-line detector applications need to observe the pulses occurring on multiple channels simultaneously, where each channel corresponds to one of the delay-line terminals. If coincident pulses are found on all channels, the firmware of the TDCs computes a single DLD event and passes it to the PC side (alternatively, the individual channel pulses may be passed to the PC side, so the software can both deliver the TDC events of the individual channels, as well as the combined DLD event). The DLD event carries a time stamp summed up from the TDC events. Additionally the detector position (x,y) has been reconstructed from time differences of the TDC events and is delivered as part of the DLD event. Again, for advanced use cases, additional information can be associated with every DLD event.

time value of TDC and DLD events

Many pipe types involve configuration with regard to a time axis. As mentioned in the previous section, TDC events carry a time stamp which is the time difference between Start pulse and Stop pulse (plus an arbitrary constant offset which can be counter-acted by tuning the CommonShift ini file parameter). DLD events are combined from multiple coincident TDC events and their time value is the sum of the TDC event time values. The time value is only useful, when the TDC is configured for an external Start pulse (Ext_Gpx_Start = YES in the ini file). The opposite setting (Ext_Gpx_Start = NO) has the TDC generating internal starts, which are not synchronous to the detected pulses or particles. In this case, the time values are random.

configuration parameter depth

This configuration parameter applies to image pipes, 1D histogram pipes and the 3D histogram pipe. These pipes commonly store numbers of detected events which can also be regarded as intensity values. The depth parameter controls the data type that is used for these intensity values. Each data type has a different size in memory. There are four data types for unsigned integer numbers of varying size and two data types for floating-point numbers of varying precision. Larger sized data types enable handling of larger intensity values. For example, setting depth = BS8 means the histogram can only store intensity values up to 255, whereas depth = BS32 enables a generous maximum intensity above 4.2 x 109. Floating-point numbers are useful, if the raw number of detected events shall be corrected by a calibration image. The calibration image introduces floating-point weightings that are within some interval around 1.0.

The depth parameter must be set to one of the enumeration values defined in bitsize_t.

configuration parameter binning and region of interest (roi)

Binning and ROI parameters appear in many pipe types. Binning has the subfields binning.x, binning.y, binning.time. ROI has the subfields roi.offset.x, roi.offset.y, roi.offset.time, roi.size.x, roi.size.y, roi.size.time.

We discuss the effect of these parameters by considering a DLD event, which contains a detector position (x,y) and a time stamp t. For a particular type and configuration of a histogram pipe, the (x,y,t) values can either be within a region of interest or outside. In the latter case, the DLD event is not added to the histogram. In the former case, the correct histogram element must be located and increased by one. To this end, a sequence of computational steps is applied to the DLD event:

  • The original x, y, t values are divided by the respective binning values, resulting in x1, y1, t1

  • the values specified in roi.offset are subtracted from x1, y1, t1, respectively, yielding x2, y2, t2. If any of the values x2, y2, t2 are below zero, the event was outside of the ROI and is discarded.

  • x2, y2, t2 are checked to be smaller than the values specified in the respective roi.size fields. If any of them do not fulfill this condition, the event was outside of the ROI and is discarded.

  • x2, y2, t2 serve as indices into the respective axes of the histogram (for those axes that the histogram contains) to the effect that one histogram element is selected. This element is increased by one. For example, an image pipe of type XY, has two axes, one for detector coordinate x and one for detector coordinate y. The intensity at image position (x2, y2) is increased by one, whereas the t2 has no significance at this point. t2 was only computed to check the region-of-interest condition for the time, thereby restricting the DLD events that formed the histogram to those that were inside a specified time interval.

This computational sequence defines some conventions how ROI and binning are interpreted.

  • the number of histogram elements is entirely defined by the subfields of roi.size. The memory requirement follows the number of histogram elements while additionally factoring in the size of one element according to the depth parameter.

  • The application of binning prior to the application of ROI means that the user-specified ROI is interpreted as given in binned units. Thus, changing the binning from 1 to 2, while leaving the roi parameters unchanged would have the effect that the histogram maps a larger detector area with half the resolution, keeping the number of histogram elements unchanged.

The above description also makes clear, why 1D histograms and 2D histograms for DLD data always use a region-of-interest definition for three axes (x, y, and time). Some of these axes are mapped onto histogram axes, the remaining ones just act as a filter.

Please note, valid values for the binning are powers of 2 (1, 2, 4, 8, 16, 32, …).

configuration parameter modulo

Modulo is an additional preprocessing step applied to the time value for those situations where DLD events are periodically generated with a repetition frequency that exceeds the capability of the Start In input (start frequencies above 5 MHz are problematic). In that case, a frequency divider is inserted between the original source of the Start pulses and the Start In input of the TDC. On the data, the frequency divider has the effect of extending the time range over a multiple of the original time period, showing multiple concatenated replicas of the original data along the time axis. The modulo operation enables adding up and eliminating the replicas, reducing the time range to the original period.

The modulo parameter is configured to the number of time bins corresponding to the time period of your original Start pulse frequency. However, for increased accuracy the modulo parameter is interpreted as a fixed-point decimal number with 5 bits after the comma. Effectively, if you have determined the number of time bins corresponding to your start period, you have to multiply this value by 32 and set this as the modulo value. Fine tuning of the modulo value can be done by repeating measurements and optimizing for sharpness of features in the data. Querying the length of one time bin in nanoseconds is possible via sc_tdc_get_binsize2().

During computation of histograms, the modulo operation is performed before binning and ROI filtering is applied.

Memory handling for pipe data (allocator_owner, allocator_cb parameters)

Due to the non-automatic memory handling in the C programming language, the question of allocation and deallocation of memory is very important. Currently, the scTDC1 has two ways of providing memory buffers for data pipes:

(1) ‘internal’ memory allocation: Memory buffers are allocated by the scTDC1 in a sequence-of-buffers fashion, such that typically every measurement produces one buffer per pipe (for those pipe types that generate 1D histograms, images, and 3D histograms): When reading such pipes, the first call to sc_pipe_read2() returns the first buffer whereas the second call to sc_pipe_read2() deallocates the first buffer and returns the second buffer. Calling either sc_pipe_close2() or sc_tdc_deinit2() deallocates all remaining buffers.

(2) ‘external’ memory allocation: The application supplies an allocator callback function in the parameter structure when creating a data pipe. In this case, the allocator callback function is invoked every time the data pipe processing algorithm needs a new buffer for the data (this will typically occur at the start of a measurement). Allocation and deallocation of the memory buffer is then the responsibility of the application. The ‘external’ memory allocation provides more flexibility to the application developer. For example, the application can choose to reserve a single memory buffer and pass it back each time its allocator callback function is invoked.

For an example demonstrating external memory allocation, see External Memory Allocation.

Overview of the various types of data pipes

The different kinds of pipes that can be opened, are listed in the enum sc_pipe_type_t, defined in File scTDC_types.h.

TDC_HISTO

For stand-alone TDC applications. Provides a histogram that resolves number of detected events as a function of the time value on a specified Stop channel input. The unit of the time value can be queried via sc_tdc_get_binsize2().

Create a variable of type sc_pipe_tdc_histo_params_t, for specifying the configuration of the pipe, then pass its address as the third argument to sc_pipe_open2(), whereas the second argument is set to TDC_HISTO.

The channel field selects the TDC Stop channel input.

Binning and a region of interest can be specified for the time values. Allowed binning values are powers of 2 (1, 2, 4, 8, 16, 32, …). The fields offset and size define the lower-end margin and the length of the time interval ROI, where TDC events are only added to the histogram if they are within the ROI. The size field also directly defines the number of histogram elements. ROI filtering works similary to what is described in configuration parameter binning and region of interest (roi), except there are no x, y values, just a time value.

Notes concerning configuration parameters:

DLD_IMAGE_XY

For DLD and camera applications. Provides an image that maps the detector area or the camera sensor area, or just a partial region of those areas.

Create a variable of type sc_pipe_dld_image_xy_params_t, fill its fields according to your preferred configuration, then pass its address to sc_pipe_open2(), whereas the second argument is set to DLD_IMAGE_XY.

Notes concerning configuration parameters:

For a code example, see Image Pipe.

DLD_IMAGE_XT

For DLD applications. Provides an image where the image horizontal axis maps the detector horizontal axis, but the image vertical axis maps the time axis of DLD events.

Create a variable of type sc_pipe_dld_image_xt_params_t, fill its fields according to your preferred configuration, then pass its address to sc_pipe_open2(), whereas the second argument is set to DLD_IMAGE_XT.

For details on configuration parameters, see DLD_IMAGE_XY.

DLD_IMAGE_YT

For DLD applications. Provides an image where the image horizontal axis maps the detector vertical axis and the image vertical axis maps the time axis of DLD events.

Create a variable of type sc_pipe_dld_image_yt_params_t, fill its fields according to your preferred configuration, then pass its address to sc_pipe_open2(), whereas the second argument is set to DLD_IMAGE_YT.

For details on configuration parameters, see DLD_IMAGE_XY.

DLD_IMAGE_3D

For DLD applications. Provides a 3D intensity matrix where the three matrix indices correspond to (1) detector coordinate x, (2) detector coordinate y, (3) time value of the DLD events. The memory requirement for one such matrix can easily reach several Gigabytes, so the roi.size parameters in the configuration need to be chosen carefully.

Create a variable of type sc_pipe_dld_image_3d_params_t, fill its fields according to your preferred configuration, then pass its address to sc_pipe_open2(), whereas the second argument is set to DLD_IMAGE_3D.

For details on configuration parameters, see DLD_IMAGE_XY.

DLD_SUM_HISTO

For DLD applications. Provides a 1D histogram showing number of events resolved by time value of the DLD event.

Create a variable of type sc_pipe_dld_sum_histo_params_t, fill its fields according to your preferred configuration, then pass its address to sc_pipe_open2(), whereas the second argument is set to DLD_SUM_HISTO.

For details on configuration parameters, see DLD_IMAGE_XY.

STATISTICS

For TDC and DLD applications. Provides a statistics record at the end of every measurement. The statistics record is defined by statistics_t.

For DLD applications, the statistics record delivers useful diagnostic information to judge the correct calibration of the detector read-out electronics and the particle load on micro-channel plates. Typically, for this purpose, the statistics_t fields counts_read[0][0] up to counts_read[0][15] and the events_found[0] as well as the events_received[0] are sufficient and should be displayed side by side in one bar graph, if the application has a graphical interface.

Create a variable of type sc_pipe_statistics_params_t, fill its fields according to your preferred configuration, then pass its address to sc_pipe_open2(), whereas the second argument is set to STATISTICS.

Notes concerning configuration parameters:

TMSTAMP_TDC_HISTO, TDC_STATISTICS, DLD_STATISTICS

(Rarely used and currently undocumented).

USER_CALLBACKS

See User callbacks interface.

DLD_IMAGE_XY_EXT

A variant of the DLD_IMAGE_XY pipe which enables an extended set of configuration parameters.

Configuration parameters are defined in sc_pipe_dld_image_xy_ext_params_t.

Currently undocumented.

BUFFERED_DATA_CALLBACKS

Used by our implementation of the Python software development kit (SDK). A realization of event data transfer different from the USER_CALLBACKS pipe with the option to largely reduce the frequency of callbacks. The less-frequent callbacks make it better suited for building interfaces to less-performant programming languages.

Configuration parameters are defined in sc_pipe_buf_callbacks_params_t.

PIPE_CAM_FRAMES

For camera applications. Provides one data set per camera frame (unlike many other pipes which provide one data set per measurement). The data set includes some frame-related meta information and the raw image data if the camera is currently set up to deliver raw images. This pipe is the most efficient way to access raw image data from the camera since the generation of copies is eliminated as much as possible. The memory buffers transmitted through this pipe are prepared independent of whether a PIPE_CAM_FRAMES pipe is open. In case, multiple PIPE_CAM_FRAMES are open, the memory buffers are shared between these pipes and every pipe instance can read at their own pace. Only those memory buffers that have been read by all PIPE_CAM_FRAMES pipes are being released.

There are zero configuration options for this pipe. Therefore, the third argument to be passed to sc_pipe_open2() is NULL (C), or nullptr (C++).

The data set delivered from sc_pipe_read2() is a memory buffer that starts with a sc_cam_frame_meta_t structure. The memory buffer remains valid until the next call to sc_pipe_read2(). If the flags field in this structure has the SC_CAM_FRAME_HAS_IMAGE_DATA bit set, the memory buffer contains a second region, which can be reached by advancing the pointer to the memory buffer by the number of bytes given in the field data_offset. The size of this second region in bytes can be calculated by width times height times [1 or 2]. The last factor is 1 if the field pixelformat equals SC_CAM_PIXELFORMAT_UINT8, or 2 if the field pixelformat equals SC_CAM_PIXELFORMAT_UINT16 (these constants are defined in sc_cam_pixelformat_t).

See Camera pipes for frame meta info, raw images and blobs for example code.

PIPE_CAM_BLOBS

For camera applications. Provides one data set per camera frame (unlike many other pipes which provide one data set per measurement). The data set comprises a list of blob coordinates pertaining to one camera frame. It can be read in a synchronized fashion with the PIPE_CAM_FRAMES pipe (even in blob mode, if the camera does not deliver raw image data, the PIPE_CAM_FRAMES pipe delivers meta data associated with each camera frame).

There are zero configuration options for this pipe. Therefore, the third argument to be passed to sc_pipe_open2() is NULL (C), or nullptr (C++).

The data set delivered from sc_pipe_read2() is a memory buffer that starts with a sc_cam_blob_meta_t structure, providing a data_offset field and the number of blobs given in the second region of the memory buffer. The second region is reached by advancing the pointer to the memory buffer by as many bytes as given in the data_offset field. The content of the second region is an array of element type sc_cam_blob_position_t and with a length given by the nr_blobs field.

The memory buffer remains valid until the next call to sc_pipe_read2().

See Camera pipes for frame meta info, raw images and blobs for example code.

User callbacks interface

Since version 1.3000.0, a new interface was added for receiving data in a sequence-of-events form which enables a lower-level, more flexible way of processing the data for the application than receiving pre-computed histograms. While such “event-based” pipes already existed before, they required decoding and delicate bit manipulation by the application developer, and they were rather limited in the size of data that could be transported per event. The new interface involves calling sc_pipe_open2() with the sc_pipe_type_t argument being set to USER_CALLBACKS (defined in File scTDC_types.h). It uses a set of callback functions provided by the application developer so that the library can actively push various kinds of events and data to the application in the correct chronological order as they appear while the library decodes the protocol stream coming from the hardware.

The set of callback functions provided by the application developer is defined in sc_pipe_callbacks and comprises callbacks for reacting to

  • the start of a measurement

  • the end of a measurement

  • a millisecond tick recorded by hardware and placed as a marker between TDC or DLD events

  • statistics data at the end of a measurement

  • TDC events

  • DLD events

For a code example, see User Callbacks Pipe.

DLD applications

The callback function for receiving DLD events obtains an array of the struct sc_DldEvent, which is defined in the file scTDC_types.h as follows:

struct sc_DldEvent
{
  unsigned long long start_counter;
  unsigned long long time_tag;
  unsigned subdevice;
  unsigned channel;
  unsigned long long sum;
  unsigned short dif1;
  unsigned short dif2;
  unsigned master_rst_counter;
  unsigned short adc;
  unsigned short signal1bit;
};

The meaning of the data fields for DLD events is given below:

start_counter : The start counter field of a detector event contains the number of start pulses applied as digital pulses on the “Start In” input of the TDC (which provides the time reference for time-resolved measurements). Time-resolved measurements require “Ext_Gpx_Start=YES” in the configuration/“ini” file of your TDC. Additionally, “ExtendedTimeRange = YES” is required to get access to (non-zero) start counter values.

time_tag : The time tag value is related to the “TAG In” input of the TDC. In the TDC configuration (“ini”) file, the configuration parameter “TimeTag” can be set to integer numbers from 0 to 6. Not all types of TDCs support the full range of modes — modes 0,1,2 are supported by most models built for DLD applications, while modes 3-6 are reserved to advanced models. “TimeTag=0” disables the “TAG In” input. “TimeTag=1” means that the time tag value delivered together with a detector event counts the number of 12.5 ns cycles elapsed since the last digital pulse on the “TAG In” input (timer mode). “TimeTag=2” means that the time tag value delivered together with a detector event counts the number of digital pulses on the “TAG In” input since the start of the current exposure. “TimeTag=3” and “TimeTag=4” have identical meaning and enable the usage of the ADC input and the State input, but disable the “TAG In”. “TimeTag=5” and “TimeTag=6” correspond to “TimeTag=1” and “TimeTag=2” but enable simultaneous use of the “State” input.

subdevice : Indicates the number of the subdevice, from which the DLD event originated, if you have a device that internally combines several TDC units.

channel : For standard 2D delay-line detectors, the channel field of an event is always zero (or it may deliver numbers from 0-3 which have no significance). In case of specialized types of detectors with multiple segments, the channel may indicate the segment.

sum : the time value of events with respect to a start/reference pulse applied to the Start In input.

dif1, dif2 : the detector coordinate x, y

master_rst_counter : the master rst counter field of a detector event counts the number of digital pulses applied to the “Master Reset Counter” input of the TDC. The value is only reset manually by the user or during initialization of the TDC. Manual reset is possible via the API function “int sc_tdc_zero_master_reset_counter (const int dd);”

adc : the ADC field of a detector event is a digital value corresponding to the analog voltage difference applied between the “ADC+” and “ADC-” (differential) inputs. The input is designed for voltage differences from -10 Volts to +10 Volts. The range from 0 to +10 Volt is mapped linearly to digital values 0 to 32767, and the range from -10 Volt to 0 V is mapped linearly to digital values from 32768 to 65536.

signal1bit : can be zero or one, corresponding to a low or high voltage level on the “State” input of the TDC. The value for the “State” input is only available if in the configuration (“ini”) file of the TDC, the parameter “TimeTag” is set to one of 3, 4, 5, 6.

TDC applications

If you are using the user callbacks pipe with a stand-alone TDC, you will be interested in TDC events, instead. The callback function for receiving TDC events obtains an array of the struct sc_TdcEvent, which is defined in the file scTDC_types.h as follows:

struct sc_TdcEvent
{
  unsigned subdevice;
  unsigned channel;
  unsigned long long start_counter;
  unsigned long long time_tag;
  unsigned long long time_data;
  unsigned long long sign_counter;
};

The meaning of the data fields for TDC events is given below:

subdevice: Indicates the number of the subdevice, from which the TDC event originated, if you have a device that internally combines several TDC units.

channel: The TDC channel on which the pulse was registered.

start_counter: the number of the start period in which the pulse was registered. The number is reset to zero at the start of a measurement.

time_data: the time of the event in number of time bins since the last start pulse

time_tag, sign_counter: the values in these fields may provide additional information related to respective inputs.