Spectra Image Profile - Create profiles from, normalize and edit 2D / 3D spectral data

This project provides tools for working on 2D spectral data or layers of 3D data, and was inspired by the WaveMetrics Image Line Profile procedure (now replaced by the internal Line Profile tool). The basic functionality is similar to the official tool at first glance, but this project has many convenient features tailored for spectroscopic data such as detector images, two-dimensional scans or intensity maps. The main functionality is the horizontal or vertical data integration between two region-of-interest (ROI) lines to create a profile, which can be saved, compared or even used to modify the data itself. Other functions (such as normalization) are applied directly to the 2D data or layer. The tool works directly on the input data, but a backup is saved upon start. Several tools to work on 1D-3D data are included as well.

I tried my best to test everything thoroughly, but users should verify the correct behavior for critical applications. Bug reports and suggestions for new features are always welcome. Special thanks to Oleg Dyachok from Uppsala University for suggesting and testing improvements for version 4.0 and beyond.

User Interface controls

To start, select a 2D or 3D wave in the Data Browser and then go to ‘Spectra Tools’ -> ‘Image Profile …’ in the top menu. Or use the command line:

A quick overview of all controls can be found below:

Line profile controls

• The profile can be calculated in horizontal (h key) or vertical (v key) direction. Diagonal or free-form modes (like in the Line Profile tool) are not available.
• The Region-of-Interest (ROI) lines can be modified via the control variables, by dragging the lines directly with the mouse on the image or by using the cursor keys on the keyboard (holding the shift key speeds up movements by 4x and the ctrl / command key slows down movement by 1/2). Holding both the ctrl / command and shift keys slows the movement down to one delta unit (one row / column) for precise placement.
• To instantly go to the full (minimal) ROI width use the respective check boxes or use the page up / page down keys.
• Normalize Profile: The profile (not the image) will be normalized to fall between 0 and 1.
• Mouse Csr Profile: An additional profile appears for comparison, which shows the current row or column under the mouse cursor (toggle with the m key). This profile is not saved.
• Snap Lines to Grid: The ROI line positions are constrained to include only full rows / columns.

The Save Profile and Snapshot buttons

The Save profile button saves the current profile next to the input data using a name ending which depends on the selected range. The ending is of the format “_H(xxx-xxx)” (horizontal mode) or “_V(xxx-xxx)” (vertical mode), where xxx give the start and end of the current range, e.g., “mydata_H(16.7-19.3)”. If the full image is selected the ending will be just “_prH” or “_prV”, respectively. Holding the shift key while pressing the button will lock the ending to “_prH” / “_prV” regardless of the selected range. The range and width is always saved inside the output wave’s note.

The Snapshot button will instead create and display a backup of the current profile for comparison. Each button press updates this snapshot to the current one. This profile will not be saved.

Undoing changes to the 2D / 3D data

• Undo All: Resets all changes made to the 2D/3D data and returns to the condition at start.
• Undo Last: Reverts the last change made to the 2D/3D data by invoking a function. This will only work once and will not undo multiple steps.
• Accept & Quit: Closes the window while keeping all edits made to the 2D/3D data so far.
• Save as: Saves the 2D/3D data under a different name and undoes all edits made to the original data.

Normalization buttons

These functions work on the current layer. Currently, three normalization functions are implemented:

• Div. by Profile: This divides the 2D image by the current profile (each column in horizontal mode and each row in vertical mode, respectively). The profile is slightly smoothed before the division to avoid amplifying noise. Holding the ctrl / command key while pressing the button prevents this smoothing step. See further below how to modify the smoothing parameter.
• Each Maximum: Finds the maximum of each row (horizontal mode) or each column (vertical mode) and divides the row / column by this value. This is useful, for example, to equalize the data in one direction (use the Keep avg. Int. checkbox to prevent downscaling of the data to 1)
• A user-defined button (here labeled ‘Ring Currrent’): Normalize by user provided information. This can be any data parameter or wave saved in the current experiment. Examples are normalization by a secondary readout, time, concentration etc. See the last section on how to define your own functionality here.
• Keep average Int. checkbox: This will preserve the average intensity when normalizing the 2D data. Normalization involves division by some normalization term, which reduces the intensity. This checkbox will rescale the data back to the intensity level before normalization.

The Delete ROI Section button

This button removes the columns (horizontal) or rows (vertical) inside the Region Of Interest. If the data at the beginning of the wave is removed, the scaling is adjusted to account for the deleted points. However, if data in the center is removed the scaling cannot be adjusted, since a waves scaling must be always linear and continuous. Be aware that this may lead to a mismatch between the image data and the scaling. If the scaling is important (e.g., a continuous energy scale), either refrain from using this feature or take care to only remove sections at the edges. If 3D data is edited, this will delete the selected columns or rows on all layers.

The Subtract Profile button

The current profile will be subtracted from each column (horizontal mode) or row (vertical mode) of the 2D image or current layer of 3D data. This can be used, for example, to remove constant features from a measurement which appear undisturbed in part of the data and thus can be selected with the ROI lines. The Scale (x) control to the right of the button can be used to scale the intensity of the profile before subtraction.

This button has an additional functionality only in horizontal mode: The spectrum can be shifted each column step according to the Y-axis delta. In other words, each increasing column the Y-scale delta value is added to (or subtracted from) the profile’s x scaling, which subtracts the profile in diagonal direction. This is useful to subtract features which change with the parameter on the Y scale like, for example, spectral features moving with increasing photon energy.

• Hold ctrl / command to subtract the profile along increasing Y delta values.
• Hold alt / option to subtract the profile along decreasing Y delta values.

Below example shows the diagonal subtraction result:

The Shift +/- buttons

These two buttons only work horizontally (regardless of the selected mode) and are used to add or subtract the Y-axis scale from the X-axis. All layers of 3D data are processed. In other words, the X-scaling of each column is adjusted by its respective Y-scaling value; the data is recalculated into a parallelogram. This will straighten features in the data which were changing with increasing Y values. An example would be a photon-energy (Y) scan of a spectrum where certain spectral features increase in their (X) emission energy. Subtracting the photon energy scale from a photoelectron spectrum in a kinetic energy scale would give binding energy, while adding the photon energy scale reverts back to kinetic energy. The effect is illustrated again with above example data:

Hold ctrl / command to cut partial data after the shift has been applied (the parallelogram is cut back into a rectangle). Note that this might fail if there is not even one complete row of data left (the skew of the parallelogram is too large to fit a rectangle inside).

Align Features functionality

This algorithm looks for similar features of each column (horizontal mode) or row (vertical mode) of the 2D image or current layer of 3D data and tries to align these features for maximum overlap by shifting each column / row by a certain amount. The algorithm uses wave correlation and shifts the data in whole point steps (no interpolation) or sub-steps (interpolated) when the Sub-steps checkbox is enabled. This can be used, for example, to correct slight shifts of peaks in a spectrum or to estimate the shift of a peak versus some other parameter like time or position:

The algorithm uses the first column (bottom to top in horizontal mode) or row (left to right in vertical mode) of the data as reference and tries to align all other columns / rows to this reference. Hold shift to use the last row / column instead (top to bottom in horizontal mode and right to left in vertical mode).

Often, it is not desired to align to the full data with lots of different features. Setting the A and/or B cursors onto the image will limit the range from which the reference is selected to maximize overlap with only one selected feature. This might still fail if many similar features are present.

• Activate Smooth to smooth the data before aligning. This can improve the results for noisy data.
• Activate Align Edges to search for maximum overlap of edge features instead of peak features. The derivative of the data is used for this purpose.
• Substeps is activated by default, which aligns features in steps smaller than the distance of data points, i.e., by using interpolation. Deactivating this feature will instead move data in integer steps.
• Some data points will be outside the input range when shifting data. Usually, these out-of-bounds data points are set to NaN. Activating Fill Gaps will instead repeat the end points of the data to fill this region.
• Activate Save Result to write the shift values into a wave with the name ending "_shft". The output is scaled by the 2D data’s wave scaling.

Note that the algorithm is ‘dumb’ in a sense that it only looks for the maximum correlation and not if peak features really match etc. In the worst case this just aligns artifacts such as noise.

Image contrast and color controls

You can choose the used color table and adjust the contrast of the image or currently selected layer by using the drop-down menu and the adjustment sliders. Select the desired color table and slide the Min and Max controls until the contrast is as desired. The sliders only have a range between the maximum and minimum data value of the image or currently selected layer, so the contrast cannot be reduced further, only increased. If you reverse the position of the Min and Max sliders then the colors of the image will be inverted. Check the log scale checkbox to switch to a logarithmic scale for both the image and profile. Switching to this mode will not work if the data only contains negative or zero values.

Additional controls for 3D data (volumes)

Layer and Merge controls

The Layer setting chooses the currently displayed layer of a 3D data set. The profile will only be created from this layer and most functions for editing data will only act on this one layer (there are exceptions like Delete ROI Section and Shift). The Delete button will delete the currently shown layer. The Merge All button will merge all (remaining) layers into a single 2D image which can then be worked on further. This can be reverted by using the Undo buttons. The scaled position of the current layer will be displayed in the top-right corner of the image.

Displaying profiles from all layers

Activating Multiple Profiles will show profiles of all layers, which are generated from the current ROI, simultaneously for comparison with the main profile from the current layer (shown in a bold line). This is useful, for example, to follow small changes in signal throughout the layers. The Separation control can be used to change the vertical spacing between the profiles. This spacing is in percent of the maximum y value. These profiles will also be saved in a wave with the ending "_map". The maximum number of profiles (layers) for this functionality is 100.

Clicking Display Z-Profile will open a small graph which plots the change of intensity in z- (i.e., layer-) direction. This can be useful to track variations across layers. The type of information displayed can be changed via the pop-up menu. Currently average, maximum or minimum intensity can be selected.

Rotating a 3D volume

3D data can be rotated by 90 degrees around the principal axes x,y,z in clockwise and counter-clockwise direction. Use the designated buttons at the bottom of the panel for this purpose. Rotations can be undone using the Undo buttons. After rotation the top-most layer is displayed at the beginning. If the layer is empty (only zeros) then the image might look all white. This is no bug, but the normal way how empty layers are displayed. Rotation might reverse the x- and/or y-axes, depending on the new orientation of the volume's contents. For this reason, the data might 'look reversed' when displayed in a normal image plot.

Quick access functions in the menu

Some useful functions are available in the Spectra Tools menu:

• Append 1D,2D Datasets into 2D Data: Combines the columns (Y dimension) of multiple datasets (e.g., multiple measurements) into one consecutive 2D data wave. The X dimension is extended to cover the full X range of all datasets if needed.
• Stack 2D Datasets into 3D Data: Stacks multiple 2D spectra (e.g., MCP images) on top of each other to produce a 3D array.
• Profile of 2D\3D Data: Same as creating a profile over the full data in horizontal or vertical mode using the profile panel. This will give 1D data for a 2D input, or 2D data or a 3D input. Hold shift while invoking the menu to define a sub-range in the respective dimension for profile generation.
• 2D Image from 3D Data: Sums 3D data in the layer (array) direction to give a 2D image (e.g., gives a sum of all MCP images of a 3D array).
• Split Up 2D\3D Data: Separates 2D / 3D data into individual waves holding columns, rows or layers. The individual waves are saved in a folder with the name of the input.

In Igor 9, these options are also available in the right-click menu of the data browser.

Creating profiles, splitting or combining data from the command line

Some quick access functions can be directly invoked. To create a profile use:

Here dimension is an integer value from 0 to 2, with the meaning of 0 = vertical profile (average rows), 1 = horizontal profile (average columns), 2 = z profile (average layers). Use the optional 'from', 'to' or the 'from_Point', 'to_Point' values to define a sub-range in terms of scaled values or point values, respectively. If both types are specified the former takes precedence.

Data can be split along a dimension via:

Here, the dimension parameter has the same meaning as above.

To reduce the fidelity of data you can bin data points by using:

Here, binX, binY, binZ should be integer values greater or equal to 1, which decide by how much the data in the respective dimension is binned. For example, using binX = 2 will result in a wave with half of the points in the x (row) dimension. Data can be binned in multiple dimensions simultaneously.

To combine data in the x- (rows), y- (column) or z- (layer) directions, use:

The dimension parameter has the same meaning as above. wave_list is a string list of waves (use full paths if the waves are not in the current folder).

Procedure settings and user button code

The procedure header has two global constants which control the smoothing factor of …

• kNormProfileSmooth: Sets the smoothing factor as an integer for profile normalization. The profile is smoothed by this factor before used for division with the 2D data.
• kAlignSmooth: The initial smoothing factor applied to each row / column before the correlation is calculated. Useful to suppress noise. The factor can be adjusted at any time inside the panel.

The user-defined (normalization) button

Two functions and one global variable are used to provide the functionality for the user button:

• kUserInfoBtnName: Defines the button’s label and can be changed to reflect the purpose of the button. This name should be short to fit inside the button.
• Profile_ActivateUserInfoButton(inputWave): A function for checking whether the conditions for using the button are fulfilled. A return value of 1 activates, any other value disables the button. One could for example check here if the required information is contained in the 2D data or loaded in the current experiment.
• Profile_ExecuteUserInfoButton(inputWave): This function is called to do the normalization (or whatever you like to do here). Return a scaling factor for scaling the intensity to its average after the normalization step should the Keep average Int. checkbox be activated (returning 0 skips this step).

You can modify the constant and the two functions directly at the beginning of the procedure file, but it is more convenient to provide your own code in a separate file or inside the experiment environment. Use the Override keyword in front of the Function / StrConstant statement to override the standard definitions inside the package. Have a look at the example implementation inside the project’s procedure, which is used to normalize the ring current of synchrotron experiments saved inside the 2D data itself.

While the user button is intended for normalization, it is really a free-for-all button for any modification to the 2D data you may want. You could instead write a function which, for example, splits the image in half or applies an interpolation or does something completely different in each experiment file.

The following code shows the current implementation of the user-definable normalization button: