|
| Command: |
 Preprocessing > Remove Spikes |
Spikes are quite common - especially in Raman spectroscopy - and have to be removed before the data is subjected to multivariate statistics. Epina ImageLab provides an efficient spike removal tool which is able to remove spikes automatically. The tool first detects the number of spikes and their intensity. Subsequently you have to define a threshold for the removal of the spikes.
| How To: |
- Select the layers to be processed.
- Define the half width of the expected spikes (don't set the half width too high, this will probably remove parts of valid peaks - usually a half width of 1 or 2 is sufficient).
- Select the detection level. The lower the level the more spikes are recognized.
- Select the type of baseline fit. Tick off "Use Linear Fit" for a linear fit, otherwise a parabolic fit is used. Please note that the parabolic fit is more sensitive but tends to detect spurious spikes at the beginning and at the end of a spectrum.
- Click the "Detect Spikes" button. This will analyze the spectra and create a list of spikes ordered by their relative intensity ("Score").
- Set a threshold for the removal of spikes with a score higher than the threshold. The best threshold varies from case to case, thus you should check the found spikes by clicking the top score entries in the list.
- Click the "Remove Spikes" button.
|
| Hint 1: |
If any of the spectra exhibits two or more spike which are close together you should repeat the spike detection and removal several times (using a half width of 1). |
| Hint 2: |
In order to mask the pixels exhibiting spikes (without removing the spikes) right-click the image and select "Add Red Pixels to a Pixel Mask". This will add all pixels exceeding the current threshold value to the pixel mask. If more than one pixel mask is available you will be asked which mask to use. |
| Hint 3: |
The spike detection algorithm is prepared to be executed in parallel on multi-core processors. The increase in speed corresponds roughly to the number of CPU cores in your system (a four core system will be faster by a factor of 4). You can fine-tune the assigned processing power in the preferences of the program (see the option Allowed CPU Usage in Multiprocessor Systems). |
| Hint 4: |
The following video  shows how to use the spike removal tool. |
Algorithm of the spike removal tool
The detection of spikes is based on the algorithm described below. Each detected spike is scored by the ratio of the height of the spike and the standard deviation of the signal noise around the spike. If a score exceeds the detection threshold (minimum 4.0) the spike is removed by replacing the corresponding signal values by the estimates obtained from the regression fit around the spike. This replacement is performed within an interval of +/- the expected half width of the spike.
Spike Detection. Spike detection is based on the assumption that neighboring data points in the spectrum are correlated to some extents (i.e. the spectrum shows bands of a certain minimum width). Types of spectra which do not fit this assumption (i.e. mass spectra or spectra containing individual lines instead of full spectra) must not be analyzed by the spike removal tool.
Each data point P0 in every spectrum is tested according to the following scheme (w is the expected half-width of the spikes):
- extract all data points within the windows [-9w...-w-1] and [w+1...9w]
- calculate a linear or a parabolic regression of these 16w points and the corresponding index number
- calculate the regression estimate y0 at index 0
- calculate the standard deviation of the residuals sy
- the test point p0 is considered to be part of a spike if sqrt(abs(yp-y0)/sy) exceeds the threshold ts.
w and ts may be set by the user. All pixels which exceed ts are indicated in red color. Hint: in order to identify even small spikes you should set the threshold to low values (meaningful values are greater than 3.0 which corresponds to a local signal/noise ratio of about 10).
|
Preprocessing 
