Suspicious Pixels

Command:

Preprocessing > Detect Suspicious Pixels

Any real world raw data may contain all kinds of disturbances or erroneous values. These wrong values may result from physical processes (e.g. high energy cosmic rays which influence the detector) or may be the result of invalid system conditions (e.g. the overdriving of the data acquisition system). Epina ImageLab provides several ways to detect such pixels which originate from erroneous data. The detection of such "suspicious" pixels may be carried out on several levels, employing different methods of detection.

In particular Epina ImageLab supports the detection of the following types of errors:

• Uncorrelated pixels (background pixels)
• Uncorrelated pixels of high noise
• Pixels lacking spectral noise
• Noise-only pixels
• Extreme Values

The general approach in detecting and removing suspicious pixels is first to detect them by selecting one of the above-mentioned procedures. Next, the detected pixels may be transferred into a pixel mask which can be used to either ignore these pixels during further calculations or to interpolate these pixels from surrounding ones.

Involved Algorithms

Uncorrelated Pixels. If we assume that the lateral resolution of a hyperspectral image is better than the smallest feature size of the target, the adjacent pixels should be correlated to some extent. This means that pixels which are uncorrelated with the neighboring pixels should be considered as background noise. The detection of such pixels is carried out by calculating the correlation r_xy of each pixel with the mean spectrum of the neighboring 8 pixels. For indicating uncorrelated pixels the value 1-r_xy² is displayed.

Weighted Correlation with Neighboring Pixels. The same as above, but the correlation is weighted by the logarithm of the standard deviation s₀ of the center pixel: the pixel values of this image are calculated according to (1-r_xy²)*ln(1+s₀).

Pixels Lacking Spectral Noise. Sometimes measurement conditions during data acquisition result in noiseless data (i.e. if the analog digital converter is driven into over- or underflow). Noiseless parts of the data are determined by fitting a polynomial to a moving window of the spectrum and calculating the ratio of the residual standard deviation and the range of the spectral data. If this ratio exceeds a certain threshold (typically in the order of 1000) the data window is considered to contain low(zero) noise data. The resulting map depicts the percentage of noiseless values. Please note that the minimum detectable length of a noiseless spectral region is 13 data values.

Noise-Only Pixels. Pixels which do not contain any information ("noise-only pixels") are detected by applying a Wald-Wolfowitz runs test for serial correlation. The depicted value is the p-value of the test. A p-value greater than the level of signifcance (0.05 for most cases) indicates that the corresponding pixel may contain a random spectrum showing no serial correlation.

Extreme Values. Spectra containing values which are smaller than the lower threshold or greater than the upper threshold are considered suspicious. The thresholds are calculated by the following equations:

QDist := Q_0.999 - Q_0.001
ThreshLow = Q_0.001 - QDist
ThreshHigh = Q_0.999 + QDist

with Q_0.001 denoting the 0.001-quantile of the distribution of the data values, and Q_0.999 denoting the 0.999-quantile. Please note that the quantile is calculated by drawing 50000 random samples from the data. Thus the recognition of extreme values may change between two calculations if a particular pixel exhibits a value which is close to the threshold.