Background Information on Spectral Descriptors

Note: The following text is a slightly edited excerpt of an article published in Spectroscopy Europe [Lohninger 2014].

Another important aspect of multisensor imaging is the statistical interpretation of the data. Although most researchers tend to use the available raw data for the development of multivariate models, we recommend introducing spectral descriptors instead which define and extract specific information. The definition and usage of spectral descriptors is a simple and efficient way (1) to cope with the "curse of dimensionality" [Bellman 1961] in multivariate data analysis and (2) to improve the structure of the information in the data space. Both the reduction of the dimensionality of the independent variable space and the improved data structure contribute to more sufficient multivariate models and faster calculations.

In order to explain the motivation behind the preference of using spectral descriptors instead of raw data let us discuss a few aspects of this approach in more detail. First, the well-known "curse of dimensionality" automatically results in an almost empty feature space which might reduce the effectiveness of multivariate models considerably. Despite the fact that in imaging the number of "samples" (i.e. the number of pixels constituing an image) is significantly higher than in classical (i.e. chemical) analysis, the feature space is nonetheless almost empty. For example, if you acquire spectra with 300 intensities measured along the wavelength axis, the feature space contains about 10⁶⁰⁰ cells (assuming that the intensity is measured with a resolution of 1%). Assuming further that the image consists of 200 by 200 pixels, we have 40000 samples - which is almost nothing compared to the huge size of the feature space.

This empty space might (and most probably will) result in poor multivariate classifiers; and furthermore, it is hard to evaluate the quality of this classifier since it is impossible to fill this space with a reasonable number of evenly distributed samples. Thus it is a good idea to reduce the size of the feature space by focusing on variables which contain information relevant to the investigated problem. At this point variable selection methods may help but will not entirely resolve the problems.

The second aspect which must not be overlooked is the chemical knowledge which is contained within the data, but which is distributed throughout the huge feature space. This knowledge can be "concentrated" in the feature space both by reducing the size of the space and by transforming the space so that chemical knowledge is encoded by derived variables ("spectral descriptors").

A small practical example should clarify the situation: Figure 2 shows the images obtained from an environmental sample analyzed by a Raman spectroscopy microscope. At the left the image resulting from raw intensities at 2917 cm^-1 is displayed, the center image shows the results from integration over the peak around 2917 cm^-1, and the right picture shows the image obtained by calculating the correlation to a template peak. This template peak is an idealized spectral peak, representing a certain class of substances (i.e. the band caused by stretching modes of CH-containing compounds).

In the first two cases can one easily recognize several more or less blurred spots. Looking at the spectra of two of these spots, we can see the difference between these two spectra. At location L1 there is a clear peak around 2917 cm^-1, while at spot L2 there are fluctuations originating from a broad underlying peak and/or excessive noise.

The three images result from three methods used to plot the information contained in the Raman spectrum around 2900 cm-1. Left: Image obtained from a single wave number (2917 cm-1); center: image obtained by integrating the intensities between 2840 and 3033 cm-1; right: image obtained by correlating a triangle template peak with the spectrum. It can be clearly seen that the correlation descriptor is much more selective than the other two approaches.

The idea behind spectral descriptors can now be used to encode the knowledge that a peak occurring at 2917 cm^-1 with a certain width most certainly will have a special meaning related to the class of substances (i.e., in this case, aliphatic compounds). So if we calculate the correlation of all spectra with a template spectrum containing only this single idealized peak we end up with an image which shows only a single large spot around L1. And indeed further analysis shows that this very spot is different from all the other spots containing organic substances, while the rest of the particles consist mostly of inorganic substances.

Hint:

The following video explains the basic ideas behind spectral descriptors.