Home Technical Documentation Data Formats Data Cube
See also: Metadata, Data Model, Overview, SetCubeCell, GetCubeCell 

The Data Cube

The data is stored in two files, a binary file containing the actual data (the pure numbers, file extension .cube) and a text file containing the metadata (which provides information about the interpretation of the pure numbers, file extension .ilab). The data cube ("pure numbers") is organized as a 4-dimensional cuboid of arbitrary side lengths. The individual data values are always 64 bit (double precision) floating point numbers according to the IEEE-754 standard.

The data cube's internal structure consists of a four-dimensional array of floating point numbers: 

        data : array[1..NumT, 1..NumL, 1..NumY, 1..NumX] of double
with NumX, NumY, NumL, and NumT specifying the number of the cube's cells along the X, Y, L (layer) and T (time) axis.

Data Cube File

The figure below shows the organisation of the binary file. The file is organized in records of 4096 bytes to ensure maximum transfer speed to/from the disk. The first record serves as a header containing the minimum information to ensure correct reading of the data (see below). All following records contain linear arrays of 512 IEEE-754 double precision values each.

The formal specification of the binary data structure is as follows: 
const
  BINDATARECSIZE = 512;  // size of binary data record = 4096 bytes

type
  T4DData = array of array of array of array of double;
  TDataRec = array[1..BINDATARECSIZE] of double;
  THeadRec = record  //has to be of same size as TDataRec
               NumX   : integer;     // 4 bytes
               NumY   : integer;     // 4
               NumL   : integer;     // 4
               NumT   : integer;     // 4
               DataID : string[255]; // 256 bytes
               Reserve: array[1..3824] of byte;  // rest to fill up
             end;
  TMetaTags = (mtDatetime, mtVersion, mtSizeX, mtSizeY, mtSizeL, mtSizeT,
               mtDescription, mtAuthor, mtAxidX, mtAxidY, mtAxidL,
               mtAxidT, mtDataCRC, mtCertiFree, mtPropsX, mtPropsY, mtPropsT, mtPropsL,
               mtSampleID, mtPhotos, mtMaskNames, mtFileType, mtTilePos, mtLayerTecDat,
               mtLinkedFiles, mtPixAttNames, mtPixAttribs);

const
  SAMPLEID = 'This is my best sample';
  METAVERSION = 4;         // current format version number of metadata (*.ilab)
  METATAGS : array[TMetaTags] of string =
                ('datetime', 'version', 'sizex', 'sizey', 'sizel', 'sizet',
                 'description', 'author', 'axidx', 'axidy', 'axidl', 'axidt',
                 'datacrc', 'certificate', 'propsx', 'propsy', 'propst', 'propsl',
                 'sampleid', 'photos', 'maskids', 'filetype', 'tilepos',
                 'layertecdat', 'linkedfiles', 'pixattnames', 'pixattribs');
The DataID parameter is a string containing arbitrary additional information which is currently not used by Epina ImageLab but may be used in future releases.

Sample Pascal code

The following Pascal code shows how to store the binary data. The data source is a 4-dimensional array which is stored in a file is given by the parameter FName. DataID should be an empty string (do not use it). 
(******************************************************************************)
procedure SaveCubeDataOnFile (FName: string; DataID: string);
(******************************************************************************)

var
  ix        : integer;
  m,k,j,i   : integer;
  poi       : integer;
  OFile     : file of TDataRec;
  DataRec   : TDataRec;
  HeadRec   : THeadRec;
  cnt       : int64;
  step      : int64;
  NumX      : integer;
  NumY      : integer;
  NumL      : integer;
  NumT      : integer;

begin
NumT := length(Data);                  // get the dimensions of the data matrix
NumL := length(Data[0]);
NumY := length(Data[0][0]);
NumX := length(Data[0][0][0]);
assignfile (OFile, FName);
rewrite (OFile);                       // open file for write
for ix:=1 to BINDATARECSIZE do         // reset data record
  DataRec[ix] := 0;
HeadRec.NumX := NumX;                  // fill the header
HeadRec.NumY := NumY;
HeadRec.NumL := NumL;
HeadRec.NumT := NumT;
HeadRec.DataID := DataId;
write (OFile, TDataRec(HeadRec));      // write header record
poi := 0;                              // now fill and write data records
cnt := 0;
step := round(1.0*NumT*NumL*NumX*NumY/100);
for m:=1 to NumT do
  for k:=1 to NumL do
    for j:=1 to NumY do
      for i:=1 to NumX do
        begin
        inc (cnt);
        if cnt mod step = 0 then
          FrmSaveCube.PBar1.Value := cnt/step;
        if poi < BINDATARECSIZE
          then begin
               inc(poi);
               DataRec[poi] := Data[m-1,k-1,j-1,i-1];
               end
          else begin
               write (OFile, DataRec); // write data record
               poi := 1;               // reset buffer pointer
               DataRec[poi] := Data[m-1,k-1,j-1,i-1];
               end;
        end;
write (OFile, DataRec);                // save the last (partially filled) data record
closefile (OFile);                     // close file
end;
After saving the actual data on the disk you have to store the meta information as well. The meta information is stored in a file with the same filename as the binary data cube but with a file extension '.ilab'. The metadata file is a text file using keywords which are defined in the constant array METATAGS (see above). The exact specification of the format of each meta tag can be found in the description of the meta tags. 
(******************************************************************************)
procedure SaveMetaDataOnFile (FName: string; NumX, NumY, NumL, NumT: integer; DataID: string);
(******************************************************************************)
// Please note: in order to make things simple most of the meta information
// is hard-coded in the following code. Of course, this has to be resolved
// in a non-example routine.

var
  OFile   : TextFile;
  i,j     : integer;
  DestDir : string;
  OldPath : string;
  astr    : string;

begin
DestDir := ExtractFilePath(FName);
assignfile (OFile, FName);
FileMode := fmOpenReadWrite;
rewrite (OFile);

// required meta tags....
// The following tags are required by Epina ImageLab for proper functioning

// format version tag
writeln (OFile, '\', METATAGS[mtVersion], ' ', IntToStr(METAVERSION));
// specification of data cube size
writeln (OFile, '\', METATAGS[mtSizeX], ' ', IntToStr(NumX));
writeln (OFile, '\', METATAGS[mtSizeY], ' ', IntToStr(NumY));
writeln (OFile, '\', METATAGS[mtSizeL], ' ', IntToStr(NumL));
writeln (OFile, '\', METATAGS[mtSizeT], ' ', IntToStr(NumT));
// scale properties of the axes
writeln (OFile, '\', METATAGS[mtPropsX], ' 1');   // x axis
writeln (OFile, '1;',NumX,':: 1.0 0.0; 1.0 0.0:N::px');
writeln (OFile, '\', METATAGS[mtPropsY], ' 1');   // y axis
writeln (OFile, '1;',NumY,':: 1.0 0.0; 1.0 0.0:N::px');
writeln (OFile, '\', METATAGS[mtPropsL], ' 1');   // layer (wavelength)
writeln (OFile, '1;',NumL,':uvvis: 1.0 400.0; 1.0 -400.0:N:1:nm');
writeln (OFile, '\', METATAGS[mtPropsT], ' 1');   // timeslots
writeln (OFile, '1;',NumT,':: 1.0 0.0; 1.0 0.0:N::');

// additional tags...
// the following tags are optional and are mainly used to
// describe the data
writeln (OFile, '\', METATAGS[mtDateTime], ' ', FormatDateTime ('yyyy-MM-dd HH:mm:ss.sss', Now));
writeln (OFile, '\', METATAGS[mtDescription], ' ',IntToStr(2));
writeln (OFile, 'Description Line 1
');
writeln (OFile, 'Description Line 2');
writeln (OFile, '\', METATAGS[mtAuthor], ' ', 'John Smith');
writeln (OFile, '\', METATAGS[mtSampleId], ' ', DataID);
writeln (OFile, '\', METATAGS[mtAxidX], ' ', 'x axis');
writeln (OFile, '\', METATAGS[mtAxidY], ' ', 'y axis');
writeln (OFile, '\', METATAGS[mtAxidL], ' ', 'lambda');
writeln (OFile, '\', METATAGS[mtAxidT], ' ', 'time');

closefile(OFile);
end;