New LMD file format

Large buffers

With increased buffers the file format does not change. A few modifications have to be done in f_evt where i_used is checked (get event), or set (put event). MBS transport and f_evt write fixed length buffers as before. The only exception is the file header: only the used part will be written because with large buffers a file header in a full buffer is not useful.

New file format without buffer headers

If we restrict the variable buffer mode to MBS-DABC setups there is no need to write files in MBS. Eventually even the transport will not run, because the subevent buffers are sent by the collector. This gives more freedom to design a new file format. This format will be written by DABC and read by fLmd functions (get event). The format is quite simpler than the old one, because it has no buffer structure causing so much complications by event spanning.

A file has a file header as before, but with variable size. Behind follow data elements with sMbsHeader headers (length, type, subtype) allowing to identify, and process or skip them. Besides event data, time stamps may be inserted from the original MBS formatted buffers to preserve this time information. Writing/reading such a file would be very straight forward. The file header contains the number of data elements and the maximum size of elements. This information must be collected throughout the file writing and written on close into the file header. The file header is an sMbsFileHeader structure.
iMaxWords is the maximum size of elements,
iElements is the number of data elements.
The header is followed by additional data (size iUsedWords).

The file size is no longer restricted to 2GB. Optionally an element index is written at the end of the file. This allows for random access of elements in the file through this index table. The table itself has 32-bit values for the element offsets (in 32-bit). It can therefore address offsets up to 16GB in the file. If larger files are needed, the table can be created with 64-bit values giving unlimited addressing.

Note: This file format needs the rewind file function because the file header must be rewritten to store iMaxWords, iElements, and optionally the offset of the index table. This function is currently not implemented in the RFIO package, but will be done.

MbsDataFormats

All structures defined in sMbs.h and fLmd.h.

LmdImplementation

All functions in fLmd.c.

-- HansEssel - 25 Oct 2007