next up previous
Next: Record Types Up: Unofficial Format of IDL Previous: File Format


Basic Record Structure

Each record has a variable length, but the beginning of the record has a standard format up until IDL version 5.4:


Format: RHDR        (format of every record header in save file)
Type Name Description
LONG RECTYPE Numerical record format code
ULONG NEXTREC0 Offset to next record, relative to start of file, low order 32-bits
ULONG NEXTREC1 Offset to next record, relative to start of file, high order 32-bits
LONG$\times$1 -- Unknown
-- -- Remainder of record


Thus, every record header contains at least four long words, but is often longer, depending on the record type. The first word contains the format type code. The type codes are described in more detail below, but there is only one format per code. The second and third words contain the offset of the next record measured from the beginning of the file. This is how a reader walks through an IDL save file: it must read each record header to determine the record type, and the offset to the next record. When the reader is finished with the current record, it must seek (or POINT_LUN) to the position of the next record.

IDL is inconsistent in how it handles file sizes of greater than 4 gigabytes. In IDL version 5.3 and earlier, there is no support for large files (i.e. NEXTREC1 must be 0). IDL version 5.4 had a special way of handling such large files, with a different record header format, as described in the next paragraph. After IDL version 5.4, I believe that IDL added support for large files by allowing NEXTREC1 to be non-zero. The position of the next record would be computed as: NEXTREC = NEXTREC0 + NEXTREC1 * $2^{32}$.

Version 5.4 of IDL changed the record format in an incompatible way for large files. When dealing with a large file, IDL version 5.4 inserts a special record (PROMOTE64). If the PROMOTE64 record is present in the file, then the record header format changes to the following hybrid:


Format: RHDR64        (format of every record header in save files after PROMOTE64 record appears)
Type Name Description
LONG RECTYPE Numerical record format code
ULONG64 NEXTREC Offset to next record, relative to start of file
LONG$\times$2 -- Unknown


Note the change of LONG to ULONG64. This change increases the length of the record to 5 32-bit words instead of 4 32-bit words, and thus makes all records produced by IDL 5.4 incompatible with previous versions of IDL. This situation is dealt with using the PROMOTE64 keyword of the save library.

NOTE: where ``RHDR'' appears below as a data type, the actual data type is either RHDR or RHDR64, depending on whether the PROMOTE64 record has appeared in the record stream or not. Before PROMOTE64 appears, the normal record header is used (RHDR) and after, the hybrid header is used (RHDR64).

Obviously, the PROMOTE64 method is pretty crazy and should not be written by new software. However, software readers may need to be aware of such issues for compatibility with older versions of IDL software.



Subsections
next up previous
Next: Record Types Up: Unofficial Format of IDL Previous: File Format
Craig Markwardt 2011-12-21