Extracting metadata from .dm3 files

Share icon

So Ive seen some scripts for doing this in imageJ, but Im terrible at java and I love the Igor environment, so I am curious how to go about doing it here, if its possible? Ive tried loading the files thru the various menu options, but the metadata waves get garbled every time so clearly I am doing it wrong so far.

Advice/tips much appreciated. Thanks.


EDIT: dm3 is a file format used by Gatan Digital Micrograph and are common in the microscopy world. Its basically a tiff with some metadata tucked in. I threw in an example file (its soot imaged with HRTEM if anybody is curious), need to rename it from .jpg to .dm3 as I couldnt upload it otherwise. 44-82969s1-87000X.jpg (4.16 MB)
So you were able to load at least the image data?

Here igor always complains with "out of memory" if I try to load your file (renamed to tiff before).

November 20, 2014 at 03:51 am - Permalink

I'm interested in an answer to this. I can't offer any help except to say that BioFormats in ImageJ displays the metadata for the file. As you know, you can extract parts of that using a script, which we do for files in other microscopy formats. We tend to export this extraction to Igor, but it would be great to do it direct.

November 20, 2014 at 04:59 am - Permalink

A quick look at the file suggests that it does not comply with TIFF standard (missing signature in the first few bytes) so perish the thought of using the ImageLoad operation to read these images. If you have sufficient information about the this file format you should be able to extract the relevant information using FBinRead (actually Open, FBinRead and Close).

A.G.
WaveMetrics, Inc.

November 20, 2014 at 12:45 pm - Permalink

I have knocked up a quick reader, based on the format description here (http://www.er-c.org/cbb/info/dmformat/#dm3) and here (http://rsb.info.nih.gov/ij/plugins/DM3Format.gj.html).
No guarantees that this is right.. but it should help you get something. Maybe someone can tidy up the code and make it more robust.
The meta data is put into a Notebook. I have suppressed the image data itself being put there.
Hope this helps,
Kurt

Code as follows:
 #pragma rtGlobals=3        // Use modern global access method and strict wave access.
Function LoadDM3()
    // User selects DM3 file
    // opens DM3 file
    // starts to read data from the file
    variable vRefNum
    string sFileOpenMessage="Select a DM3 file"
    string sFileFilter="DM3 files (*.dm3):.dm3;All Files:.*.*;"
    Open/R/Z=2/F=sFileFilter/M=sFileOpenMessage vRefNum
    // keep data from file open
    variable vErr=V_flag
    string sFullName=S_fileName
    string sCleanFileName=CleanUpName(ParseFilePath(3, sFullName, ":", 0, 0),0)
    if (vErr == -1)
        return -1           // User cancelled.
    endif
    if (vErr != 0)
        return vErr
    endif
    variable/G gvLayer=0 // indent or nested group layer
    variable/G gvPrintData=1 // =1 to allow printing of data, 0 to forbid (eg image data itself)
    NewNotebook/F=0/N=NBFileData
    LoadDM3NBSetData("Loading data from "+sFullName)
    string sData
    variable vData
    // start reading data from the file
    do
        // image file version
        FBinRead/B=2/F=3 vRefNum,vData // big-endian, 4 bytes
        if(vData!=3) // should be 3
            LoadDM3NBSetData("Error in file format (version)")
            break
        endif
        // number of bytes in file
        FBinRead/B=2/F=3 vRefNum,vData // big-endian, 4 bytes
        variable vNumBytes = vData
        LoadDM3NBSetData("Root tag directory size: "+num2str(vData))
        // byte order of tag data
        FBinRead/B=2/F=3 vRefNum,vData // big-endian, 4 bytes
        variable/G gvByteOrder // this will be the code in FBinRead/B=(gvByteOrder)
        if(vData==0) // big-endian
            gvByteOrder=2
            LoadDM3NBSetData("Data byte order: big-endian")
        elseif(vData==1) // little-endian
            gvByteOrder=3
            LoadDM3NBSetData("Data byte order: little-endian")
        else
            LoadDM3NBSetData("Error in file format (byte order)")
            break
        endif
        // Tag Group
        LoadDM3TagGroup(vRefNum)
        break
    while(1)
    FStatus vRefNum
    if(V_flag!=0) // is valid
        Close vRefNum       // Close the file.
    endif
    return vErr         // Zero signifies no error.
End

Function LoadDM3NBSetData(sData)
    string sData
    NVAR gvLayer
    NVAR gvPrintData
    if(gvPrintData)
        sData=padstring("",gvLayer*2,0x20)+sData+"\r"
        Notebook NBFileData setData=sData
    endif
End

Function LoadDM3TagGroup(vRefNum)
    variable vRefNum
    // loads a DM3 file Tag Group
    NVAR gvLayer
    variable vFlag
    variable vData
    gvLayer+=1
    // is this group sorted?
    FBinRead/F=1 vRefNum,vData //  1 byte
    variable vGroupSorted
    vGroupSorted=vData
    LoadDM3NBSetData("Tag Group Sorted = "+num2str(vGroupSorted))
    // is this group open?
    FBinRead/F=1 vRefNum,vData //  1 byte
    variable vGroupOpen
    vGroupOpen=vData
    LoadDM3NBSetData("Tag Group Open = "+num2str(vGroupOpen))
    // number of tags in group
    FBinRead/B=2/F=3 vRefNum,vData //  big-endian, 4 bytes
    variable vNumTags
    vNumTags=vData
    LoadDM3NBSetData("Number of tags in this group = "+num2str(vNumTags))
    // tag entries
    variable vTags
    for(vTags=0;vTags<vNumTags;vTags+=1)
        vFlag=LoadDM3TagEntry(vRefNum)
        if(vFlag!=0) // had an error
            return -1
        endif
    endfor // vTags
    gvLayer-=1
    return 0
End

Function LoadDM3TagEntry(vRefNum)
    variable vRefNum
    // loads a DM3 file Tag Entry
    NVAR gvLayer
    variable vFlag
    variable vData
    string sData
    // Tag type
    FBinRead/F=1 vRefNum,vData //  1 byte
    variable vTagType
    vTagType=vData // 21 = data, 20 = group
    if(vTagType==21) // data - TagType
        LoadDM3NBSetData("Tag type: 21 (Data)")
    elseif(vTagType==20) // Group
        LoadDM3NBSetData("Tag type: 20 (Group)")
    elseif(vTagType==0) // end of file
        LoadDM3NBSetData("End of file.")
        return -1
    else
        LoadDM3NBSetData("Error: Tag type.")
        return -1      
    endif
    // Tag label length
    FBinRead/B=2/F=2 vRefNum,vData //  2 bytes
    variable vTagLabelLength
    vTagLabelLength=vData
    // Tag label
    if(vTagLabelLength==0) // zero length
        sData="_no label_"
    else
        FReadLine/N=(vTagLabelLength) vRefNum, sData
    endif
    LoadDM3NBSetData("Tag Label: "+sData)
    NVAR gvPrintData
    if(cmpStr(sData,"Data")==0)
        gvPrintData=0 // forbid printing data
    else
        gvPrintData=1 // allow printing data
    endif
    // Tag instance
    if(vTagType==21) // data - TagType
        vFlag=LoadDM3TagType(vRefNum)
        if(vFlag!=0)
            return -1
        endif
    elseif(vTagType==20) // Group
        vFlag=LoadDM3TagGroup(vRefNum) // note: this is recursive
        if(vFlag!=0)
            return -1
        endif
    endif
    return 0
End

Function LoadDM3TagType(vRefNum) // tag data
    variable vRefNum
    string sData
    variable vData
    NVAR gvLayer
    FReadLine/N=(4) vRefNum, sData
    if(cmpstr(sData,"%%%%")!=0)
        LoadDM3NBSetData("Error in Tag (expected %%%%).")
        return -1
    endif
    // length of encoded type definition
    FBinRead/B=2/F=3 vRefNum,vData //  4 bytes
            // for a simple type this will = 1,
            // for a string this will = 2,
            // an array of a simple type will = 3,
            // structs have 1+2*f where f=number of fields in struct
        variable vDefinitionLength=vData       
        // encoded type info
    make/FREE/O/I/N=(vDefinitionLength) wEncodingType // 4-byte wave
    FBinRead/B=2/F=3 vRefNum,wEncodingType
   
    variable vIndex,vIndex2,vDataType,vSize
    switch(vDefinitionLength)
        case 1: // simple type
            LoadDM3EncodedType(vRefNum,wEncodingType[0],wEncodingType)
            break
        case 2:  // string type
            FBinRead/B=2/F=3 vRefNum,vData //  4 bytes
            LoadDM3EncodedType(vRefNum,vData,wEncodingType)
            break
        case 3: // array of simple type
            // wEncodingType[0]=20 - array
            // wEncodingType[1]= tag data type for array members
            // wEncodingType[2]= size of array
            LoadDM3NBSetData("Array Data ("+num2str(wEncodingType[2])+")")
            for(vIndex=0;vIndex<wEncodingType[2];vIndex+=1) // loop round fields
                LoadDM3EncodedType(vRefNum,wEncodingType[1],wEncodingType)
            endfor // vIndex
            break
        default: // struct or array type
            switch(wEncodingType[0])
                case 15: // struct type
                    // wEncodingType[0]=15 - group
                    // wEncodingType[1]=0 - usually
                    // wEncodingType[2]= number fields
                    // wEncodingType[3+field*2]= length field name =0
                    // wEncodingType[4+field*2]= tag data type for value[field]
                    LoadDM3NBSetData("Struct Data ("+num2str(wEncodingType[2])+")")
                    for(vIndex=0;vIndex<wEncodingType[2];vIndex+=1) // loop round fields
                        LoadDM3EncodedType(vRefNum,wEncodingType[4+vIndex*2],wEncodingType)
                    endfor // vIndex
                    break
                case 20: // array of structs type
                    // wEncodingType[0]=20 - array
                    // wEncodingType[1]=15 - group
                    // wEncodingType[2]=0 - (group name) usually
                    // wEncodingType[3]=number of entries in struct (field)
                    // wEncodingType[4+field*2]= length field name =0
                    // wEncodingType[5+field*2]= tag data type for value[field]
                    // wEncodingType[...]
                    // wEncodingType[last]= size of array (i.e. number of repeats of struct)
                    LoadDM3NBSetData("Array Data ("+num2str(wEncodingType[2])+")")
                    for(vIndex2=0;vIndex2<wEncodingType[DimSize(wEncodingType,0)-1];vIndex2+=1) // loop round array
                        for(vIndex=0;vIndex<wEncodingType[3];vIndex+=1) // loop round fields
                            LoadDM3EncodedType(vRefNum,wEncodingType[5+vIndex*2],wEncodingType)
                        endfor // vIndex
                    endfor // vIndex2
                    break
            endSwitch
    endSwitch
    return 0
End

Function LoadDM3EncodedType(vRefNum,vEncodingType,wEncodingType)
    variable vRefNum
    variable vEncodingType
    wave wEncodingType
    variable vData, vIndex
    string sData
    NVAR gvByteOrder
    NVAR gvLayer
    switch(vEncodingType)
        case 2: // short
            FBinRead/B=(gvByteOrder)/f=2 vRefNum,vData
            LoadDM3NBSetData("Short data: "+num2str(vData))
            break
        case 3: // long
            FBinRead/B=(gvByteOrder)/f=3 vRefNum,vData
            LoadDM3NBSetData("Long data: "+num2str(vData))
            break
        case 4: // ushort
            FBinRead/B=(gvByteOrder)/f=2/U vRefNum,vData
            LoadDM3NBSetData("UShort data: "+num2str(vData))
            break
        case 5: // ulong
            FBinRead/B=(gvByteOrder)/f=3/U vRefNum,vData
            LoadDM3NBSetData("ULong data: "+num2str(vData))
            break
        case 6: // float
            FBinRead/B=(gvByteOrder)/f=4 vRefNum,vData
            LoadDM3NBSetData("Float data: "+num2str(vData))
            break
        case 7: //double
            FBinRead/B=(gvByteOrder)/f=5 vRefNum,vData
            LoadDM3NBSetData("Double data: "+num2str(vData))
            break
        case 8: // boolean
            FBinRead/F=1 vRefNum,vData //  1 byte
            LoadDM3NBSetData("Boolean data = "+num2str(vData))
            break
        case 9: // char
            FReadLine/N=(1) vRefNum, sData
            LoadDM3NBSetData("Char data = "+sData)
            break
        case 10: // octet
            FBinRead/F=1 vRefNum,vData //  1 byte
            LoadDM3NBSetData("Octet data = "+num2str(vData))
            break
        case 15: // struct
           
            break
        case 18: // string
            for(vIndex=0;vIndex<wEncodingType[1];vIndex+=1) // loop round string length
                FBinRead/B=(gvByteOrder)/f=2 vRefNum,vData
                LoadDM3NBSetData("Unicode string data = "+num2str(vData)+ " ("+num2char(vData)+")")
            endfor
            break
        case 20: // array
       
            break
    endSwitch
End

November 21, 2014 at 01:16 am - Permalink

Wow guys, thanks!

I had almost given up on this, and got distracted by work for some time, only to come back and find all this great help. I was otherwise looking at doing this manually for several hundred images (and that is just one week's work).

I love these forums, really.

EDIT: Acquisition time is given as the ushort data (all the meta data is provided in the raw format), I've decoded the following

0-9 = 48-57
: = 58
space = 32
P = 80
M = 77
A should follow from this to be equal to 65 (77 - 12)

Now to hijack this script so that it pulls this out from a group of files, translates it and writes it to a wave. Fun fun =)

May 5, 2015 at 05:20 am - Permalink

All right, I got it running so that you select a folder, and then it will open each file, pull the time stamp, and put that data into a wave entry for easy use. Problem is just that it runs _real_ slow. I check to see if some of the metadata (like timestamps) had permanent positions in the .dm3 file, but no such luck. So I am forced to let the script read through the file piece by piece (with notebook disabled), until it finds the correct header, at which point it extracts the data.

Is it possible to search the entire open file for a specific string and then get that string position? This would be significantly faster, I imagine.

Here is the modified code if anybody wants to use it. Its ugly after I got done with it.

 #pragma rtGlobals=3        // Use modern global access method and strict wave access.
//Notebook function now commented out
Function extract_timestamps()

    string list_dm3_files, prompt_alert,file_name
    variable num_files
    variable/G i
    //prompt for path to data folder
    newpath/o dm3_data_folder
    pathinfo dm3_data_folder
   
    // load file names into string, sorting appropriately
    list_dm3_files = IndexedFile(dm3_data_folder, -1, ".dm3")
   
    // Sort using combined alpha and numeric sort
    list_dm3_files = SortList(list_dm3_files, ";", 16)

   

    //print list_dm3_files
    num_files = itemsinlist(list_dm3_files)
   
   
   
    if(num_files == 0)
        prompt_alert = "Could not find any DM3 data files in the chosen folder"
        doAlert 0, prompt_alert
        return 0
    endif
   
    //make an output wave to store timestamps
    make/o/d/n=(num_files) timestamps
   
   
    // cycle through each file, one at a time  
    for(i=0;i<num_files;i+=1)
    file_name = stringfromlist(i,list_dm3_files)
    LoadDM3(file_name)
    endfor
    appendtotable timestamps
    modifytable format(timestamps) = 7
    SetScale d 0,0,"dat", timestamps
   
End



Function LoadDM3(file_name)
    string file_name
    variable vRefNum
    Open/R/Z=2/P=dm3_data_folder vRefNum as file_name
    variable vErr=V_flag
   



    // User selects DM3 file
    // opens DM3 file
    // starts to read data from the file
    //variable vRefNum
    //string sFileOpenMessage="Select a DM3 file"
    //string sFileFilter="DM3 files (*.dm3):.dm3;All Files:.*.*;"
    //Open/R/Z=2/F=sFileFilter/M=sFileOpenMessage vRefNum
    // keep data from file open
    //variable vErr=V_flag
    //string sFullName=S_fileName
    //string sCleanFileName=CleanUpName(ParseFilePath(3, sFullName, ":", 0, 0),0)
    //if (vErr == -1)
    //  return -1           // User cancelled.
    //endif
    //if (vErr != 0)
    //  return vErr
    //endif
    variable/G gvLayer=0 // indent or nested group layer
    variable/G gvPrintData=1 // =1 to allow printing of data, 0 to forbid (eg image data itself)
    //NewNotebook/F=0/N=NBFileData
    //LoadDM3NBSetData("Loading data from "+sFullName)
    string sData
    variable vData
    // start reading data from the file
    do
        // image file version
        FBinRead/B=2/F=3 vRefNum,vData // big-endian, 4 bytes
        if(vData!=3) // should be 3
            //LoadDM3NBSetData("Error in file format (version)")
            break
        endif
        // number of bytes in file
        FBinRead/B=2/F=3 vRefNum,vData // big-endian, 4 bytes
        variable vNumBytes = vData
        //LoadDM3NBSetData("Root tag directory size: "+num2str(vData))
        // byte order of tag data
        FBinRead/B=2/F=3 vRefNum,vData // big-endian, 4 bytes
        variable/G gvByteOrder // this will be the code in FBinRead/B=(gvByteOrder)
        if(vData==0) // big-endian
            gvByteOrder=2
            //LoadDM3NBSetData("Data byte order: big-endian")
        elseif(vData==1) // little-endian
            gvByteOrder=3
            //LoadDM3NBSetData("Data byte order: little-endian")
        else
            //LoadDM3NBSetData("Error in file format (byte order)")
            break
        endif
        // Tag Group
        LoadDM3TagGroup(vRefNum)
        break
    while(1)
    FStatus vRefNum
    if(V_flag!=0) // is valid
        Close vRefNum       // Close the file.
    endif
    return vErr         // Zero signifies no error.
   
End
 
Function LoadDM3NBSetData(sData)
    string sData
    NVAR gvLayer
    NVAR gvPrintData
    if(gvPrintData)
        sData=padstring("",gvLayer*2,0x20)+sData+"\r"
        //Notebook NBFileData setData=sData
    endif
End
 
Function LoadDM3TagGroup(vRefNum)
    variable vRefNum
    // loads a DM3 file Tag Group
    NVAR gvLayer
    variable vFlag
    variable vData
    gvLayer+=1
    // is this group sorted?
    FBinRead/F=1 vRefNum,vData //  1 byte
    variable vGroupSorted
    vGroupSorted=vData
    //LoadDM3NBSetData("Tag Group Sorted = "+num2str(vGroupSorted))
    // is this group open?
    FBinRead/F=1 vRefNum,vData //  1 byte
    variable vGroupOpen
    vGroupOpen=vData
    //LoadDM3NBSetData("Tag Group Open = "+num2str(vGroupOpen))
    // number of tags in group
    FBinRead/B=2/F=3 vRefNum,vData //  big-endian, 4 bytes
    variable vNumTags
    vNumTags=vData
    //LoadDM3NBSetData("Number of tags in this group = "+num2str(vNumTags))
    // tag entries
    variable vTags
    for(vTags=0;vTags<vNumTags;vTags+=1)
        vFlag=LoadDM3TagEntry(vRefNum)
        if(vFlag!=0) // had an error
            return -1
        endif
    endfor // vTags
    gvLayer-=1
    return 0
End
 
Function LoadDM3TagEntry(vRefNum)
    variable vRefNum
    // loads a DM3 file Tag Entry
    NVAR gvLayer
    variable vFlag
    variable vData
    string sData
    // Tag type
    FBinRead/F=1 vRefNum,vData //  1 byte
    variable vTagType
    vTagType=vData // 21 = data, 20 = group
    if(vTagType==21) // data - TagType
        //LoadDM3NBSetData("Tag type: 21 (Data)")
    elseif(vTagType==20) // Group
        //LoadDM3NBSetData("Tag type: 20 (Group)")
    elseif(vTagType==0) // end of file
        //LoadDM3NBSetData("End of file.")
        return -1
    else
        //LoadDM3NBSetData("Error: Tag type.")
        return -1      
    endif
    // Tag label length
    FBinRead/B=2/F=2 vRefNum,vData //  2 bytes
    variable vTagLabelLength
    vTagLabelLength=vData
    // Tag label
    if(vTagLabelLength==0) // zero length
        sData="_no label_"
    else
        FReadLine/N=(vTagLabelLength) vRefNum, sData
    endif
   
    // EDIT of original code, introducing case for exporting data directly into IGOR
   
    if(cmpStr(sData, "Acquisition Time")==0)
    //read 5x to move to correct position
    FReadLine/N=(4) vRefNum, sData
    FBinRead/B=2/F=3 vRefNum,vData
    FBinRead/B=2/F=3 vRefNum,vData
    FBinRead/B=2/F=3 vRefNum,vData
    FBinRead/B=2/F=3 vRefNum,vData

    //read actual timestamp data
    wave timestamps
    Nvar gvByteOrder,i
    variable byte_num,hours,minutes,seconds,total_seconds
    variable timestampLength
    string timepiece
    string timestamp

    FBinRead/B=(gvByteOrder)/f=2/U vRefNum, vData  // read the first piece
    //print vData
    vData -= 48 // translates from unsigned short data format
    timepiece = num2str(vData)
    timestamp = timepiece
   
    FBinRead/B=(gvByteOrder)/f=2/U vRefNum, vData  // read the second piece
    //print vData
    vData -= 48
    if (vData != 10) // check for the semi-colon and describe length of stamp
    timepiece = num2str(vData)
    timestampLength = 6
    else
    timepiece = ":"
    timestampLength = 5
    endif
    timestamp = timestamp + timepiece

   
    for(byte_num=0;byte_num<timestampLength;byte_num+=1)
        FBinRead/B=(gvByteOrder)/f=2/U vRefNum, vData  // read the rest of it
        //print vData
        vData -= 48
        if (vData != 10) // check for the semi-colon
            timepiece = num2str(vData)
        else
            timepiece = ":"
        endif
    timestamp = timestamp + timepiece
    endfor
    print "The timestamp is :", timestamp
    hours = str2num(StringFromList(0, timestamp ,":"))
    // set 1 pm - 7 pm to military hours
    if (hours < 8)
        hours += 12
    endif
    total_seconds = hours * 3600
    minutes = str2num(StringFromList(1,timestamp,":"))
    total_seconds += minutes*60
    seconds = str2num(StringFromList(2,timestamp,":"))
    total_seconds += seconds
    timestamps[i] = total_seconds
   


    // continue with normal code
      else
    LoadDM3NBSetData("Tag Label: "+sData)
    NVAR gvPrintData
    if(cmpStr(sData,"Data")==0)
        gvPrintData=0 // forbid printing data
    else
        gvPrintData=1 // allow printing data
    endif
    // Tag instance
    if(vTagType==21) // data - TagType
        vFlag=LoadDM3TagType(vRefNum)
        if(vFlag!=0)
            return -1
        endif
    elseif(vTagType==20) // Group
        vFlag=LoadDM3TagGroup(vRefNum) // note: this is recursive
        if(vFlag!=0)
            return -1
        endif
    endif

    endif // End of the introduced case see above
    return 0
End
 
Function LoadDM3TagType(vRefNum) // tag data
    variable vRefNum
    string sData
    variable vData
    NVAR gvLayer
    FReadLine/N=(4) vRefNum, sData
    if(cmpstr(sData,"%%%%")!=0)
        //LoadDM3NBSetData("Error in Tag (expected %%%%).")
        return -1
    endif
    // length of encoded type definition
    FBinRead/B=2/F=3 vRefNum,vData //  4 bytes
            // for a simple type this will = 1,
            // for a string this will = 2,
            // an array of a simple type will = 3,
            // structs have 1+2*f where f=number of fields in struct
        variable vDefinitionLength=vData       
        // encoded type info
    make/FREE/O/I/N=(vDefinitionLength) wEncodingType // 4-byte wave
    FBinRead/B=2/F=3 vRefNum,wEncodingType
 
    variable vIndex,vIndex2,vDataType,vSize
    switch(vDefinitionLength)
        case 1: // simple type
            LoadDM3EncodedType(vRefNum,wEncodingType[0],wEncodingType)
            break
        case 2:  // string type
            FBinRead/B=2/F=3 vRefNum,vData //  4 bytes
            LoadDM3EncodedType(vRefNum,vData,wEncodingType)
            break
        case 3: // array of simple type
            // wEncodingType[0]=20 - array
            // wEncodingType[1]= tag data type for array members
            // wEncodingType[2]= size of array
            //LoadDM3NBSetData("Array Data ("+num2str(wEncodingType[2])+")")
            for(vIndex=0;vIndex<wEncodingType[2];vIndex+=1) // loop round fields
                LoadDM3EncodedType(vRefNum,wEncodingType[1],wEncodingType)
            endfor // vIndex
            break
        default: // struct or array type
            switch(wEncodingType[0])
                case 15: // struct type
                    // wEncodingType[0]=15 - group
                    // wEncodingType[1]=0 - usually
                    // wEncodingType[2]= number fields
                    // wEncodingType[3+field*2]= length field name =0
                    // wEncodingType[4+field*2]= tag data type for value[field]
                    //LoadDM3NBSetData("Struct Data ("+num2str(wEncodingType[2])+")")
                    for(vIndex=0;vIndex<wEncodingType[2];vIndex+=1) // loop round fields
                        LoadDM3EncodedType(vRefNum,wEncodingType[4+vIndex*2],wEncodingType)
                    endfor // vIndex
                    break
                case 20: // array of structs type
                    // wEncodingType[0]=20 - array
                    // wEncodingType[1]=15 - group
                    // wEncodingType[2]=0 - (group name) usually
                    // wEncodingType[3]=number of entries in struct (field)
                    // wEncodingType[4+field*2]= length field name =0
                    // wEncodingType[5+field*2]= tag data type for value[field]
                    // wEncodingType[...]
                    // wEncodingType[last]= size of array (i.e. number of repeats of struct)
                    //LoadDM3NBSetData("Array Data ("+num2str(wEncodingType[2])+")")
                    for(vIndex2=0;vIndex2<wEncodingType[DimSize(wEncodingType,0)-1];vIndex2+=1) // loop round array
                        for(vIndex=0;vIndex<wEncodingType[3];vIndex+=1) // loop round fields
                            LoadDM3EncodedType(vRefNum,wEncodingType[5+vIndex*2],wEncodingType)
                        endfor // vIndex
                    endfor // vIndex2
                    break
            endSwitch
    endSwitch
    return 0
End
 
Function LoadDM3EncodedType(vRefNum,vEncodingType,wEncodingType)
    variable vRefNum
    variable vEncodingType
    wave wEncodingType
    variable vData, vIndex
    string sData
    NVAR gvByteOrder
    NVAR gvLayer
    switch(vEncodingType)
        case 2: // short
            FBinRead/B=(gvByteOrder)/f=2 vRefNum,vData
            //LoadDM3NBSetData("Short data: "+num2str(vData))
            break
        case 3: // long
            FBinRead/B=(gvByteOrder)/f=3 vRefNum,vData
            //LoadDM3NBSetData("Long data: "+num2str(vData))
            break
        case 4: // ushort
            FBinRead/B=(gvByteOrder)/f=2/U vRefNum,vData
            //LoadDM3NBSetData("UShort data: "+num2str(vData))
            break
        case 5: // ulong
            FBinRead/B=(gvByteOrder)/f=3/U vRefNum,vData
            //LoadDM3NBSetData("ULong data: "+num2str(vData))
            break
        case 6: // float
            FBinRead/B=(gvByteOrder)/f=4 vRefNum,vData
            //LoadDM3NBSetData("Float data: "+num2str(vData))
            break
        case 7: //double
            FBinRead/B=(gvByteOrder)/f=5 vRefNum,vData
            //LoadDM3NBSetData("Double data: "+num2str(vData))
            break
        case 8: // boolean
            FBinRead/F=1 vRefNum,vData //  1 byte
            //LoadDM3NBSetData("Boolean data = "+num2str(vData))
            break
        case 9: // char
            FReadLine/N=(1) vRefNum, sData
            //LoadDM3NBSetData("Char data = "+sData)
            break
        case 10: // octet
            FBinRead/F=1 vRefNum,vData //  1 byte
            //LoadDM3NBSetData("Octet data = "+num2str(vData))
            break
        case 15: // struct
 
            break
        case 18: // string
            for(vIndex=0;vIndex<wEncodingType[1];vIndex+=1) // loop round string length
                FBinRead/B=(gvByteOrder)/f=2 vRefNum,vData
                //LoadDM3NBSetData("Unicode string data = "+num2str(vData)+ " ("+num2char(vData)+")")
            endfor
            break
        case 20: // array
 
            break
    endSwitch
End

May 13, 2015 at 05:44 am - Permalink

daggaz wrote:

Is it possible to search the entire open file for a specific string and then get that string position? This would be significantly faster, I imagine.

Just a thought - you could use FStatus to find the length of the file, then use PadString and FBinRead to read the entire file into a string variable. Then finally search this for your specific string using strsearch.
I don't know whether this would be faster.
HTH,
Kurt

May 13, 2015 at 11:43 pm - Permalink

Thanks Kurt, I'll play around with that. Its got to be faster than calling multiple recursive functions to search the file.

May 18, 2015 at 12:13 am - Permalink