LoadWave error - how to determine number of lines in a file before opening?

Share icon

I'll paste in the problem code and then explain the problem:

    if(WaveExists(SPP200Paths))
        sort /A SPP200Paths, SPP200Paths
        tmpWaveList = ""
        for(i =0;i<dimsize(SPP200Paths,0);i+=1)
            wfprintf currentFile, "%s" /R=(i,i) SPP200Paths
            LoadWave/J/M/U={0,0,1,0}/D/A=SPP200tmp/E=0/K=0/V={"\t,"," $",0,1}/L={19,20,0,0,0}/Q currentFile
            tmpWaveList += "SPP200tmp" + num2str(i) + ";"
        endfor
            Concatenate /NP=0 /KILL tmpWaveList, SPP200
            SetScale /P y, 0,1,SPP200
            ModifyControl CAPS_Analyse_CAS disable=0
//  columnLabel = "SPP200_Ch1 [counts]"
//  display SPP200[][(FindDimLabel(SPP200,1,columnLabel))]      
    endif


The file paths contained in the text wave SPP200Paths can refer to files which contain header data, but nothing else. I don't want to read the header data in this part of the function, - that's taken care of elsewhere. When the loop hits a file containing nothing but header information, I get the error "LoadWave error: "no data was found in file"". If I try to get rid of this warning using the /C flag to LoadWave, the Concatenate command generates errors (Concatenate error: "expected wave name").

So at last to the question - is there an elegant way to determine the number of rows in these delimited text files before calling LoadWave? If I could put some sort of IF statement in the loop it'd fix the problem i.e.

        if (LinesInCurrentFile > 20)
                Loadwave ...
        endif


But how would one go about determining LinesInCurrentFile?
First, try calling GetRTError(1) after calling LoadWave to see if there was an error. If so, you can assume the file contained no data.

If that is not good enough, you will have to parse the file yourself. Here is an example of a function that looks for a line that starts with two numbers:

static Function FindFirstDataLine(refNum)
    Variable refNum
    
    Variable lineNumber = 0
    do
        String text
        FReadLine refNum, text
        if (strlen(text) == 0)
            // No more lines
            return -1           // Signifies data not found
        endif
        
        // This looks for a line with blank space followed by a float followed by blank space followed by a float.
        String regExp = "^[[:blank:]]*[[:digit:]]+\\.[[:digit:]]+[[:blank:]]*[[:digit:]]+\\.[[:digit:]]+"
        if (GrepString(text, regExp) != 0)
            break               // Found it
        endif
            
        lineNumber += 1
    while(1)
    
    return lineNumber
End


This is called like this:

    Open/R/P=$pathName refNum as fileName
    firstDataLine = FindFirstDataLine(refNum)   
    Close refNum

October 5, 2010 at 08:16 am - Permalink

Thanks for the comments. I wrote some quick code which seems to suggest that if I use the /C flag to LoadWave then GetRTError doesn't pick up a runtime error. I'll have to look into either parsing the headers before reading, or checking the length of the wave returned by LoadWave tomorrow.

October 5, 2010 at 09:46 am - Permalink

Quote:
if I use the /C flag to LoadWave then GetRTError doesn't pick up a runtime error


You should not use /C. It is intended for experiment recreation procedures only.

Omit /C and use GetRTError.

October 5, 2010 at 12:59 pm - Permalink

hrodstein wrote:

You should not use /C. It is intended for experiment recreation procedures only.
Omit /C and use GetRTError.

Thanks for your help. I got so annoyed by the debugger popping up during expected occurrences (empty files) that I went with your other suggestion.

Static Function FindFirstDataLine(currentFile)
    String currentFile
    Variable refNum
    Variable lineNumber = 0
    String text = ""
    Open/R refNum as currentFile
    do
        FReadLine refNum, text
        FStatus refNum
        if (!GrepString(text, "[(?!)a-z*][^NaN]") && strlen(text) > 3)
            break
        endif
        if (V_filePos == V_logEOF)
            return -1
        endif
        lineNumber += 1
    while(1)
    Close refNum
    return lineNumber
End

October 7, 2010 at 08:04 am - Permalink

This may not work in your situation, but if the files that contain only header data are quite small relative to files that contain actual data, you could use GetFileFolderInfo and check the output variable V_logEOF to see if it's above some threshold.

October 8, 2010 at 06:43 am - Permalink