How can you get the General Text load protocol to read the 0th line for a name instead of auto

Share icon

I am trying to upload a file type that comes from a Hitachi Absorbance spectrophotometer, an example file type will be uploaded below. The General text load protocol works relatively well without any tweaks but still requires the naming of each wave when the file name is already in the 0th line and therefore could be automated. The only problem I'm having is that when I try to assign the 0th line as the name line such as this (the first value is responsible for the name line):

LoadWave/G/D/W/L={0,0,0,0,0} "FilePath:"

the general text load protocol reads the 0 as "Auto" which automatically looks for names in the line above the start of the data lines. Is there any way in which I could get the general text load protocol to actually read "0" instead of auto? Another thought I had was that the starting line is always a set number of lines away from the start of the first data line (36 above). Is there any way to set a name line 36 lines above relative to the data line? Any other suggestions?

Thank you so much!

 Kl_Conc_Dep_21June22.txt (79.68 KB)

Hi,

Can you do it in two steps.  First open the file and read the first line to get the name.  Then you can run the general text load function with the previously stored name.

Andy

July 5, 2022 at 06:58 am - Permalink

I need to look into the behavior of Load General Text with your file as it is not behaving as I expect.

I can load your data using Load Delimited Text like this:

LoadWave/J/D/W/A/O/E=1/L={0,36,0,0,0}/V={"\t, ","",0,1}

The /V flag is necessary to include space in the list of delimters accepted by LoadWave/J. This is needed to get LoadWave/J to recognize line 0 ("##TITLE= Kl_Conc_Dep_10_21June22") as consisting of two wave names.

The last "1" means "Ignore blanks at end of wave" which is necessary because LoadWave/J treats "##END=" as a line of data and thus creates a blank row at the end of the waves.

But /W is not really appropriate because "##TITLE= Kl_Conc_Dep_10_21June22" is not two wave names.

You can load the file as delimited text with specific wave names, using /B instead of /W, like this:

LoadWave/J/D/B="N=XData;N=YData;"/A/O/E=1/L={0,36,0,0,0}/V={"\t, ","",0,1}

The file appears to be a JCAMP file which should be loadable by JCAMPLoadWave. However when I execute this:

JCAMPLoadWave/P=Load_General_Text/R/H/D/V/W

It loads only one column into one wave named wave0. I'm not sure why that is happening - possibly the file is not a proper JCAMP file.

Update: I see this in the "Loading JCAMP Files" help topic:

Because JCAMP-DX is intended primarily for evenly-spaced data, a single wave is produced for each data set.

I also see that JCAMPLoadWave has set the X scaling of the output wave appropriately based on the first column of data in the file (i.e., x0=700, dx=-0.5). So JCAMPLoadWave appears to be working correctly.

JCAMPLoadWave uses wave names like wave0 and does not base the wave name on the TITLE field of the file. If you want the wave to be named "Kl_Conc_Dep_10_21June22", then you will need to read the name from the file and rename the wave after loading it. I'll show you how to do this later if time permits.

 

July 5, 2022 at 01:38 pm - Permalink

Thanks, I've decided to just write my own upload script from scratch using FReadLine since I don't know how I would combine both the general text loading command and the FReadline command. I'll keep you updated. I would still be interested to know if anyone knows a way to deal with the problem directly and input just a "0" and not have the command read it as "Auto".

Thanks for the help,

Isaac 

July 5, 2022 at 02:21 pm - Permalink

Maybe this is still useful as starting point; this loads ESMA spectrum files (e.g. from SEM-EDS measurements). The spectrum wave is named according to file name. The structure is similar, i.e. header + data, although here it is only spectrum per file. Note that the header is saved as wave note.

If you change the constant DataKey to "*XYDATA*" you should be able to load the first spectrum in your file. Scaling could easily be achieved by extracting values from  ##FIRSTX and ##LASTX.

 

static strConstant DataKey = "*SPECTRUM*"
static strConstant EnergyKey ="*XPERCHAN*"
 
function LoadEMSAFile(S_fileName)
    String S_fileName       // full path to file to be loaded; "" for dialog
    variable refNum
       
    if(strlen(S_fileName) > 0)
        // full path is specified, load file directly
        Open/R refNum as S_fileName
    else
        // get dialog and valid reference to file, then open
        Open/D/R/F="*.txt" refNum
        if (strlen(S_FileName) == 0)
            return -1
        endif
        Open/R refNum as S_fileName
    endif
 
    string wNote=""
    variable i=0
    variable Scaling, DataStart
 
    do
        // read file line by line
        string line
        FReadLine refNum, line
        if(strlen(line) == 0)                      
            break                                      
        endif
 
        // save header line to string
        wNote += line
 
        // extract value for x-scaling
        if(StringMatch(line, EnergyKey))
            sscanf line, "#XPERCHAN    :%f", Scaling
        endif
 
        // stop reading when DataKey is hit
        if(StringMatch(line, DataKey))
            DataStart = i+1
            break
        endif
        i+=1
    while(1)
    Close refNum
 
    // load spectral part of the data, scale wave, add header as wave note, and rename wave to file name
    LoadWave/B="N='_skip_'; N=Counts;"/O/A/G/D/L={0,DataStart,0,0,0} S_fileName
    wave Counts
    SetScale/P x 0, Scaling, "keV", Counts
    Note/K Counts, wNote
    String wName = ParseFilePath(3, S_fileName, ":", 0, 0)
    Rename counts, $wName
   
    return 1
end

July 6, 2022 at 06:33 am - Permalink

I attempted to solve this using the JCAMPLoadWave operation but JCAMPLoadWave does not work with the file in question.

Igor's JCAMPLoadWave supports version 4.24 of the JCAMP standard which is described at
http://www.jcamp-dx.org/protocols/dxir01.pdf

That spec says:


(2.3.1) Restriction to single spectra. These specifica- tions apply in detail to simple JCAMP-DX files as de- scribed in Section 3.3.1. This limitation to a single spec- trum per JCAMP-DX file is in recognition of the fact that present infrared data systems are designed to store only one spectrum per internal file.

So version 4.24 does not support multiple spectra in one file. However, your file says that it is version 4.24 (see the JCAMP-DX label on the second line of the file) but it contains six spectra. So it is not really a version 2.4 JCAMP-DX file and Igor's JCAMPLoadWave loads only the first spectrum in the file.
 

July 6, 2022 at 08:34 pm - Permalink

Thank you so much hrodstein,

That code works really well! I had to add a "/P"  to the SetScale command in line 155 ish because it was ignoring the dx value otherwise, but now it works swimmingly.   

    WAVE w = $name      // Create wave reference
    SetScale /P x, x0, dx, "", w

Super sweet solution and very eye opening to me as a relatively new Igor user who is trying to learn to code. 

Edit: I tweaked your code further to accommodate our lab's preferences hrodstein's JCAMP loading procedure but slightly tweaked for our odd lab preferences | Igor Pro by WaveMetrics

Thanks again,

Isaac

July 8, 2022 at 06:45 am - Permalink

I had to add a "/P"  to the SetScale command in line 155

Thanks. I have fixed that.

 

July 8, 2022 at 10:10 am - Permalink