Loading a text file with space and tabs

Hello, 

I would like to load a series of .csv files that unfortunately contain a mixture of "tabs" and spaces. The procedure that I have to load the data works well for the data that is separated with tabs but not at all for the spaces. I have included an example of my data files plus the IGOR procedure used for loading. 

Any help is appreciated

Function Loadsmps() // Loads smps data
 
// Prompts to select folder into which data is loaded
String PathToFldr="",CtrlName="",PromptStr="Select folder to load smps data into:",CurrentFldr=getdatafolder(1) // variables for Browse4Folder
Browse4Folder(PathToFldr,CtrlName,PromptStr,CurrentFldr)    // prompts user to select an experiment
 
// Get folder with data in it
    NewPath/M="Select folder containing smps data:"/O/Q pathname    // prompts user to select path to data folder
    PathInfo pathname
// Abort if cancel is pressed
    If(V_flag==0)       
    Abort "proceedure cancelled"    
    EndIf
// Make matrix for smps data
    Make/o/n=(0,175) smpsDataMatrix
// Strings and Variables 
Variable numfolders,n, subdirs, numfiles, filenum, refnum   // variables
String folderlist, filename, name, currentfolder            //    strings
// Counts number of files in folder         
NumFiles=ItemsInList(IndexedFile(pathname,-1,".csv"))                               // # files in folder    , specify extensions e.g. ".txt"
    
    For(filenum=0;filenum<numfiles;filenum+=1) // loops through each file name  
        filename=s_path+StringFromList(filenum,IndexedFile(pathname,-1,".csv")) // get full path to data file   , specify extensions e.g. ".txt"
        If(Stringmatch(FileName,"*dmps*")==1)
            Open /R/P=pathname/Z refNum as filename // open
            FStatus refNum  // check there is data in file
                If(V_logEOF>10) // specify minimum file size to load e.g. >10bytes
                    Load_smps_File(filename, refnum)    // calls function to load file  // unique function for specific file format
                    Wave  Temp_smpsDataMatrix0  
                    //  add loaded waves to loaded data
                    InsertPoints DimSize(smpsDataMatrix,0),DimSize(Temp_smpsDataMatrix0,0), smpsDataMatrix
                    smpsDataMatrix[DimSize(smpsDataMatrix,0)-DimSize(Temp_smpsDataMatrix0,0),*][]=Temp_smpsDataMatrix0[p-DimSize(smpsDataMatrix,0)+DimSize(Temp_smpsDataMatrix0,0)][q]
                    currentfolder=GetDataFolder(1)          // root folder for experiment   
                Endif   // end data to folder allocation
        EndIf
        
    EndFor          // end loop through files of a dp
    
    Close/A             // close open files
        
    Killwaves Temp_smpsDataMatrix0
//  Do additional analytsis on smps data
    Processsmps()   // adds a time wave
 
End
 
 
 
Function Load_smps_File(filename, refnum)   // load single smps file
    String filename
    Variable refnum
    print "Loading: " + filename
    LoadWave/V={"\t, ","",0,0}/q/a/n=Temp_smpsDataMatrix/m/o/j/k=1/L={0,3,0,0,0} filename
 
End
 
Function Processsmps()      // Additional processing of smps data
    Wave smpsDataMatrix
    Make/o/d/n=(DimSize(smpsDataMatrix,0)) smpsTime
    Make/o/d/n=(DimSize(smpsDataMatrix,0)) smpsdate
    SetScale d,0,0,"dat" smpsTime 
        SetScale d,0,0,"dat" smpsdate
    smpsDate=Date2Secs(smpsDataMatrix[p][0],smpsDataMatrix[p][1],smpsDataMatrix[p][2])+smpsDataMatrix[p][3]*3600+smpsDataMatrix[p][4]*60+smpsDataMatrix[p][5]
smpsTime=smpsDataMatrix[p][3]*3600+smpsDataMatrix[p][4]*60+smpsDataMatrix[p][5]
End
End
 
e#pragma rtGlobals=1        // Use modern global access method.
 
Function Browse4Folder(PathToFldr,CtrlName,PromptStr,CurrentFldr)   // based on mg code from htdma toolkit
String PathToFldr   // path to data folder
String PromptStr    // Prompt text in data browser
String CurrentFldr // current data folder
String CtrlName
 
variable exit=0 // stops trying when exit reaches 6
do
    string/g BrowserList
    string /g TempFldr=CurrentFldr
    variable/g Flag
    execute "CreateBrowser prompt=\""+PromptStr+"\", select=tempfldr"   // prompt for file
    killstrings/z tempfldr
    svar BrowserList=s_browserlist
    nvar Flag=V_Flag
    // print browser_list
    if (!Flag)
            PathToFldr="cancelled"      
            break
        endif
        
        if (itemsinlist(BrowserList)==1)
            if (stringmatch("root:ROOT;", BrowserList))     //data browser returns strange path if root is selected
                BrowserList="root"
            endif
            PathToFldr=stringfromlist(0,BrowserList)+":"
            if(DataFolderExists(PathToFldr))
                break
            else
                doalert 0, "you must select a data folder, not a wave, string or variable! => try again"
            endif
        else
            doalert 0, "none or multiple data folders were selected => try again"
        endif
        exit+=1     
    while (exit<6)      //maximum six attempts
End

 

20190115-dmps_cpc.csv.csv (18.23 KB) 20190116-dmps_cpc.csv.csv (27.47 KB)

Since your file does not contain string date or date/time data, and since it does not contain blank data as far as I can tell, you can use LoadWave/G (Load General Text) instead of LoadWave/J (Load Delimited Text).

Load General Text automatically accepts spaces or a tab as a delimiter. It also automatically skips the header to find the numeric data, so you don't need the /L or /K flags.

This worked for me to load the files you attached:

LoadWave/G/O/M/N=Temp_smpsDataMatrix/Q

For more information, execute:

DisplayHelpTopic "Loading General Text Files"

DisplayHelpTopic "Loading Delimited Text Files"

Hello hrodstein, 

Thank you very much for getting back to me and apologies for the delay in my reply. However, I am having the same problem. The first 14 columns load correctly and then the 14th column is copied for the remainder of the matrix. 

When I check the raw files it seems that there are tabs at the start and then these change to spaces afterwards.

Thanks again for your help.

 

 

In reply to by LaMP

[quote=LaMP]The first 14 columns load correctly and then the 14th column is copied for the remainder of the matrix.[/quote]

I don't see that. For me, the waves load correctly using the command the I listed above.

I am using Igor Pro 8.04.

I will attach a zip archive containing my test experiment and the result of executing the LoadWave command above as well as the files that I loaded. The files are your files renamed for simplicity.

If you need further help, please specify your Igor version and include a self-contained example like the one I am attaching that reproduces the problem.

LAMP1_0.zip (30.4 KB)

Hello again, 

Problem solved. I had copied the new code snipet into the wrong load procedure :/, so that is why I didnt see any change! Thanks again for your help. 

For info, I am still working in IGOR 6. 

Forum

Support

Gallery

Igor Pro 10

Learn More

Igor XOP Toolkit

Learn More

Igor NIDAQ Tools MX

Learn More