
Loading a text file with space and tabs

LaMP
Hello,
I would like to load a series of .csv files that unfortunately contain a mixture of "tabs" and spaces. The procedure that I have to load the data works well for the data that is separated with tabs but not at all for the spaces. I have included an example of my data files plus the IGOR procedure used for loading.
Any help is appreciated
Function Loadsmps() // Loads smps data // Prompts to select folder into which data is loaded String PathToFldr="",CtrlName="",PromptStr="Select folder to load smps data into:",CurrentFldr=getdatafolder(1) // variables for Browse4Folder Browse4Folder(PathToFldr,CtrlName,PromptStr,CurrentFldr) // prompts user to select an experiment // Get folder with data in it NewPath/M="Select folder containing smps data:"/O/Q pathname // prompts user to select path to data folder PathInfo pathname // Abort if cancel is pressed If(V_flag==0) Abort "proceedure cancelled" EndIf // Make matrix for smps data Make/o/n=(0,175) smpsDataMatrix // Strings and Variables Variable numfolders,n, subdirs, numfiles, filenum, refnum // variables String folderlist, filename, name, currentfolder // strings // Counts number of files in folder NumFiles=ItemsInList(IndexedFile(pathname,-1,".csv")) // # files in folder , specify extensions e.g. ".txt" For(filenum=0;filenum<numfiles;filenum+=1) // loops through each file name filename=s_path+StringFromList(filenum,IndexedFile(pathname,-1,".csv")) // get full path to data file , specify extensions e.g. ".txt" If(Stringmatch(FileName,"*dmps*")==1) Open /R/P=pathname/Z refNum as filename // open FStatus refNum // check there is data in file If(V_logEOF>10) // specify minimum file size to load e.g. >10bytes Load_smps_File(filename, refnum) // calls function to load file // unique function for specific file format Wave Temp_smpsDataMatrix0 // add loaded waves to loaded data InsertPoints DimSize(smpsDataMatrix,0),DimSize(Temp_smpsDataMatrix0,0), smpsDataMatrix smpsDataMatrix[DimSize(smpsDataMatrix,0)-DimSize(Temp_smpsDataMatrix0,0),*][]=Temp_smpsDataMatrix0[p-DimSize(smpsDataMatrix,0)+DimSize(Temp_smpsDataMatrix0,0)][q] currentfolder=GetDataFolder(1) // root folder for experiment Endif // end data to folder allocation EndIf EndFor // end loop through files of a dp Close/A // close open files Killwaves Temp_smpsDataMatrix0 // Do additional analytsis on smps data Processsmps() // adds a time wave End Function Load_smps_File(filename, refnum) // load single smps file String filename Variable refnum print "Loading: " + filename LoadWave/V={"\t, ","",0,0}/q/a/n=Temp_smpsDataMatrix/m/o/j/k=1/L={0,3,0,0,0} filename End Function Processsmps() // Additional processing of smps data Wave smpsDataMatrix Make/o/d/n=(DimSize(smpsDataMatrix,0)) smpsTime Make/o/d/n=(DimSize(smpsDataMatrix,0)) smpsdate SetScale d,0,0,"dat" smpsTime SetScale d,0,0,"dat" smpsdate smpsDate=Date2Secs(smpsDataMatrix[p][0],smpsDataMatrix[p][1],smpsDataMatrix[p][2])+smpsDataMatrix[p][3]*3600+smpsDataMatrix[p][4]*60+smpsDataMatrix[p][5] smpsTime=smpsDataMatrix[p][3]*3600+smpsDataMatrix[p][4]*60+smpsDataMatrix[p][5] End End e#pragma rtGlobals=1 // Use modern global access method. Function Browse4Folder(PathToFldr,CtrlName,PromptStr,CurrentFldr) // based on mg code from htdma toolkit String PathToFldr // path to data folder String PromptStr // Prompt text in data browser String CurrentFldr // current data folder String CtrlName variable exit=0 // stops trying when exit reaches 6 do string/g BrowserList string /g TempFldr=CurrentFldr variable/g Flag execute "CreateBrowser prompt=\""+PromptStr+"\", select=tempfldr" // prompt for file killstrings/z tempfldr svar BrowserList=s_browserlist nvar Flag=V_Flag // print browser_list if (!Flag) PathToFldr="cancelled" break endif if (itemsinlist(BrowserList)==1) if (stringmatch("root:ROOT;", BrowserList)) //data browser returns strange path if root is selected BrowserList="root" endif PathToFldr=stringfromlist(0,BrowserList)+":" if(DataFolderExists(PathToFldr)) break else doalert 0, "you must select a data folder, not a wave, string or variable! => try again" endif else doalert 0, "none or multiple data folders were selected => try again" endif exit+=1 while (exit<6) //maximum six attempts End

Forum

Support

Gallery
Igor Pro 9
Learn More
Igor XOP Toolkit
Learn More
Igor NIDAQ Tools MX
Learn More
Since your file does not contain string date or date/time data, and since it does not contain blank data as far as I can tell, you can use LoadWave/G (Load General Text) instead of LoadWave/J (Load Delimited Text).
Load General Text automatically accepts spaces or a tab as a delimiter. It also automatically skips the header to find the numeric data, so you don't need the /L or /K flags.
This worked for me to load the files you attached:
For more information, execute:
DisplayHelpTopic "Loading General Text Files"
DisplayHelpTopic "Loading Delimited Text Files"
October 22, 2019 at 02:31 pm - Permalink
Hello hrodstein,
Thank you very much for getting back to me and apologies for the delay in my reply. However, I am having the same problem. The first 14 columns load correctly and then the 14th column is copied for the remainder of the matrix.
When I check the raw files it seems that there are tabs at the start and then these change to spaces afterwards.
Thanks again for your help.
October 25, 2019 at 01:06 pm - Permalink
In reply to Hello hrodstein, Thank you… by LaMP
I don't see that. For me, the waves load correctly using the command the I listed above.
I am using Igor Pro 8.04.
I will attach a zip archive containing my test experiment and the result of executing the LoadWave command above as well as the files that I loaded. The files are your files renamed for simplicity.
If you need further help, please specify your Igor version and include a self-contained example like the one I am attaching that reproduces the problem.
October 25, 2019 at 02:56 pm - Permalink
Hello again,
Problem solved. I had copied the new code snipet into the wrong load procedure :/, so that is why I didnt see any change! Thanks again for your help.
For info, I am still working in IGOR 6.
October 25, 2019 at 11:17 pm - Permalink