![](/profiles/wavemetrics/themes/wavemetrics/logo.png)
Loading an entire folder of csvs
![](/sites/default/files/styles/thumbnail/public/default_images/Artboard%201_1.png?itok=jeHOCIXy)
chemgoof
The csv data looks like:
Device Info
Serial Number 9310-01 93101011006
Record Type Differential
Unit Type Counts
Volume Type m3
Temperature Type ∞C
Minimum Average Maximum Std. Dev. Std. Err.
Channel 1 233943 246551.11 256851 5983.59 1410.35
Channel 2 162532 167664.5 170236 1991.57 469.42
Channel 3 112368 120411.83 124389 2941.69 693.36
Channel 4 3175 4390.89 4982 455.34 107.33
Channel 5 161 269.06 358 46.63 10.99
Channel 6 0 2.06 7 1.89 0.45
Record # Location Date & Time Instrument Status Sample Time Laser Status Ch1 Size (um) Ch1 Differential Ch1 Alarm
1 Unknown 2/14/14 14:54 (ok) 0:01:00 (ok) 0.3 241881
2 Unknown 2/14/14 14:55 (ok) 0:01:00 (ok) 0.3 244010
3 Unknown 2/14/14 14:56 (ok) 0:01:00 (ok) 0.3 244577
4 Unknown 2/14/14 14:57 (ok) 0:01:00 (ok) 0.3 237892
5 Unknown 2/14/14 14:58 (ok) 0:01:00 (ok) 0.3 233943
6 Unknown 2/14/14 14:59 (ok) 0:01:00 (ok) 0.3 248205
7 Unknown 2/14/14 15:00 (ok) 0:01:00 (ok) 0.3 243650
8 Unknown 2/14/14 15:01 (ok) 0:01:00 (ok) 0.3 246622
9 Unknown 2/14/14 15:02 (ok) 0:01:00 (ok) 0.3 255866
10 Unknown 2/14/14 15:03 (ok) 0:01:00 (ok) 0.3 256851
11 Unknown 2/14/14 15:04 (ok) 0:01:00 (ok) 0.3 252314
12 Unknown 2/14/14 15:05 (ok) 0:01:00 (ok) 0.3 246870
13 Unknown 2/14/14 15:06 (ok) 0:01:00 (ok) 0.3 240077
14 Unknown 2/14/14 15:07 (ok) 0:01:00 (ok) 0.3 245555
15 Unknown 2/14/14 15:08 (ok) 0:01:00 (ok) 0.3 247434
16 Unknown 2/14/14 15:09 (ok) 0:01:00 (ok) 0.3 252234
17 Unknown 2/14/14 15:10 (ok) 0:01:00 (ok) 0.3 251683
18 Unknown 2/14/14 15:11 (ok) 0:01:00 (ok) 0.3 248256
There are a total of 28 columns (I omitted for space) and I want to start at row 16 for the actual data.
The old code is
#pragma rtGlobals=3 // Use modern global access method and strict wave access. Menu "Aethalometer" //"Load 1st Aethalometer File...", LoadAethalometerFile("", "", 1) "Load Warehouse Aethalometer Files...", LoadWareFiles("") "Load Golf Course Aethalometer Files...", LoadGolfFiles("") End // LoadAethalometerFile(pathName, fileName, makeTable) // Returns a semicolon-separated list of waves loaded or "" if cancel. Function LoadWareAethalometerFile(pathName, fileName, makeTable) String pathName // Name of an Igor symbolic path or "". String fileName // Name of file or full path to file. Variable makeTable // 1 to make table, 0 to not make table // First get a valid reference to a file. if ((strlen(pathName)==0) || (strlen(fileName)==0)) // Display dialog looking for file. Variable refNum Open/D/R/F="*.csv"/P=$pathName refNum as fileName fileName = S_fileName // S_fileName is set by Open/D if (strlen(fileName) == 0) // User cancelled? return -1 endif endif // Now load the data. The /V flag specifies the accepted delimiters in the data file. // Add the /A flag if you don't want the "Loading Delimited Text" dialog. // Add the /O flag if you want to overwrite existing waves with the same names. // "Date" and "Time" are not available as wave names because they are Igor function names String columnInfoStr = "N=Ware_timewave;N=Ware_vTime;N=Ware_BC_ng_m3;N=Ware_UV_ng_m3;N=Ware_LPM;N=Ware_SBZ_IR;N=Ware_SBO_IR;N=Ware_RBZ_IR;N=Ware_RBO_IR;N=Ware_Bypass_Fraction_IR;N=Ware_Attenuation_IR;N=Ware_SBZ_UV;N=Ware_SBO_UV;N=Ware_RBZ_UV;N=Ware_RBO_UV;N=Ware_Bypass_Fraction_UV;N=Ware_Attenuation_UV;" String delimiters = "," // Comma-delimited String skipChars = "\" " // Skip quotes around date and time and spaces before commas LoadWave /J /D/A /K=0 /V={delimiters,skipChars,0,0} /B=columnInfoStr /R={English, 1, 3, 2, 1, "DayOfMonth-Month-Year", 40} /P=$pathName fileName Variable numWavesLoaded = V_flag // V_flag is set by LoadWave if (numWavesLoaded != 17) Print "Error loading file - wrong number of waves loaded" return -1 endif // Create reference to waves created by LoadWave Wave Ware_timewave,Ware_vTime Ware_timewave += Ware_vTime // Add time to date to obtain date/time KillWaves/Z Ware_vTime // This is no longer needed return 0 // Success End Function ConcatenateWareAethalometerFile(pathName, fileName) String pathName String fileName // Create a new data folder NewDataFolder/O/S WareAethalometerTemp Variable result = LoadWareAethalometerFile(pathName, fileName, 0) SetDataFolder :: // Back to original data folder if (result != 0) return result // Error loading new waves endif // Concatenate new waves onto old Concatenate/NP {:WareAethalometerTemp:Ware_timewave}, Ware_timewave Concatenate/NP {:WareAethalometerTemp:Ware_BC_ng_m3}, Ware_BC_ng_m3 Concatenate/NP {:WareAethalometerTemp:Ware_UV_ng_m3}, Ware_UV_ng_m3 Concatenate/NP {:WareAethalometerTemp:Ware_LPM}, Ware_LPM Concatenate/NP {:WareAethalometerTemp:Ware_SBZ_IR}, Ware_SBZ_IR Concatenate/NP {:WareAethalometerTemp:Ware_SBO_IR}, Ware_SBO_IR Concatenate/NP {:WareAethalometerTemp:Ware_RBZ_IR}, Ware_RBZ_IR Concatenate/NP {:WareAethalometerTemp:Ware_RBO_IR}, Ware_RBO_IR Concatenate/NP {:WareAethalometerTemp:Ware_Bypass_Fraction_IR}, Ware_Bypass_Fraction_IR Concatenate/NP {:WareAethalometerTemp:Ware_Attenuation_IR}, Ware_Attenuation_IR Concatenate/NP {:WareAethalometerTemp:Ware_SBZ_UV}, Ware_SBZ_UV Concatenate/NP {:WareAethalometerTemp:Ware_SBO_UV}, Ware_SBO_UV Concatenate/NP {:WareAethalometerTemp:Ware_RBZ_UV}, Ware_RBZ_UV Concatenate/NP {:WareAethalometerTemp:Ware_RBO_UV}, Ware_RBO_UV Concatenate/NP {:WareAethalometerTemp:Ware_Bypass_Fraction_UV}, Ware_Bypass_Fraction_UV Concatenate/NP {:WareAethalometerTemp:Ware_Attenuation_UV}, Ware_Attenuation_UV // Kill temp data folder KillDataFolder/Z :WareAethalometerTemp return 0 // Success End Function LoadWareFiles(pathName) String pathName // Name of symbolic path or "" to get dialog String fileName Variable index=0 if (strlen(pathName)==0) // If no path specified, create one NewPath/O temporaryPath // This will put up a dialog if (V_flag != 0) return -1 // User cancelled endif pathName = "temporaryPath" endif Variable result do // Loop through each file in folder fileName = IndexedFile($pathName, index, ".csv") if (strlen(fileName) == 0) // No more files? break // Break out of loop endif result = ConcatenateWareAethalometerFile(pathName, fileName) if (result == 0) // Did LoadAndGraph succeed? // Print the graph. fileName = WinName(0, 1) // Get the name of the top graph //String cmd //sprintf cmd, "PrintGraphs %s", fileName //Execute cmd // Explained below. //DoWindow/K $fileName // Kill the graph //KillWaves/A/Z // Kill all unused waves endif index += 1 while (1) if (Exists("temporaryPath")) // Kill temp path if it exists KillPath temporaryPath endif Wave Ware_timewave,Ware_vTime,Ware_BC_ng_m3,Ware_UV_ng_m3,Ware_LPM,Ware_SBZ_IR,Ware_SBO_IR,Ware_RBZ_IR,Ware_RBO_IR,Ware_Bypass_Fraction_IR,Ware_Attenuation_IR,Ware_SBZ_UV,Ware_SBO_UV,Ware_RBZ_UV,Ware_RBO_UV,Ware_Bypass_Fraction_UV,Ware_Attenuation_UV Edit Ware_timewave,Ware_BC_ng_m3,Ware_UV_ng_m3,Ware_LPM,Ware_SBZ_IR,Ware_SBO_IR,Ware_RBZ_IR,Ware_RBO_IR,Ware_Bypass_Fraction_IR,Ware_Attenuation_IR,Ware_SBZ_UV,Ware_SBO_UV,Ware_RBZ_UV,Ware_RBO_UV,Ware_Bypass_Fraction_UV,Ware_Attenuation_UV ModifyTable format(Ware_timewave)=8, width(Ware_timewave)=150 Sort Ware_timewave Ware_BC_ng_m3,Ware_UV_ng_m3,Ware_LPM,Ware_SBZ_IR,Ware_SBO_IR;DelayUpdate Sort Ware_timewave Ware_RBZ_IR,Ware_RBO_IR,Ware_Bypass_Fraction_IR;DelayUpdate Sort Ware_timewave Ware_Attenuation_IR,Ware_SBZ_UV,Ware_SBO_UV,Ware_RBZ_UV;DelayUpdate Sort Ware_timewave Ware_RBO_UV,Ware_Bypass_Fraction_UV,Ware_Attenuation_UV;DelayUpdate Sort Ware_timewave Ware_timewave DoWindow/K WareAethalometer DoWindow/C/T WareAethalometer,"Warehouse Aethalometer Data" return 0 // Signifies success. End End
I've altered it as:
#pragma rtGlobals=3 // Use modern global access method and strict wave access. Menu "Aerotrak" "Load Warehouse Aerotrak Files...", LoadWareFiles("") End // LoadAerotrakFile(pathName, fileName, makeTable) // Returns a semicolon-separated list of waves loaded or "" if cancel. Function LoadWareAerotrakFile(pathName, fileName, makeTable) String pathName // Name of an Igor symbolic path or "". String fileName // Name of file or full path to file. Variable makeTable // 1 to make table, 0 to not make table // First get a valid reference to a file. if ((strlen(pathName)==0) || (strlen(fileName)==0)) // Display dialog looking for file. Variable refNum Open/D/R/F="*.csv"/P=$pathName refNum as fileName fileName = S_fileName // S_fileName is set by Open/D if (strlen(fileName) == 0) // User cancelled? return -1 endif endif // Now load the data. The /V flag specifies the accepted delimiters in the data file. // Add the /A flag if you don't want the "Loading Delimited Text" dialog. // Add the /O flag if you want to overwrite existing waves with the same names. // "Date" and "Time" are not available as wave names because they are Igor function names String columnInfoStr = "N=Record;N=Location;N=DateW;N=Instrument_Status;N=Sample_Time;N=Laser_Status;N=Ch1_Size_um;N=Ch1_Differential;N=Ch1_Alarm;N=Ch2_Size_um;N=Ch2_Differential;N=Ch2_Alarm;N=Ch3_Size_um;N=Ch3_Differential;N=Ch3_Alarm;N=Ch4_Size_um;N=Ch4_Differential;N=Ch4_Alarm;N=Ch5_Size_um;N=Ch5_Differential;N=Ch5_Alarm;N=Ch6_Size_um;N=Ch6_Differential;N=Ch6_Alarm;N=Flow_Status;N=Volume_m3;N=Temperature;N=Humidity" String delimiters = "\t" //Tab-delimited String skipChars = "\" " // Skip quotes around date and time and spaces before commas LoadWave /J /D/A /K=0 /V={delimiters,skipChars,0,0} /B=columnInfoStr /R={English, 1, 3, 2, 1, "DayOfMonth-Month-Year", 40} /L={0,16,0,0,0}/P=$pathName fileName Variable numWavesLoaded = V_flag // V_flag is set by LoadWave if (numWavesLoaded != 28) Print "Error loading file - wrong number of waves loaded" return -1 endif // Create reference to waves created by LoadWave //Wave Ware_timewave,Ware_vTime //Ware_timewave += Ware_vTime // Add time to date to obtain date/time //KillWaves/Z Ware_vTime // This is no longer needed return 0 // Success End Function ConcatenateWareAerotrakFile(pathName, fileName) String pathName String fileName // Create a new data folder NewDataFolder/O/S WareAerotrakTemp Variable result = LoadWareAerotrakFile(pathName, fileName, 0) SetDataFolder :: // Back to original data folder if (result != 0) return result // Error loading new waves endif // Concatenate new waves onto old Concatenate/NP {:WareAerotrakTemp:Record}, Record Concatenate/NP {:WareAerotrakTemp:Location}, Location Concatenate/NP {:WareAerotrakTemp:DateW}, DateW Concatenate/NP {:WareAerotrakTemp:Instrument_Status}, Instrument_Status Concatenate/NP {:WareAerotrakTemp:Sample_Time}, Sample_Time Concatenate/NP {:WareAerotrakTemp:Laser_Status}, Laser_Status Concatenate/NP {:WareAerotrakTemp:Ch1_Size_um}, Ch1_Size_um Concatenate/NP {:WareAerotrakTemp:Ch1_Differential}, Ch1_Differential Concatenate/NP {:WareAerotrakTemp:Ch1_Alarm}, Ch1_Alarm Concatenate/NP {:WareAerotrakTemp:Ch2_Size_um}, Ch2_Size_um Concatenate/NP {:WareAerotrakTemp:Ch2_Differential}, Ch2_Differential Concatenate/NP {:WareAerotrakTemp:Ch2_Alarm}, Ch2_Alarm Concatenate/NP {:WareAerotrakTemp:Ch3_Size_um}, Ch3_Size_um Concatenate/NP {:WareAerotrakTemp:Ch3_Differential}, Ch3_Differential Concatenate/NP {:WareAerotrakTemp:Ch3_Alarm}, Ch3_Alarm Concatenate/NP {:WareAerotrakTemp:Ch4_Size_um}, Ch4_Size_um Concatenate/NP {:WareAerotrakTemp:Ch4_Differential}, Ch4_Differential Concatenate/NP {:WareAerotrakTemp:Ch4_Alarm}, Ch4_Alarm Concatenate/NP {:WareAerotrakTemp:Ch5_Size_um}, Ch5_Size_um Concatenate/NP {:WareAerotrakTemp:Ch5_Differential}, Ch5_Differential Concatenate/NP {:WareAerotrakTemp:Ch5_Alarm}, Ch5_Alarm Concatenate/NP {:WareAerotrakTemp:Ch6_Size_um}, Ch6_Size_um Concatenate/NP {:WareAerotrakTemp:Ch6_Differential}, Ch6_Differential Concatenate/NP {:WareAerotrakTemp:Ch6_Alarm}, Ch6_Alarm Concatenate/NP {:WareAerotrakTemp:Flow_Status}, Flow_Status Concatenate/NP {:WareAerotrakTemp:Volume_m3}, Volume_m3 Concatenate/NP {:WareAerotrakTemp:Temperature}, Temperature Concatenate/NP {:WareAerotrakTemp:Humidity}, Humidity // Kill temp data folder KillDataFolder/Z :WareAerotrakTemp return 0 // Success End Function LoadWareFiles(pathName) String pathName // Name of symbolic path or "" to get dialog String fileName Variable index=0 if (strlen(pathName)==0) // If no path specified, create one NewPath/O temporaryPath // This will put up a dialog if (V_flag != 0) return -1 // User cancelled endif pathName = "temporaryPath" endif Variable result do // Loop through each file in folder fileName = IndexedFile($pathName, index, ".csv") if (strlen(fileName) == 0) // No more files? break // Break out of loop endif result = ConcatenateWareAerotrakFile(pathName, fileName) if (result == 0) // Did LoadAndGraph succeed? // Print the graph. fileName = WinName(0, 1) // Get the name of the top graph //String cmd //sprintf cmd, "PrintGraphs %s", fileName //Execute cmd // Explained below. //DoWindow/K $fileName // Kill the graph //KillWaves/A/Z // Kill all unused waves endif index += 1 while (1) if (Exists("temporaryPath")) // Kill temp path if it exists KillPath temporaryPath endif Wave DateW,Instrument_Status,Sample_Time,Laser_Status,Ch1_Size_um,Ch1_Differential,Ch1_Alarm,Ch2_Size_um,Ch2_Differential,Ch2_Alarm,Ch3_Size_um,Ch3_Differential,Ch3_Alarm,Ch4_Size_um,Ch4_Differential,Ch4_Alarm,Ch5_Size_um,Ch5_Differential,Ch5_Alarm,Ch6_Size_um,Ch6_Differential,Ch6_Alarm,Flow_Status,Volume_m3 Edit DateW,Instrument_Status,Sample_Time,Laser_Status,Ch1_Size_um,Ch1_Differential,Ch1_Alarm,Ch2_Size_um,Ch2_Differential,Ch2_Alarm,Ch3_Size_um,Ch3_Differential,Ch3_Alarm,Ch4_Size_um,Ch4_Differential,Ch4_Alarm,Ch5_Size_um,Ch5_Differential,Ch5_Alarm,Ch6_Size_um,Ch6_Differential,Ch6_Alarm,Flow_Status,Volume_m3 ModifyTable format(DateW)=8, width(DateW)=150 Sort DateW DateW,Instrument_Status,Sample_Time,Laser_Status,Ch1_Size_um;DelayUpdate Sort DateW Ch1_Differential,Ch1_Alarm,Ch2_Size_um,Ch2_Differential,Ch2_Alarm;DelayUpdate Sort DateW Ch3_Size_um,Ch3_Differential,Ch3_Alarm,Ch4_Size_um;DelayUpdate Sort DateW Ch4_Differential,Ch4_Alarm,Ch5_Size_um,Ch5_Differential,Ch5_Alarm;DelayUpdate Sort DateW Ch6_Size_um,Ch6_Differential,Ch6_Alarm,Flow_Status,Volume_m3 DoWindow/K WareAerotrak DoWindow/C/T WareAerotrak,"Warehouse Aerotrak Data" return 0 // Signifies success. End
I believe the loadwave command is at fault, but I am getting an error. I want the files to load starting at line 16 and name the waves as specified. I really only added /L{0,16,0,0} to the loadwave command, but I'm getting an error and it only loads "record" and "location" as individual waves. Any ideas or easier solutions? Thanks
Exactly what is the error?
Also, if you posted a couple of sample files, zipped, that will make it possible for someone to investigate.
December 9, 2014 at 06:37 pm - Permalink
Delimited text load from "20130923_162617 9310-01 93101011006.csv"
Data length: 188, waves: Record_Location
Error loading file - wrong number of waves loaded
I've attached a couple of the files. The files are .csv.
December 10, 2014 at 12:26 pm - Permalink
Now error reads:
Delimited text load from "20140513_142452 9310-01 93101011006.csv"
LoadWave/B error while parsing T= flag for column 27
Error loading file - wrong number of waves loaded
December 10, 2014 at 12:40 pm - Permalink
December 10, 2014 at 12:54 pm - Permalink
December 10, 2014 at 12:55 pm - Permalink