Loading csv files

hrodstein wrote: It's not clear to me if the file already has the column names in it. In any event, if you want to use the same wave names for every file of this type that you load you can use the LoadWave B flag to specify the wave names. Then omit the /W flag so LoadWave will not look for them in the file. You may need to tell LoadWave the zero-based line number where the data begins using the /L flag.

If this does not help, post a sample file or a representative excerpt along with the Igor command you are trying to use. Then I should be able to advise the best approach.

Ideally, I'd like to combine date and time into one wave and then eventually graph time vs voltage over several weeks.

Oh, and the other thing I noticed is when I load the data, it skips the first two lines so the data for 12:50 and 12:55 are not there.

Menu "Load Waves"
	"Load ChemGoof File...", LoadChemGoofFile("", "")		// Add menu item to Data->Load Waves submenu
End

// LoadChemGoofFile(pathName, fileName)
// Returns a semicolon-separated list of waves loaded or "" if cancel.
Function LoadChemGoofFile(pathName, fileName)
	String pathName		// Name of an Igor symbolic path or "".
	String fileName			// Name of file or full path to file.

	// First get a valid reference to a file.
	if ((strlen(pathName)==0) || (strlen(fileName)==0))
		// Display dialog looking for file.
		Variable refNum
		Open/D/R/P=$pathName refNum as fileName
		fileName = S_fileName			// S_fileName is set by Open/D
		if (strlen(fileName) == 0)		// User cancelled?
			return -1
		endif
	endif
	
	// Now load the data. The /V flag specifies the accepted delimiters in the data file.
	// Add the /A flag if you don't want the "Loading Delimited Text" dialog.
	// Add the /O flag if you want to overwrite existing waves with the same names.
	
	// "Date" and "Time" are not available as wave names because they are Igor function names
	String columnInfoStr = "N=vDateTime;N=vTime;N=Flow;N=VRef;N=VMeas;"
	String delimiters = ","						// Comma-delimited
	String skipChars = "\""						// Skip quotes around date and time
	LoadWave /J /D /A /E=1 /K=0 /W /V={delimiters,skipChars,0,0} /B=columnInfoStr /R={English,1,3,1,2,"Year-Month-DayOfMonth",40} /P=$pathName fileName
	Variable numWavesLoaded = V_flag			// V_flag is set by LoadWave
	if (numWavesLoaded != 5)
		Print "Error loading file - wrong number of waves loaded"
		return -1
	endif
	
	Wave vDateTime = vDateTime		// Create reference to wave created by LoadWave
	Wave vTime = vTime				// Create reference to wave created by LoadWave
	vDateTime += vTime				// Add time to date to obtain date/time
	KillWaves/Z vTime					// This is no longer needed

	// Tweak date/time table formatting
	ModifyTable format(vDateTime)=8, width(vDateTime)=150

	return 0							// Success
End

Function/S LoadAethalometer()											//Commandline function name/syntax
	Variable refNum													//Define refNum as number
	String message = "Select one or more files"							//Defines text names/labels to use
	String outputPaths													//Defines "outputPaths" as text string
	String fileFilters = "Data Files (*.txt,*.dat,*.csv):.txt,.dat,.csv;"	//Dictates which file extensions can be chosen
		     fileFilters += "All Files:.*;"
 
	Open/D/R/MULT=1/F=fileFilters/M=message refNum				//Show open window to choose file(s) from
	outputPaths = S_fileName											//Sets output path as string variable in carriage-return-delimited list of the ful path
 
	if (strlen(outputPaths) == 0)										//Says print cancelled if you click cancel
		Print "Cancelled"
	else
		Variable numFilesSelected = ItemsInList(outputPaths, "\r")		//Defines number of files chosen to open
		Variable i														//Sets i as a number
		for(i=0; i<numFilesSelected; i+=1)								//Sets limits on i so if i=0, i is less than # files or i is bigger than 1, then do the following
			String columnInfoStr = "N=timewave;N=vTime;N=Flow;N=VRef;N=VMeas;"
			//Above sets names of wave labels being loaded
			String delimiters = ","										//defines text name "delimiters" as comma
			String skipChars = "\""										//defines "skipChars" as skipping the quote character
			String path = StringFromList(i, outputPaths, "\r")			//sets "path" as a string from the list of files loaded
			Printf "%d: %s\r", i, path									//Prints the Path and filename beingd loaded
			LoadWave/A/D/E=1/J/K=0/R={English, 1, 3, 2, 1, "DayOfMonth-Month-Year", 40}/B=columnInfoStr/V={delimiters,skipChars,0,0} path //Loads the data from the delimited files
			Wave timewave = timewave									//Defines timewave as a wave
			Wave vTime = vTime										//      "         vTime	  "    "
			ModifyTable format(timewave)=8							//Format column to combined Date and Time
			timewave += vTime											//Sets timewave equal to itself plus the vTime data
			RemovefromTable vTime									//Take the wave off the table
			Killwaves/Z vTime											//Erase the full vTime wave
		endfor
	endif
 
	return outputPaths		// Will be empty if user cancelled; Stops procedure
	
End

Okay. So changed Loadwave syntax to /E=2 and /K=1, but that then causes the concurrent files to load as blank cells and also kills all the data in the top file except the date...
So K=0 causes the waves to be detected as text and K=1 kills all the data. /E=2 with /K=0 gives me exactly what I needed in terms of format of all the numbers going into one table, but that doesn't help me when I need to do a concurrent calculation using those waves and they come out as text... I'M SO CLOSE! THIS IS SO FRUSTRATING!

Thanks

And now my waves are loading as textwaves

Menu "Load Waves"
	"Load ChemGoof File...", LoadChemGoofFile("", "", 1)
	"Concatenate ChemGoof File...", ConcatenateChemGoofFile("", "")
End

// LoadChemGoofFile(pathName, fileName, makeTable)
// Returns a semicolon-separated list of waves loaded or "" if cancel.
Function LoadChemGoofFile(pathName, fileName, makeTable)
	String pathName		// Name of an Igor symbolic path or "".
	String fileName			// Name of file or full path to file.
	Variable makeTable		// 1 to make table, 0 to not make table

	// First get a valid reference to a file.
	if ((strlen(pathName)==0) || (strlen(fileName)==0))
		// Display dialog looking for file.
		Variable refNum
		Open/D/R/P=$pathName refNum as fileName
		fileName = S_fileName			// S_fileName is set by Open/D
		if (strlen(fileName) == 0)		// User cancelled?
			return -1
		endif
	endif
	
	// Now load the data. The /V flag specifies the accepted delimiters in the data file.
	// Add the /A flag if you don't want the "Loading Delimited Text" dialog.
	// Add the /O flag if you want to overwrite existing waves with the same names.
	
	// "Date" and "Time" are not available as wave names because they are Igor function names
	String columnInfoStr = "N=vDateTime;N=vTime;N=Flow;N=VRef;N=VMeas;"
	String delimiters = ","						// Comma-delimited
	String skipChars = "\" "						// Skip quotes around date and time and spaces before commas
	LoadWave /J /D/A /K=0 /V={delimiters,skipChars,0,0} /B=columnInfoStr /R={English,1,3,1,2,"Year-Month-DayOfMonth",40} /P=$pathName fileName
	Variable numWavesLoaded = V_flag			// V_flag is set by LoadWave
	if (numWavesLoaded != 5)
		Print "Error loading file - wrong number of waves loaded"
		return -1
	endif
	
	// Create reference to waves created by LoadWave
	Wave vDateTime, vTime, Flow, VRef, VMeas
	
	vDateTime += vTime				// Add time to date to obtain date/time
	KillWaves/Z vTime					// This is no longer needed

	if (makeTable)
		// Tweak date/time table formatting
		Edit vDateTime, Flow, VRef, VMeas
		ModifyTable format(vDateTime)=8, width(vDateTime)=150
	endif

	return 0							// Success
End

Function ConcatenateChemGoofFile(pathName, fileName)
	String pathName
	String fileName

	// Create a new data folder
	NewDataFolder/O/S ChemGoofTemp
	
	Variable result = LoadChemGoofFile(pathName, fileName, 0)
	
	SetDataFolder ::		// Back to original data folder
	
	if (result != 0)
		return result	// Error loading new waves
	endif
	
	// Concatenate new waves onto old
	Concatenate/NP {:ChemGoofTemp:vDateTime}, vDateTime
	Concatenate/NP {:ChemGoofTemp:Flow}, Flow
	Concatenate/NP {:ChemGoofTemp:VRef}, VRef
	Concatenate/NP {:ChemGoofTemp:VMeas}, VMeas
	
	// Kill temp data folder
	KillDataFolder/Z :ChemGoofTemp
	
	return 0			// Success
End

I was just hoping there was a way to open multiple files at once using the /MULT and then have the concatenation embedded. This helps me see the logic in the programming though so this is great.

Function AppendMultipleChemGoofFiles()
	String list = GetMultipleTextFileDialog()
	if (strlen(list) == 0)
		return -1			// User cancelled
	endif

	Variable numFiles = ItemsInList(list, "\r")
	Variable index
	for(index=0; index<numFiles; index+=1)
		String fullPath = StringFromList(index, list, "\r")
		if (strlen(fullPath) == 0)
			break			// No more files
		endif
		ConcatenateChemGoofFile("Igor", fullPath)	// The path Igor will be ignored because we are using a full path
	endfor
End

Function SortChemGoofWaves()
	// Create reference to waves in current data folder
	Wave vDateTime, Flow, VRef, VMeas
	Sort vDateTime, vDateTime, Flow, VRef, VMeas
End

Menu "Load Waves"
	//"Load ChemGoof File...", LoadChemGoofFile("", "", 1)
	"Load ChemGoof File...", LoadAllFiles("")
End
 
// LoadChemGoofFile(pathName, fileName, makeTable)
// Returns a semicolon-separated list of waves loaded or "" if cancel.
Function LoadChemGoofFile(pathName, fileName, makeTable)
	String pathName		// Name of an Igor symbolic path or "".
	String fileName			// Name of file or full path to file.
	Variable makeTable		// 1 to make table, 0 to not make table
 
	// First get a valid reference to a file.
	if ((strlen(pathName)==0) || (strlen(fileName)==0))
		// Display dialog looking for file.
		Variable refNum
		Open/D/R/P=$pathName refNum as fileName
		fileName = S_fileName			// S_fileName is set by Open/D
		if (strlen(fileName) == 0)		// User cancelled?
			return -1
		endif
	endif
 
	// Now load the data. The /V flag specifies the accepted delimiters in the data file.
	// Add the /A flag if you don't want the "Loading Delimited Text" dialog.
	// Add the /O flag if you want to overwrite existing waves with the same names.
 
	// "Date" and "Time" are not available as wave names because they are Igor function names
	String columnInfoStr = "N=vDateTime;N=vTime;N=Flow;N=VRef;N=VMeas;"
	String delimiters = ","						// Comma-delimited
	String skipChars = "\" "						// Skip quotes around date and time and spaces before commas
	LoadWave /J /D/A /K=0 /V={delimiters,skipChars,0,0} /B=columnInfoStr /R={English,1,3,1,2,"Year-Month-DayOfMonth",40} /P=$pathName fileName
	Variable numWavesLoaded = V_flag			// V_flag is set by LoadWave
	if (numWavesLoaded != 5)
		Print "Error loading file - wrong number of waves loaded"
		return -1
	endif
 
	// Create reference to waves created by LoadWave
	Wave vDateTime, vTime, Flow, VRef, VMeas
 
	vDateTime += vTime				// Add time to date to obtain date/time
	KillWaves/Z vTime					// This is no longer needed
 
	if (makeTable)
		// Tweak date/time table formatting
		Edit vDateTime, Flow, VRef, VMeas
		ModifyTable format(vDateTime)=8, width(vDateTime)=150
	endif
 
	return 0							// Success
End
 
Function ConcatenateChemGoofFile(pathName, fileName)
	String pathName
	String fileName
 
	// Create a new data folder
	NewDataFolder/O/S ChemGoofTemp
 
	Variable result = LoadChemGoofFile(pathName, fileName, 0)
 
	SetDataFolder ::		// Back to original data folder
 
	if (result != 0)
		return result	// Error loading new waves
	endif
 
	// Concatenate new waves onto old
	Concatenate/NP {:ChemGoofTemp:vDateTime}, vDateTime
	Concatenate/NP {:ChemGoofTemp:Flow}, Flow
	Concatenate/NP {:ChemGoofTemp:VRef}, VRef
	Concatenate/NP {:ChemGoofTemp:VMeas}, VMeas
 
	// Kill temp data folder
	KillDataFolder/Z :ChemGoofTemp
 
	return 0			// Success
End
Function LoadAllFiles(pathName)
	String pathName			// Name of symbolic path or "" to get dialog
	String fileName
	Variable index=0

	if (strlen(pathName)==0)			// If no path specified, create one
		NewPath/O temporaryPath			// This will put up a dialog
		if (V_flag != 0)
			return -1						// User cancelled
		endif
		pathName = "temporaryPath"
	endif

	Variable result
	do			// Loop through each file in folder
		fileName = IndexedFile($pathName, index, ".csv")
		if (strlen(fileName) == 0)			// No more files?
			break									// Break out of loop
		endif
		result = ConcatenateChemGoofFile(pathName, fileName)
		if (result == 0)							// Did LoadAndGraph succeed?
													// Print the graph.
			fileName = WinName(0, 1)				// Get the name of the top graph
			//String cmd
			//sprintf cmd, "PrintGraphs %s", fileName
			//Execute cmd						// Explained below.

			//DoWindow/K $fileName			// Kill the graph
			//KillWaves/A/Z					// Kill all unused waves
		endif
		index += 1
	while (1)

	if (Exists("temporaryPath"))			// Kill temp path if it exists
		KillPath temporaryPath
	endif

	return 0						// Signifies success.
End 

Function AppendMultipleChemGoofFiles()
	String list = GetMultipleTextFileDialog()
	if (strlen(list) == 0)
		return -1			// User cancelled
	endif
 
	Variable numFiles = ItemsInList(list, "\r")
	Variable index
	for(index=0; index<numFiles; index+=1)
		String fullPath = StringFromList(index, list, "\r")
		if (strlen(fullPath) == 0)
			break			// No more files
		endif
		ConcatenateChemGoofFile("Igor", fullPath)	// The path Igor will be ignored because we are using a full path
	endfor
End

 String list = GetMultipleTextFileDialog() 

Function/S GetMultipleTextFileDialog()
	Variable refNum
	String message = "Select one or more files"
	String outputPaths
	String fileFilters = "Data Files (*.txt):.txt;"
	fileFilters += "All Files:.*;"

	Open /D /R /MULT=1 /F=fileFilters /M=message refNum
	outputPaths = S_fileName
	
	return outputPaths		// "" if user cancelled
End