LoadWave not finding column headers in CSV
Hello,
I work at a small business that uses Igor Pro for the majority of our data analysis/visualization purposes. I have always manually copied data from Excel and pasted into Igor, but today we were wondering how we could use the LoadWave function to make a macro that imports the data we plot to make things easier for employees that are less familiar with Igor/our datasets. Our data is often split across multiple CSVs, so we wanted to make a macro that could load specific columns from the first file in a dataset, then append to those columns when you run it and select the next file in the dataset.
I've run into one main issue while working on this: Igor does not seem to be able to extract the column headers from our CSVs! I'm not sure why, but when I run LoadWave with the /W/A flags (full command example: LoadWave/W/A/D/H/J/L={0,1,0,0,1}/P=$pathName/J fileName), Igor imports the data as wave0 instead of the column header. I found a procedure that seems to do what I would want (https://www.wavemetrics.com/code-snippet/load-waves-and-append), but currently I would end up with wave0, wave1, etc. in the Igor tables instead of the column headers, which isn't ideal. Would greatly appreciate any assistance figuring out this data headers issue!
Charles
Hi Charles,
Have tried just a manual loadwave with the default settings on your csv file? In your command you have unusual flags such as the /J flag twice and the /L={0,1,0,0,1} which means to load only the first column and the data starts at line 1. The headers are at line 0 so line 1 for the data is a pretty expected value.
I tried a pretty standard csv file with this command
LoadWave/J/D/W/K=0 "Macintosh HD:Users:andy.hegedus:Desktop:result.txt"
and it opens up a dialog window to confirm the wave names. It has read the names from the first line - line 0.
If I choose the auto name and go option, the command looks like this
LoadWave/J/D/W/A/K=0 "Macintosh HD:Users:andy.hegedus:Desktop:result.txt"
Notice the addition of the /A flag. The waves get loaded as expected with the names in the first line though with some Igor Pro made adjustments. two of the fields where called X and Y and since these are reserved names, they were loaded and named xW and yW.
Andy
November 18, 2024 at 04:50 pm - Permalink
Charles: If you attach your .csv file, or a truncated version of it that illustrates the issue, I will investigate.
Also please let me know which version of Igor you are using.
November 18, 2024 at 04:58 pm - Permalink
hegedus, hrodstein,
Thank you for being willing to offer input, I actually had an epiphany about this last night. The reason Igor was having trouble with our column headers was because a large chunk of the columns in our dataset have headers that are just integers instead of strings. I changed those columns to include a couple characters before the integers and Igor was able to load the column headers properly using the /W/A flags!
So, if anyone stumbles on this thread in the future with similar issues, check to make sure all the headers in your datafiles are things that would be recognized as strings by Igor.
Charles
November 19, 2024 at 08:16 am - Permalink
Good job figuring this out.
If all of your files use identical column names, another approach is to use the /B flag.
November 19, 2024 at 11:07 am - Permalink