Data table time based splitting and exporting
Mirrorball
Hi,
I am new to Igor Pro and have limited programming knowledge. I would like to get advices on splitting data tables and exporting to csv.
I have a data table that I created from the 10hz data waves (10-15 columns including a timestamp column). I need to export this data as csv but also split into more manageable 30-min files, starting at the hour or half-hour. The data table can cover multiple days so it is a lot of data. I would like the csv file name to correspond with the date and time for the beginning of the period (eg YYYY-MM-DD hhmm).
Any suggestion would be appreciated
Thanks
Hi,
One approach is to use the extract function to create subsets on the data based on the wave with the time values:
Example:
waves: TimeWave,Data1,Data2,Data3,....
Extract/O timeWave,timeTemp, TimeWave>=StartTime && TimeWave<(startTime+30*60*60)
Repeat for the other data waves. (Note: I added the overwrite flag to allow it to be simply included in a loop)
Then I would do a save function
Save/J timeTemp,Data1Temp,Data2,2temp,... as NewName
To get new name you can use the basic format of secs2date(startTime,-2)+" "+sec2Time(startTime,3)
My tweaks on that would be to add a bit of header text so I know what the file is coming from and to remove the colon from the timestamp portion
NewName ="headerName_"+secs2date(startTime,-2)+" "+replacestring(":",sec2Time(startTime,3),"")
you can then create function to loop through your data for example set the startTime to the minimum value of you time wave and then increment by 30 min (or equivalent amount of seconds) in each iteration through the loop.
Andy
July 8, 2021 at 10:33 am - Permalink
Thanks, I tried your approach and it seems to work. My issue now is to integrate it into a loop that would do this every 18000 rows. I just need guidance on how to set up a loop function or an example that would suits my data. I've search other posts on the forum but haven't been able to find something similar. Again, my programming skills are very limited but I want to learn.
Thanks
July 13, 2021 at 08:17 am - Permalink
In reply to Hi, One approach is to use… by hegedus
Hi,
Could you please share the full function? I am a new starter for Igor and try to write a function to split my data based on date/time.
Regards,
Mortoo
May 23, 2022 at 09:25 am - Permalink