Changing the format of the data in table
Hi,
I have a 2D wave in which the first column is time but the rest of the columns are data. The problem that I have run into is that the whole 2D wave is showing up in Date/Time format (info panel says data units: dat). I can open the wave and change the format to general but the moment I close the wave and reopen, I find it returned to the Date/Time format.
As a result, I can't perform any operations on the data and am stuck. Is there a way to change the format of this 2D wave to "general" from the command line or by some code?
I should note that this 2D wave was created by concatenating a 1D time wave (that was in Date/time format and double float) with a regular 2D wave (that had data). I have a feeling this somehow turned the whole concatenated wave into Date/Time format. However, I can't figure out how to revert the format to general.
Many Thanks and Regards,
Peeyush
I think I worked it out. Changed the format from the button at the right bottom of the wave listing in the data browser. Erased dat and reduced double float to single float. Now it's alright.
November 5, 2020 at 11:12 pm - Permalink
I don't think this will work. Date/Time waves are marked by the 'dat' data units scale which applies for the whole wave, i.e., all contents will be handled as date/time or not. I am not aware of a way to mark individual data columns differently. So the only solution would be to keep your date/time and your other data in separate waves. Is there some specific reason that you want to have all in one wave?
November 5, 2020 at 11:19 pm - Permalink
Unless you can work with elapsed time, changing Date/Time data to single precision will not preserve time.
November 6, 2020 at 12:45 am - Permalink
I think this is a bad idea. Execute this for background information:
DisplayHelpTopic "Date/Time Waves"
Note that date/time waves must be double-precision and the units must be "dat".
I recommend that you keep your date/time data in a separate 1D wave.
I think by "open the wave" and "close the wave" you mean display the wave in a table and close the table. When you display a date wave (marked by "dat" units) in a new table, Igor applies date/time format. However, if you save the table as a recreation macro and then recreate the table, any formatting you apply before saving the table are re-applied when the table is recreated.
For background information on table recreation macros, execute:
DisplayHelpTopic "Saving a Window as a Recreation Macro"
You can apply different formats to different columns of a 2D wave. Execute this for details:
DisplayHelpTopic "Modifying Column Properties"
But concatenating your date/time wave with your data will cause problems and I don't recommend it.
November 6, 2020 at 05:47 am - Permalink
Thanks a ton, all!
Hi Chozo; Hi Hrodstein,
So, the scene is that I have multiple experiments, each containing 1-day worth of data from a sampling campaign. Now, I am operating a separate base experiment where all data has to be knitted into one full campaign data wave and then analyzed. I have written codes for everything else, but it's the pulling data from each 1-day experiment into the base experiment that has proven to be cumbersome.
So I thought it to be somewhat easier/quicker to concatenate the time wave with data in individual 1-day experiments, and then save a table copy, to merge into the base experiment where all analysis is happening. I could surely save time 1D time wave and 2D data wave separately for each experiment and merge them individually with the base program/experiment but it's just more time-consuming.
What worked is I manually changed the properties of the concatenated waves by erasing "dat" and setting the wave to be single precision. Then it all flowed. However, the time resolution of the 1st column i.e. the date/time column did reduce considerably. Like you said, having the time wave separate would have preserved its resolution.
I don't know whether there is a way to pull a table from one experiment directly into another without first saving a table copy and merging it. However, if there is one, please do let me know as this will make my life at least an order of magnitude easier!
Hi Tony,
Dropping the precision indeed reduced the resolution of the time wave considerably but after graphing and selecting the Date/Time radio button on the x-axis tab, I was able to get the x-axis to correct date (and time with low res) which luckily is ok this time since I just need to show the time series. The data I have has a time resolution of 1 second for multiple days so the reduced resolution isn't too bad.
[QUICK QUESTION]
The time wave is per second resolution, and there was an hour missing in the data. So I displayed the wave in "general" mode, copied the wave (which was a pure number, probably total seconds since 1904) to excel, added 3600 rows and gave an increment of 1 second on each row. Then copied back to IGOR. It checked out fine. I tried doing this directly in IGOR but didn't succeed because wave data units is "dat". I mean succeeded in inserting 3600 rows but couldn't give the increment to patch the Date/time data in those rows. Is there a way to do this?
Many Thanks and Regards!!!!
November 6, 2020 at 08:15 am - Permalink
If your sampling is really once per second, and you are able to insert missing values (with NaN?), then it would possibly be best to use the X scaling to represent the times, and keep the whole thing in a single 1D wave. If there are missing values, you will need to add something to your code to insert points when you concatenate the data sets.
November 6, 2020 at 10:27 am - Permalink
I'm not sure if this helps but...
You can load data into the current experiment from an experiment file using the LoadData operation.
The waves do not need to be displayed in a table as they exist in the experiment file independent of any graph or table.
As John mentions, if you can use X scaling for to represent the date/time, that would be ideal.
DisplayHelpTopic "Waves - The Key Igor Concept"
DisplayHelpTopic "The Waveform Model of Data"
November 6, 2020 at 10:41 am - Permalink
In reply to Thanks a ton, all! Hi… by Peeyush Khare
you will end up with something like 2^8 s resolution. if you switch to elapsed time and store the start time you retain the resolution and can always choose to recreate the full precision time wave.
you can also plot single precision elapsed time with an X-offset to obtain a date/time axis with sub-second resolution.
November 7, 2020 at 02:09 am - Permalink