Extracting the date and time from the first lines of a .txt file

Hi,

Here is something to start with, it opens the file for reading and looks for the line that contains "start" and it parses that line to get the year, month, day, hour, minute, and seconds as integer variables. It then gets the seconds count for that day and adds on the time converted to seconds.  In the end it returns a single variable with the seconds at the start time.  If you are going to automate you will probably want to pass some file sting reference.

Andy

Hi Andy, 

Thanks very much.

This reads out the date and time very well. But how can I integrate this into my matrix that I am loading?  I am currently adapting some code to work with this new data set? As I have several (hundred) different text files of different format to load. 

On a side note: I am getting an error when using this code " Can't convert a text wave to or from another type." I don't understand yet as I can load it without any trouble using the load/general text option in the tool bar.

Hi,

If the file you are loading is the same as the example txt files you provided, then I think the L flag needs to be modified.  Currently it is starting at the first line which is part of the header and it is determined to be text. The data appears to start on line 20 (zero based counting).  I used 

/L={0,20,0,0,0}

To successfully load the example file.  Note if the header rows vary from file to file you will need to determine where the data starts.

Andy

Hmmm.... When downloaded on my Macintosh, the line endings get stripped out...

On Windows I get very long lines; the line with the column labels on it includes what looks like data:

t_rel	cn_bin	n	cn_nb	n_nb	run_modus	flow	Pch	Ti1	Ti2	Ti3	Ti4	Ti5	0.1	0.10975	0.12045	0.13219	0.14508	0.15923	0.17475 (and so forth for a long time)

Do others get something else?

Hi John, 

Yes there is a lot of columns but I dont need them (at least for the moment). I am interested in the data in the columns CN_NB and n_NB and the t_rel (creating a date time wave). 

Thanks

So regarding the error above, answered by ANDY. I used a different piece of code snippet and it worked. I will check why I was getting this error. 

However, I would still like to read the start time and calculate a time wave from the seconds since start. The read_return function works well but I am unsure how/where I should integrate this into the code..

Hi,

I will assume that one of your columns is the relative time stamp, t_rel.  The constraint within Igor is that the data modeling type must be the same for all columns in a matrix.  This means if you want to create a column with the full date/time stamp and have Igor use it as such all the other columns will be considered date/time data types as well.  Probably not what you want.  This behavior is similar to arrays in numpy in Python. Pandas Dataframes in Python allow for different data types for each columns.  I have used labels frequently to fake a text column within a numeric wave, but that is another story.

One suggestion assuming that the t_rel column contains the number of seconds from start time. I would create a new wave, realTime, with the same length of rows as your matrix and set its values

I would the use setscale to set to date time with

When plotting use the realTime wave as the x.

Andy

Hello Andy, 

The read_return() function just reads out the date from the file being loaded. However I have a number of files  to load, each having a different start time. 

Would it be possible to load them into a wave and do what you suggested above as a separate function?

Thanks

Hi,

There are a variety of ways to handle this depending on your work flow and final objective.  The function returns a number which is the seconds from 1/1/1904. You can store in a variety of places depending on what happens next. Options include:

1. After a data file is loaded a separate wave with a similar file name perhaps with "_time" suffix is created that contains the full date/time data.

2. A master wave that holds the starting time that is then used latter in a function to dynamically create the time wave.

3. You could store in a wave note as a key value pair that is later read.

It really depends on your ultimate work flow.  For example I often have to process multiple data files to extract calculated results such as peak position, slope, .....  In those cases I create a workflow that does all the loading and calculations on a single data set.  I then append the net results to a separate wave with the sample name as the row label.  When I run through I overwrite the individual data files since I am interested only in the calculated results.

If however you are interested in comparing run to run on a graph then I might load the data with sample Id as matrix name and a matching time wave with a name linked to the data matrix wave.

Andy

Hi Andy, 

Thank you for your response. I am more interested in the later option, to compare the data (particle concentrations) as it varies with time, but also to have the posssibilty to compare the data with other parameters. For this reason I need a time stamp that can be used for eventually averaging concentrations. 

Thanks