Loading am/pm time columns into a matrix

Share icon

Hello, I am loading a text file containing about 41 columns into a single matrix.
The first few columns are dedicated to time: "day, month, year, hour, minute, second, Am/pm" and the rest contain data: data1, data2, data3...etc

When I load the text file, the column with the am/pm information is left blank. I presume this is because I am only telling the matrix to load numerical data.

Is there a way to load this information (am/pm) into the matrix, or even better, convert the time stamp to a 24hr format..
I copied in my code below, and I attached an example text file

Function Load_PAM_File(filename, refnum) // load single PAM file
String filename
Variable refnum
print "Loading: " + filename
loadwave/v={"/: \t", "", 1,1}/q/a/n=Temp_PAMDataMatrix/m/o/j/k=1/l={0,0,0,0,0} filename // loads PAM data as a matrix

End

Thank you in advance PAMData20160321.txt (5.63 MB)
It usually does not make sense to load data of different kinds into a matrix unless you are thinking of the matrix as a convenient collection of 1D columns. I presume this is the case in your situation.

The use of AM/PM is unfortunate. To handle this requires loading the AM/PM column as a text wave and then using it to adjust the date/time wave.

I wrote a file loader (see below) for your file. It loads the date/time into a 1D wave and the rest of the data into a matrix.

It calls LoadWave three times:
1. Load the date/time data
2. Load the AM/PM column
3. Load the remaining columns as a matrix

Menu "Load Waves"
    "Load PAM File...", LoadPAMFile("", "")
End
 
// AMPMTimeAdjustment(theTime, flag)
// Returns the necessary adjustment in seconds to convert a time from AM/PM representation
// to 24 hour representation.
Function AMPMTimeAdjustment(theTime, flag)
    Variable theTime
    String flag                 // Must be either "a" or "p"
    
    Variable adjustment = 0
    Variable secondsIn12Hours = 43200
    
    if (CmpStr(flag, "a") == 0)
        // AM: If the hour is 12:xx, subtract seconds in 12 hours
        if (theTime >= secondsIn12Hours)
            adjustment = -secondsIn12Hours
        endif
    else
        // PM: Adds 12 hours unless hour is 12:xx.
        if (theTime < secondsIn12Hours)
            adjustment = secondsIn12Hours
        endif
    endif
 
    return adjustment
End
 
Function LoadPAMFile(pathName, fileName)
    String pathName     // Name of an Igor symbolic path or "".
    String fileName         // Name of file or full path to file.
 
    // First get a valid reference to a file.
    if ((strlen(pathName)==0) || (strlen(fileName)==0))
        // Display dialog looking for file.
        Variable refNum
        Open/D/R/F="*.txt"/P=$pathName refNum as fileName
        fileName = S_fileName           // S_fileName is set by Open/D
        if (strlen(fileName) == 0)      // User cancelled?
            return -1
        endif
    endif
 
    String columnInfoStr
    String delimiters
    
    // First load the date/time column
    delimiters = "\t"
    columnInfoStr = "N=DateAndTime,F=8;"            // Load as date/time
    LoadWave /J /Q /L={0,1,0,0,1} /V={delimiters,"",0,0} /A /O /B=columnInfoStr /R={English, 1, 2, 1, 1, "Month/DayofMonth/Year", 40} /P=$pathName fileName
    Wave DateAndTime
    
    // Next load the AM/PM column
    
    delimiters = "\t" + " "                         // Need space so that the AM/PM column will be taken as distinct from the preceding time column
    columnInfoStr = "N='Date',F=6;"                 // Skip first column (date)
    columnInfoStr += "N='Time',F=7;"                // Skip first column (time)
    columnInfoStr += "N=AMPM,F=-2;"             // Load as text
    LoadWave /J /Q /L={0,1,0,2,1} /V={delimiters,"",0,0} /A /O /B=columnInfoStr /P=$pathName fileName
    Wave/T AMPM
    
    // Account for AM/PM affect on time
    DateAndTime += AMPMTimeAdjustment(DateAndTime, AMPM)
    KillWaves/Z AMPM                    // This is no longer needed
    
    // Finally load the rest of the columns as a matrix
    delimiters = "\t"                                   // We will treat everything up to the first tab as text to easily skip it
    columnInfoStr = "N='_skip_',F=-2;"              // Treat as text
    columnInfoStr += "N=PAMDataMatrix,F=0;"     // Load as numeric
    LoadWave /J /Q /M /L={0,1,0,1,0} /V={delimiters,"",0,0} /A /O /B=columnInfoStr /P=$pathName fileName
    Wave PAMDataMatrix
    
    Printf "Loaded PAM data from \"%s\"\r", fileName
    
    Edit DateAndTime, PAMDataMatrix
    ModifyTable format(DateAndTime)=8, width(DateAndTime)=160
 
    return 0                            // Success
End

April 5, 2016 at 03:58 pm - Permalink

Hello,
Thank you for your quick reply, and for the time you took to write the code. However, I am having a bit of trouble to implement the code or at least getting it to work!

I have attached the igor procedure, with your code copied into it.

Thanks very much
PAM_loader.ipf (5.87 KB)

April 6, 2016 at 12:32 am - Permalink

Quote:
However, I am having a bit of trouble to implement the code or at least getting it to work

It's always helpful if you specify exactly what the problem is, for example, what error message you or what incorrect behavior you are getting.

Quote:
I have attached the igor procedure, with your code copied into it.

I can't compile your file because it uses functions (e.g., Browse4Folder) that I don't have.

Anyhow, your PAM_loader.ipf file does not even call the code that I wrote.

Start by copying the code I wrote to a separate procedure file in a new experiment. Compile that procedure file and choose Data->Load Waves->Load PAM File. Select a file and load the data to confirm that it works correctly. Note what output waves are created.

Next open the separate procedure file and carefully read the code so you understand what the inputs are and what the outputs are. If necessary, step through the LoadPAMFile using the debugger to understand it.

Then try to integrate my code into your code. If this fails, create a simplified, self-contained example that I can run and debug and post it with a specific question.

April 6, 2016 at 10:34 am - Permalink

Hello,
Thank you for your reply.
Apologies for not being specific about the errors.
I have managed to get the code to work. I am now trying to implement your code into my loop function so that it would load all files in the folder into a single time wave.

In my loop function, it looks for a folder containing the data. However, when I try to combine the loop function with your code it keeps looking for individual files and does not load all files in the folder.

I also get a wavewrite error "index out of range for wave "dateandtime".
I call the function using "loaddateandtime()"
The procedure file is attached PAM_loaderV2.ipf

I am a bit stuck and would be very grateful for a bit of advice.
Thanks again
PAM_loaderV2.ipf (8.31 KB)

April 7, 2016 at 12:34 pm - Permalink

I reorganized and rewrote your procedure. The result is in the attached file.

I recommend that you read the code and comments. It is most useful to read the higher level functions first which means starting at the bottom of the file.
PAM File Loader 2016-04-07.zip (2.69 KB)

April 7, 2016 at 11:20 pm - Permalink

Hello,
Thank you very much for all this work. The function works perfectly.

April 8, 2016 at 01:23 am - Permalink