
MatrixOp wave averaging and getting it really fast

thomas_braun
I want to average some columns from some waves. These waves have many points.
I'm currently using fWaveAverage but need to get something quicker (IP7).
I've come up with the following test procedures (number of points are realistic, number of waves are larger):
Function DoStuff() variable timer variable numPoints = 2e5 variable numCols = 12 variable numWaves = 3 Make/N=(numPoints, numCols)/O input1, input2, input3 input1[][] = enoise(p*q) input2[][] = enoise(p*q) input3[][] = enoise(p*q) /// #1 timer = startmstimer MatrixOp/O input1Single = col(input1, 1) MatrixOp/O input2Single = col(input2, 5) MatrixOp/O input3Single = col(input3, 10) fWaveAverage("input1Single;input2Single;input3Single;", "", 0, 0, "root:average1", "") Wave average1 print stopmstimer(timer)/1e6 Make/O/N=(numWaves, numPoints) matrix /// #2 timer = startmstimer Multithread matrix[0][] = input1[q][1] Multithread matrix[1][] = input2[q][5] Multithread matrix[2][] = input3[q][10] MatrixOp/O average2 = averageCols(matrix)^t printf "%g, %g\r", stopmstimer(timer)/1e6, EqualWaves(average1, average2, 1) /// #3 timer = startmstimer MatrixOp/FREE data = col(input1, 1) MultiThread matrix[0][] = data[q] MatrixOp/FREE data = col(input2, 5) MultiThread matrix[1][] = data[q] MatrixOp/FREE data = col(input3, 10) MultiThread matrix[2][] = data[q] MatrixOp/O average3 = averageCols(matrix)^t printf "%g, %g\r", stopmstimer(timer)/1e6, EqualWaves(average1, average3, 1) /// #4 timer = startmstimer MatrixOP/FREE matrix = setRow(matrix, 0, col(input1, 1)) MatrixOP/FREE matrix = setRow(matrix, 1, col(input2, 5)) MatrixOP/FREE matrix = setRow(matrix, 2, col(input3, 10)) MatrixOp/O average4 = averageCols(matrix)^t printf "%g, %g\r", stopmstimer(timer)/1e6, EqualWaves(average1, average4, 1) /// #5 timer = startmstimer MatrixOp/O average5 = averageCols(setRow(setRow(setRow(matrix, 1, col(input2, 5)), 0, col(input1, 1)), 2, col(input3, 10)))^t printf "%g, %g\r", stopmstimer(timer)/1e6, EqualWaves(average1, average5, 1) End
with the results
•dostuff()
0.023102
0.00567959, 1
0.0080306, 1
0.00736661, 1
0.0038697, 1
So 5 looks best, but the problem is that neither the column from the input waves nor the number of input waves are known beforehand.
Is there some way of construction the MatrixOP line from example 5 at runtime? Using something like Execute "MatrixOP" is a bit messy for my taste.
Thanks a lot for reading that far ;)
Thomas
I don't like version (5) because it is difficult to read. I am also confused as to why you chose to use setRow() with ^t when setCol() is an available option and is more efficient for large number of points.
I also don't know that it is more efficient to combine the 3 columns as opposed to executing something like:
A.G.
May 13, 2016 at 01:39 pm - Permalink
I'm not fond of that either, very much write only code.
Well in the end I want to have a 1D wave with the averages and averageCols returns a 1xm wave, so I thought transposing it is the right thing to do.
Also if I use setCol I have to transpose matrix again as I want to average not each input wave but the same points from all input waves.
I've added a two more tests:
and these now run with
•dostuff() 1: 0.0247083, 1 2: 0.00551995, 1 3: 0.00792857, 1 4: 0.00731161, 1 5: 0.00384668, 1 6: 0.00865464, 1 7: 0.00467676, 1
May 16, 2016 at 12:07 pm - Permalink
It was a little difficult to understand what you wanted; I hope I got it right.
Generally I noticed that FastOP is faster than MatrixOP for some of the operations. I wanted to MultiThread the heaviest calculation, but apparently that has almost no effect on MatrixOP calculations. I also noticed that it was faster to do one operation at a time with FastOP and MatrixOP instead of multiple operations, e.g. it's faster to do a=a+b and then a=a+c, than it is to do them in one step a=a+b+c.
If you want to use EqualWaves, I suggest doing it outside the timed sequence, since EqualWaves took a not insignificant amount of time to calculate when I tested it.
May 17, 2016 at 04:49 am - Permalink
FWIW, I just added one further optimization in MatrixOP for the special case of transposing 1D arrays.
A.G.
May 17, 2016 at 11:03 am - Permalink
EqualWaves
, I *think* it is executed afterstopmstimer
.@AG: Thanks, I'll give it a try.
May 20, 2016 at 03:55 pm - Permalink