Add WaveMinMax
thomas_braun
In many of my code bases I call both WaveMin and WaveMax to calculate the minimum and maximum of a wave. As that is usually faster than calling WaveStats.
But from an algorithmic point of view if you want to calculate both you would only have to traverse the wave elements once. Also C++ learned that a while ago, see std::minmax.
So my request is to add a function WaveMinMax which uses multiple return value syntax to return both the minimum and the maximum of a wave.
You know that WaveStats/M=0 does essentially that, yes? /M=0 skips the slower calculations.
August 18, 2018 at 12:11 pm - Permalink
Yes I do know that. But my feeling was that this is still slower.
And some testing with
SetRandomSeed 1
variable maxExp = 24
variable numRuns = 10
variable i, j, elapsedTime
variable refTime, ret, wmin, wmax
Make/D/O/N=(maxExp, 3) result = NaN
SetDimLabel 1, 0, stats, result
SetDimLabel 1, 1, extrema, result
SetDimLabel 1, 2, mtrxop, result
SetScale/I x, 0, 2^maxExp*4, "byte", result
SetScale d, 0, 0, "s", result
for(i = 1; i < maxExp; i += 1)
Make/FREE/N=(2^i)/R data = trunc(enoise(1e6))
refTime = stopmstimer(-2)
for(j = 0; j < numRuns; j += 1)
WaveStats/Q/M=0 data
endfor
elapsedTime = stopmstimer(-2) - refTime
result[i][%stats] = elapsedTime/1e6
refTime = stopmstimer(-2)
for(j = 0; j < numRuns; j += 1)
wmin = WaveMin(data)
wmax = WaveMax(data)
endfor
elapsedTime = stopmstimer(-2) - refTime
result[i][%extrema] = elapsedTime/1e6
refTime = stopmstimer(-2)
for(j = 0; j < numRuns; j += 1)
matrixop/O mmin = minVal(data)
matrixop/O mmax = maxVal(data)
endfor
elapsedTime = stopmstimer(-2) - refTime
result[i][%mtrxop] = elapsedTime/1e6
AbortOnValue !(V_min == wmin), 125
AbortOnValue !(V_max == wmax), 125
AbortOnValue !(V_min == mmin[0]), 125
AbortOnValue !(V_max == mmax[0]), 125
endfor
Display/K=1
AppendToGraph result[][0]/TN=stats
AppendToGraph result[][1]/TN=extrema
AppendToGraph result[][2]/TN=matrixOP
ModifyGraph mode=4
ModifyGraph log=1
ModifyGraph rgb(extrema)=(1,39321,19939)
ModifyGraph rgb(matrixOP)=(0,0,65535)
Legend
End
gives the attached graph.
So WaveMin/WaveMax is the fastest.
August 21, 2018 at 05:26 am - Permalink
That's interesting. Your bottom axis label says, "Number of Elements" but the tick label says, "10 MByte". Which is it?
What sort of hardware and OS are you using for this test?
August 21, 2018 at 09:30 am - Permalink
I just looked at our source code, and the underlying code actually gets the min and max together, then WaveMin and WaveMax return the one that is asked for. So I started implementing WaveMinMax for Igor 9 and immediately ran into a problem. How do I return the result? The result has two values... I suppose we could return a complex number, but that's an ugly hack.
August 21, 2018 at 09:38 am - Permalink
I've adapted the code and graph. The x units are bytes.
I'm on Windows 7 with an i7-3930K@3.2GHz and 64GB RAM.
So mutiple return syntax is not available for builtin functions?
August 21, 2018 at 09:49 am - Permalink
[Note: I wrote this response before John's response but forgot to post it]
First of all, it's technically WaveStats /M=1, not /M=0, though it appears that /M=0 gives the same results as /M=1. But that's not how the flag is documented.
Also, the documentation for WaveMin says "For a floating-point wave, WaveMin runs about three times faster than getting the same information using WaveStats. For an integer wave, WaveMin runs about ten times faster than WaveStats. The advantage may not hold for short waves.". A similar statement is present for WaveMax. I'm not sure when that was written, but it's not completely accurate at least for large waves. I believe that's because WaveStats is automatically multithreaded and for large waves that improves its performance substantially. WaveMin and WaveMax are not automatically multithreaded.
I don't believe that adding a built-in function that uses the new multi-return syntax is technically possible, and I suspect that it would require a lot of work to make it possible. As far as I understand, built-in functions can't take parameters by reference, and that's how user-defined multi-return functions are implemented under the hood.
August 21, 2018 at 10:59 am - Permalink
By the way, you're not handling the X axis correctly. You need to do a y vs. x plot because the increment between your x values is not actually uniform. Here's corrected code that I used:
SetRandomSeed 1
variable maxExp = 24
variable numRuns = 10
variable i, j, elapsedTime
variable refTime, ret, wmin, wmax, wsum
Make/D/O/N=(maxExp, 3) result = NaN
Make/D/O/N=(maxExp-1) numElements
SetDimLabel 1, 0, stats, result
SetDimLabel 1, 1, extrema, result
SetDimLabel 1, 2, mtrxop, result
SetScale d, 0, 0, "s", result
for(i = 1; i < maxExp; i += 1)
numElements[i-1] = 2^i
Make/FREE/N=(2^i)/R data = trunc(enoise(1e6))
refTime = stopmstimer(-2)
for(j = 0; j < numRuns; j += 1)
WaveStats/Q/M=1 data
endfor
elapsedTime = stopmstimer(-2) - refTime
result[i][%stats] = elapsedTime/1e6
refTime = stopmstimer(-2)
for(j = 0; j < numRuns; j += 1)
wmin = WaveMin(data)
wmax = WaveMax(data)
wsum = sum(data)
endfor
elapsedTime = stopmstimer(-2) - refTime
result[i][%extrema] = elapsedTime/1e6
refTime = stopmstimer(-2)
for(j = 0; j < numRuns; j += 1)
matrixop/O mmin = minVal(data)
matrixop/O mmax = maxVal(data)
endfor
elapsedTime = stopmstimer(-2) - refTime
result[i][%mtrxop] = elapsedTime/1e6
AbortOnValue !(V_min == wmin), 125
AbortOnValue !(V_max == wmax), 125
AbortOnValue !(V_min == mmin[0]), 125
AbortOnValue !(V_max == mmax[0]), 125
endfor
Display/K=1
AppendToGraph result[][0]/TN=stats vs numElements
AppendToGraph result[][1]/TN=extrema vs numElements
AppendToGraph result[][2]/TN=matrixOP vs numElements
ModifyGraph mode=4
ModifyGraph log=1
ModifyGraph rgb(extrema)=(1,39321,19939)
ModifyGraph rgb(matrixOP)=(0,0,65535)
ModifyGraph tickUnit(bottom)=1
Label bottom, "Number of elements"
Legend
End
Running the above gives Graph9.png.
The strange shape of the red line is due to the fact that WaveStats is automatically multithreaded. If I first execute
MultiThreadingControl setMode=0
and the I rerun the test, I get results in Graph10.png.
August 21, 2018 at 11:39 am - Permalink
Adam: You are right, I did not handle the x axis correctly.
So If I really want to speed that up I would need to use a paralell implementation for minmax. This starts to get serious work.
For now can we get the sentence
from WaveMin/WaveMax removed as that is cleary not the case?
Thanks Jim, John and Adam.
August 22, 2018 at 01:37 pm - Permalink
Are you actually experiencing performance problems? I was under the impression that you were looking for cleaner code and not so much performance improvements. Yes, it would of course be faster if we had a WaveMinMax that returned both the minimum and maximum, but using WaveMin followed by WaveMax (for short waves) is still quite fast.
August 22, 2018 at 02:52 pm - Permalink
The directive is to get it as fast as possible with reasonable effort. Having cleaner code would be a nice side effect as well.
August 23, 2018 at 05:12 am - Permalink