Halving the size of a 2D wave: different speed options

Share icon

I am developing code in which I have to repetitively and quickly extract half a 2D wave's data, i.e. extract the first half of the columns from a DP N=(501,400) 'wave0' into an N=(501,200) 'wave1'. I want to retain the full original wave, so reducing its size is not a good option and I don't want to duplicate a copy first.

The most obvious and concise method is simply to use

wave1=wave0[p][q]

where the 'q' index from the small wave goes from 0-199. The tested execution time on my Intel laptop with IP8 (64bit) is 3372 microsecond/iteration. The next easiest improvement is to try MultiThread

MultiThread w1=wave0[p][q]

This reduced the execution time to 1485 microseconds. Finally, I found that the 'redimension' option for MatrixOP does not truncate the source wave.

MatrixOP/O/S wave1=redimension(wave0,501,200)

This was by far the fastest method, taking 146 microseconds. [All times are subject to many microseconds fluctuation.]

I wanted to bring this to Igor Pro's users attention to reinforce the old point that the most concise code is not always fastest, and to solicit other suggestions.

... and then there is also MatrixOP subRange() which is more flexible but probably slower.

May 8, 2020 at 10:35 am - Permalink

And Duplicate/RMD=[][0,199] takes about 26 microseconds on my Macintosh. Here's my test function:

Function test()

    Make/N=100/O times=0
    Wave m2d
    Variable i
    for (i = 0; i < 100; i++)
        Variable start = StopMSTimer(-2)
        Duplicate/RMD=[][199] m2d, reduced
        Variable theend = StopMSTimer(-2)
        KillWaves reduced
        times[i] = theend - start
    endfor
end

First, 

make/n=(501,400) m2d=p+1000*q

After,

•print sum(times)/100
  26.0882

I'd be curious how this works out for you. I did this using Igor 8.04.

May 8, 2020 at 11:26 am - Permalink

Your only problem here is that your Duplicate copied only 1 column.

May 8, 2020 at 12:18 pm - Permalink

Here's a fixed version of John's function that also uses the right wave type:

Function test()
    Make/O/D/N=(501,400) m2d=p+1000*q
    Make/N=100/O times=0
    Wave m2d
    Variable i
    for (i = 0; i < 100; i++)
        Variable start = StopMSTimer(-2)
        Duplicate/RMD=[][0,199] m2d, reduced
        Variable theend = StopMSTimer(-2)
        KillWaves reduced
        times[i] = theend - start
    endfor
    print sum(times)/100
end

On my Windows machine with IP 8.04, I get:

•test()
  42.057

 

A variation on that that uses the MatrixOP command you came up with is a bit slower:

Function test2()
    Make/O/D/N=(501,400) m2d=p+1000*q
    Make/N=100/O times=0
    Make/O/D/N=(501,200) reduced
    Wave m2d
    Variable i
    for (i = 0; i < 100; i++)
        Variable start = StopMSTimer(-2)
        MatrixOP/O/S reduced=redimension(m2d, 501, 200)
        Variable theend = StopMSTimer(-2)
        KillWaves reduced
        times[i] = theend - start
    endfor
    print sum(times)/100
end

•test2()
  69.734

If I recall correctly, the MatrixOp command will make an allocation that's the size of the output wave instead of writing directly into the existing output wave without allocating memory.

May 8, 2020 at 01:30 pm - Permalink

That's correct; MatrixOP always requires some allocations that depend on the expression.  If the result is an MD wave it may need to allocate the MD array and copy it to the selected output.

May 8, 2020 at 01:36 pm - Permalink

Thanks for all the responses. Adam, I tried on my machine your 'Duplicate/RMD=[][0,199] m2d, reduced' version [with 1000 trials in 'test()' to reduce fluctuations] and got about 60-65 microseconds, a significant improvement over my last result above. That seems the best way to go. I hadn't appreciated the subtleties involving the MatrixOP memory allocation.

May 8, 2020 at 04:22 pm - Permalink

That was kind of boneheaded. I did the right test on my machine, I think. Maybe.

Copying only one row certainly would be fast....

May 8, 2020 at 05:03 pm - Permalink