Averaging part of a 2D wave by index stored in another 1D wave

I think I understand. I would write a function to do this as follows:

In a loop go through puffind and for each value duplicate the 2D data twice, using conditional statements that copy the data if it exceeds the value (or whatever it is you need) and set the rest as 0, second version will set the values to logical (1 and 0). Use MatrixOP to Sum by column for both and divide the first (sum of columns) by the second (n) to get the average.

There are probably more elegant solutions, but I thought I would chip in.

Hi,

Another option is to use the extract function. 

The Extract operation finds data in srcWave  wherever LogicalExpression  evaluates to TRUE and stores any matching data sequentially in destWave, which will be created if it does not already exist.
Parameters
srcWave  is the name of a wave.
destWave  is the name of a new or existing wave that will contain the result.
LogicalExpression  can use any comparison or logical operator in the expression.
Flags
/FREE    Creates a free destWave (see Free Waves).
    /FREE is allowed only in functions and only if a simple name or wave reference structure field is specified for destWave.
/INDX    Stores the index in destWave  instead of data from srcWave.
/O    Allows destWave to be the same as srcWave  (overwrite source).

Andy
 

sjr51 wrote:

I think I understand. I would write a function to do this as follows:

In a loop go through puffind and for each value duplicate the 2D data twice, using conditional statements that copy the data if it exceeds the value (or whatever it is you need) and set the rest as 0, second version will set the values to logical (1 and 0). Use MatrixOP to Sum by column for both and divide the first (sum of columns) by the second (n) to get the average.

There are probably more elegant solutions, but I thought I would chip in.

The problem here is that I need to average each bundle of data (when the logical==1) by the numbers of elements from first 1 to the last 1 of that bundle and then proceed to the next bundle of true values and do the averaging again. I cannot have just one average from the selected data.

I have also tried the extract but it cannot extract each bundle to a separate (or temporary wave) every time the group of true values is found in logical...Also, you cannot index source, dest or logical during the process of extraction.

Any other solution? 

 

OK it seems I didn't understand what you're trying to do. How about this: logical is 683 rows, counts_raw is 683 rows by 50 cols, so you want to average the rows where logical == 1?

Hi,

I am also not completely clear on the objective, but I will give it a trial.

I took your counts_raw wave (632x50), logical (632 pnts), and your puffind (221 points with data that goes from 0-632).

My approach

Transpose your counts_raw to a new wave of 50x632 pmts.

Wavestats/pcst to get stats for each of the 632 columns.

Then duplicate the puffind wave to a new wave, results

results[] =M_wavestats[%avg][puffind[p]]/logical[puffind[p]]

This gives the avg for each point in the puffind wave and a NaN where the logical wave value is 0.

to remove the points with Nans

wavetransform zapnans result

Am I close to what you need?

Andy

It seems hard to avoid any explicit loop if you want to average blocks of data in counts_raw, where logical == 1.

Here is my take on my interpretation of the task (you may want to remove the /FREE flags to see what happens in detail):

Hope this is what you want.

ChrLie wrote:

It seems hard to avoid any explicit loop if you want to average blocks of data in counts_raw, where logical == 1.

Here is my take on my interpretation of the task (you may want to remove the /FREE flags to see what happens in detail):

function foo(data, logic)
    wave data, logic
   
    // make a range wave that indicates the start/stop indices over which counts_raw is averaged
    Duplicate/FREE/O logic w, range
    Redimension/N=(numpnts(logic) +1) w
    // find the beginning and end of blocks of 1's
    Range = abs(w[p] - w[p+1])
    Range = Range[p] == 1 ? p : NaN
    WaveTransform zapNaNs Range
    // note that start "marker" index needs to be raised by 1
    // even row number = start, uneven = stop
    Range[0, ;2] += 1
   
    // Make result wave, how many blocks are there?
    variable n = numpnts(Range)
    Make/D/O/N=(n/2, Dimsize(data, 1)) M_Result
   
    // loop through Range and average raw data over blocks of rows
    variable i, idx = 0
    for(i=0; i<n; i+=2)
        Duplicate/FREE/R=[range[i], range[i+1]][] data, block
        MatrixOP/FREE avg = averageCols(Block)
        M_Result[idx][] = avg[0][q]
        idx +=1
    endfor
end

Hope this is what you want.

Than you very much! That works very well. I have learned about selective indexing from your code :) Much appreciated! 

Ps. Thank you for all the other answers , you've been a great help too!