Standard deviation between layers of a 3DWave

If I understand your goal correctly and if speed is not a crucial issue, it seems you could you use WaveStats/RMD[row][col][] and iterate over all rows and columns, providing you with count, average, standard deviation for each beam, which you would then have to store during the loop.

It also seems like you could reorient the data so that you can use Wavestats/PCST, like so:

There are, perhaps, faster implementations with matrixop, but if you decide you want yet another statistic in the future, that statistic might already be accessible with this approach.

The approach outlined above is indeed very elegant.  It is however not clear to me what the OP meant by:

(a) "Is there a way to obtain it across layers of a 3D wave for each element of each matrix?"

and

(b) "subject each element in each matrix to an expression like SQRT(Variance)?"

It is important to specify which variance the OP needs to calculate.  Since a variance calculation requires the subtraction of a mean, it is important to specify which mean is being calculated (e.g., mean of a column, mean of a layer or of a beam).  Note that in the special case of zero mean you can factor out the (1/(n-1)) from the variance and simply add the contributions from various columns, layers, etc. 

Igor wrote:

The approach outlined above is indeed very elegant. 

Indeed, I don't need speed in this case and I thank for the solution. I just thought there exists an equivalent of "sumbeams" parameter to obtain standard deviation.

Igor wrote:

It is however not clear to me what the OP meant by:

(a) "Is there a way to obtain it across layers of a 3D wave for each element of each matrix?"

I presume it should be across beams. If I understand the idea of a beam correctly, it's a higher dimension, column-like structure going across the wave layers. I need stdev of beams.

Igor wrote:

and

(b) "subject each element in each matrix to an expression like SQRT(Variance)?"

It is important to specify which variance the OP needs to calculate.  Since a variance calculation requires the subtraction of a mean, it is important to specify which mean is being calculated (e.g., mean of a column, mean of a layer or of a beam).  Note that in the special case of zero mean you can factor out the (1/(n-1)) from the variance and simply add the contributions from various columns, layers, etc. 

Same as above. I need standard deviation, or the square root of the variance, which is the same thing, isn't it? 

FWIW, I added to IP9 a new MatrixOP function called varBeams().  The method suggested above may be the best solution in IP8.

Why not in 8beta? I've just recently upgraded to 8...

This is a new feature and IP8 is not getting any new features at this point.

Looks like very short life cycle...Will my IP8 licence compatible with IP9?

@Jakub: We released Igor 8 more than 2 years ago (May 22, 2018). Your IP8 license won't work with IP9, but we often provide free upgrades to users that purchased a new license or upgrade within some period before a new major version is released. As far as I know we have not yet made a decision on whether we will do that for IP9 and if so what the cutoff date will be. The IP9 beta is taking longer to put together than we had expected and the pandemic isn't helping.

Hi, it seems that in MatrixOP functions are randomly decided for some dimensions and not others. There is a varCols varBeams but why not VarRows? Similar to averagecols() only. Currently I need to transpose my matrices depending on the function I need.

There is no randomness in MatrixOP functions. 

When you consider the packing of wave data into memory, it becomes apparent that varCols() is easier to implement and more efficient to execute than a varRows() equivalent. It may be either more efficient to transpose the matrix or add code that performs the equivalent in order to compute the variance of each row.  If built-in functionality is not going to execute significantly faster than executing varCols(w^t) then the incentive to implement the MatrixOp function is not great.

... and it should be obvious that varBeams() can't easily be replaced unless you do something in combination with transposeVol() that, by its nature, can't be combined into a simple, single line expression.

A.G.

AG -- The fundamental base to your comments is to be respected. However, I must also side with @Claudioez in one regard. I have also struggled to appreciate exactly what MatrixOP is doing when trying to work with rows versus columns in a matrix. I tripped also over exactly his question ... Why do we have a host of operations on Cols and Beams but never (or with rarity) on Rows??? ... at numerous times, for example in doing image processing and, more recently, it translating MatLab code to Igor Pro code.

https://www.wavemetrics.com/forum/general/translating-matlab-matrix-mul…

Even you admitted in the above to grief in appreciating the syntax required in using ROW-MAJOR applications (MatLab) versus COLUMN-MAJOR applications (Igor Pro). And, in my review, the Igor Pro manual itself is ... confusing.

To this point, and following up on the underlying concerns from @Claudioez, I suggest that better clarification is needed in the Igor Pro manual. I give an example to consider to add on page III-143 in MatrixOp and Wave Dimensions. 

***
Individual entrees in a multi-dimensional wave in Igor Pro are referenced as though the primary indices are for rows, followed by columns, layers, and chunks (see page II-92 for the Overview on Multidimensional Waves). MatrixOP treats a 1D wave as a column vector *even though the indices for the units in such a 1D wave are labelled as row indices*. Furthermore, a 2D wwave in Igor Pro is accessed by wwave[row][column] but is treated in MatrixOP as having columns first followed by rows next. The approach in MatrixOP should be considered to be in agreement with what you see as the layout when you display a 2D wave in a table. The elements along the row indices for the 2D wave are displayed vertically (downward) as a column, while the elements in the column indices for the 2D wave are displayed horizontally (across) along rows. Finally, when dealing with translations from matrix operations expressed in other applications, MatrixOP in Igor Pro is column-major in its approach (by comparison for example to applications such as MatLab where matrix operations are row-major).
***

Finally, I suggest also that the information below in some form be included somewhere appropriate as a direct comment to the frustration from @Claudioez (and me and perhaps others).

***
Some confusion may arise in using MatrixOP due to the distinction in what operations are provided or otherwise appear to be missing. By specific example, MatrixOP includes a range of operations to extract information from or modify Cols (Columns for MatrixOP but accessed by [row] indices in the 2D wave) but no comparative operations often appear to exist directly to extract information from or modify Rows (Rows for MatrixOP but accessed by [column] indices in the 2D wave).The recommended approach to operate on directly on Rows in a 2D wave using MatrixOP is to transpose the 2D wave and use the companion operation on Cols. By example, when you would wish to do a varRows(wwave) that does not exist in MatrixOP, you should instead do varCols(wwave^t).
***

I hope this helps appreciate that the frustrations are not truly about apparent "randomness" in Igor Pro. Rather, I suggest the frustrations point to a need for improved clarity on the otherwise rather hard to grasp topic that row and column indices for 2D waves are not the same as Cols and Rows in MatrixOP, and that Row operations in MatrixOP are not provided simply because transposing and using Cols operations is effectively as efficient.

Considering that MatrixOP is enhanced from one IP version to the next, I invite anyone who is missing functionality, e.g., varRows() to submit a feature request in a short email to support@wavemetrics.com. 

FWIW: at this point, there are 12 new MatrixOP functions for IP10.

Thanks AG!

For the record, MatrixOP (IP10) has 19 pairs of functions that support both rows and cols variants:

    1.    row(), col()
    2.    normalizerows(), normalizecols()
    3.    sumcols(), sumRows()
    4.    rotateRows(), rotateCols()
    5.    numRows(), numCols()
    6.    rowRepeat(), colRepeat()
    7.    scaleCols(), scaleRows()
    8.    catRows(),    catCols()
    9.    setCol(),    setRow()
    10.    productRows(), productCols()
    11.    productRow(), productCol()
    12.    subWaveC(), subWaveR()
    13.    reverseCol(), reverseRow()
    14.    reverseCols(), reverseRows()
    15.    minCols(), minRows()
    16.    maxCols(), maxRows()
    17.    indexRows(), indexCols()
    18.    addRows(), addCols()
    19.    subtractRows(), subtractCols()

MatrixOP has only 6 functions that support cols only:

    1.    varcols()
    2.    averageCols()
    3.    bitReverseCol()
    4.    removeCol()
    5.    removeCols()
    6.    setColsRange()

The row variants for the first two may be implemented in IP10 but most are trivial to obtain using a simple transpose.