matrix data sorting with many columns

Share icon

Greetings,

I am wanting to sort a 2D wave with 900+ columns. I want to sort the matrix wave such that the rows are rearranged with all values of column 1, for example, in ascending order (similar to an Excel data sort). I've been doing some Google searching and reviewing priors posts on this forum. I can work with either a single 2D wave or 900+ 1D waves. I've got the 'matrix' data available in either format. The wave lengths are about 25e3 rows, just for reference.

I have reviewed this topic: Sorting two waves simultaneously, a la Excel

In my case, the example method from Howard would require listing the 900+ explicit 1D waves in the Sort command (or so it seems to me). I've messed around a bit with trying to sort the 2D wave directly, but thus far no luck.

Thanks. Josh
I searched in the code snippets information and found the function MDtextsort(). I converted the sort process to a numeric sort (per the instructions), and everything worked great. I just made a new function call MDsort(). My 2D wave was sorted by a particular column. Yahoo!

#pragma rtGlobals=1 //Use modern global access method

Function MDsort(w,keycol)
    Wave w
    variable keycol
 
    variable ii
 
 
    make/o/n=(dimsize(w,0)) key
    make/o/n=(dimsize(w,0)) valindex
 
    key[] = w[p][keycol]
    valindex=p
 
    sort key,key,valindex
 
    duplicate/o w, M_newtoInsert
 
    for(ii=0;ii<dimsize(w,0);ii+=1)
        M_newtoInsert[ii][] = w[valindex[ii]][q]
    endfor
 
    duplicate/o M_newtoInsert,w
    killwaves/z key,valindex,M_newtoInsert
End


Joshua Lorenz
Kato Engineering

July 10, 2013 at 09:47 am - Permalink

It sounds as if you need to use SortColumns which is not available in IP6.

One way of accomplishing what you want is to split your data into columns and then apply the Sort operation with a list of wave names corresponding to the columns. The difficulty here is that with 900 columns you will quickly exceed the maximum length of the command line. The solution is then to run multiple Sort operations, say in groups of 50 columns or so, always providing a copy of the unsorted column 1 as the key wave.

A.G.
WaveMetrics, Inc.

July 10, 2013 at 04:39 pm - Permalink

There's also the operation, MakeIndex, which is meant to be paired with IndexSort, except that neither are MultiDimensional aware.

MakeIndex key, valindx

does the same thing as MDSort's
make/o/n=(dimsize(w,0)) valindex
valindex=p
sort key,key,valindex

, and
IndexSort valindex, wavetobesorted

is equivalent to
wavetobesorted = copyofwavetobesorted[valindex[p]]

except I think it does the sorting in place (Igor doesn't need to make a complete copy of the unsorted wave the way the second form does). It would be nice to have MD aware IndexSort precisely for large MD waves, but copy and assign works. Finally, I don't see why MDSort needs the for loop:
M_newtoInsert = w[valindex[p]][q]

should do the same thing as explicitly stepping through rows with the for loop and the ii variable.

July 10, 2013 at 01:52 pm - Permalink