Elegantly extracting data from 2D wave and using same button to open/close a panel

Share icon

Hi, 

I have three questions.. tried to hunt down the answers in help topics and reference pdfs but couldn't succeed. I have a 2D data wave that contains 35000 rows and 300 columns. I am tasked with two things - 1) delete columns with 90% cells containing zeros and 2) extract a 2D wave out of this data such as rows 5000 to 6000 with all columns. 

I am able to do both these tasks but my code involves for looping etc. which takes time to execute and I really want to make the algorithm more elegant and easy on the processor. Please advise how I could do this and if there are any IGOR library functions that could do the heavy lifting here. 

Here is how I'm doing this right now: 

1) Running a for loop in reverse that puts individual columns into a temp wave. Then I set all zeros in temp to nan and call V_numNaNs/V_npnts for the 90% check. If above 90%, I say Deletepoints/M=1 i, 1, 2Dwave. But all this takes time since the for loop has to run through each column loading 35000 rows into the temp wave each time. Can this be done more elegantly and in a faster way? 

2) For this I am using Duplicate/RMD=[5000,6000][] 2Dwave, 2Dsubwave. Interestingly in the wave information section in data browser, the 2Dsubwave pulled with this syntax still shows the "Startrow:" value as 5000 when it should be 0 i.e. the index point of the first row of the 2Dsubwave. Not sure how to fix this but this causes problems with image plot axis labels for the 2Dsubwave. Please advise whether there is a better way such as using MatrixOp somehow to extract 2D data from a 2D wave. 

3) Last thing is regarding panels. I need to be able to open and close a panel by clicking the same button. I just couldn't figure out how to do this. I tried locating any detail in the reference pdfs but didn't succeed. Please advise, and also on how to lock the code. I'm building a software in IGOR wherein I may need to lock some parts of the code so that they couldn't be accessed by the ctrl+T functionality. 

Sorry for several questions.. figured it was better to ask them all together..

Sincerely, 

Peeyush 

1) 

•matrixop/O counts = sumcols(equal(junk, 0))
•redimension/D counts
•counts /= dimsize(junk,0)
•extract/INDX counts, countsi, counts[p][q]>0.9

The original matrix wave is "junk", for real code you need a wave reference. At the end, countsi contains a list of columns with more than 90% zeroes.

2) Duplicate/RMD copies wave scaling. Does your original matrix have an x0 of 5000 for the X scaling?

3) Check for existence of the panel window. If it exists, kill it. If it doesn't exist, make it.

March 31, 2022 at 03:50 pm - Permalink

Thanks a lot, johnweeks! 

Regarding your question in 2, I'm not sure how to check scaling. Basically, when I duplicate/RMD, the new wave that's created shows the start point right at the row from where I duplicated in the mother wave. Kindly see the image below as an example. I had a mother wave matrix_quant from which I duplicated a small part as matrix_quant_PCB. Now the wave information below says the Start is 75 but I want it to read 0 otherwise my tick labels in the image plot using matrix_quant_PCB do not work. 

Kindly advise. 

 

April 1, 2022 at 03:52 am - Permalink

You need to adjust the wave scaling for this. To do it manually, just press the small pencil icon and dial in 0. Or use SetScale in code.

April 1, 2022 at 05:18 am - Permalink

And show us the same info for the original wave matrix_quant. I'm betting it is the same, and Duplicate has simply copied it.

April 1, 2022 at 09:12 am - Permalink

Thanks a lot chozo. I'll give SetScale a shot.. trying to automate this so need to go the coding route. 

@johnweeks, below is the same info for the original matrix_quant wave which is the mother wave for the data in matrix_quant_PCB. Thoughts? 

April 1, 2022 at 10:09 am - Permalink

>sheepish grin<

Seems Igor is being helpful. You might be able to see that a lot of the time, if your wave scaling is set to something meaningful, like the time of acquisition, or latitude and longitude, it is very useful to have the duplicated chunk keep its scaling.

So back to chozo-

SetScale/P x 0,1,matrix_quant_PCB

 

April 1, 2022 at 04:04 pm - Permalink

I realize that I am somewhat late to this party but let me suggest a MatrixOP based solution.  I will go through the steps so that you can see how I arrive at the final function.  The following code assumes that you have no NaNs in your input "junk" matrix.

Variable rows=dimsize(junk,0)
Variable thresh=0.9*rows

The actual count is:

MatrixOP/O/FREE aa=sumcols(equal(junk,0))

To combine this with the threshold:

MatrixOP/O aa=greater(sumcols(equal(junk,0)),thresh)

Set cols that are above the threshold to NaN:

MatrixOP/O bb=setNaNs(aa,aa)

Scale the original matrix with this row vector:

MatrixOP/O cc=scaleCols(junk,bb+1)

Remove all the NaNs:

MatrixOP/O dd=zapNaNs(cc)

Reshape the 1D wave back to a 2D matrix:
 

Redimension/N=(rows,numpnts(dd)/rows) dd

Now combine all the above into compound MatrixOP expressions:

Function processJunk(wave junk)

    Variable rows=dimsize(junk,0)
    Variable thresh=0.9*rows
    MatrixOP/O/FREE aa=greater(sumcols(equal(junk,0)),thresh)
    MatrixOP/O/FREE cc=zapNaNs(scaleCols(junk,setNaNs(aa,aa)+1))
    Redimension/N=(rows,numpnts(cc)/rows) cc
    Duplicate/O/RMD=[5000,6000][] cc,outWave
End

 

April 4, 2022 at 09:52 am - Permalink

Thanks a lot, @johnweeks for confirming! 

Thank you very much Igor for the detailed response. The syntax are also very helpful because yesterday I was stuck with using the "within" logical operator with MatrixOp since it was throwing error on my syntax which I couldn't parse. From your response, I see that I can replace greater with within and it should work fine.. 

Sincerely, 

Peeyush

April 5, 2022 at 03:54 am - Permalink