Variable name dereferencing?

Share icon

I often find myself needing to write a function that generates many parameters, which will eventually be stored in an output wave. Copying and pasting all the variable names into the output wave sometimes gives me a headache and the feeling of onset of carpal tunnel syndrome. I would be interested to hear how other users handle such situations. It's possible (ok, likely) that I am simply an inefficient organizer of data. I also see a possible solution to the issue, but I haven't found a way to implement in Igor.

My solution would be to get a list of the local variable names that I want to store in the output wave, then loop through the names, then dereference the variable names to assign the local variable values:
//...at the end of some long function that has generated a bunch of local variables that need to be stored
String localVarList=variablelist("*",";",-1)    //let's imagine -1 variableTypeCode selects local variables
Make/o/d/n=(Itemsinlist(localVarList)) output
output[]=$stringfromlist(p,localVarList)        //does not compile, as expected


Another option that comes to mind is to make the output wave at the start of the function and then use dimension labels to give each index a meaningful name. I have always hesitated to do this because I have assumed that [%labelString] indexing involves string searches/comparisons to find the label's index, likely adding a lot of run time over many calls to the function.

Here's a function I made recently where I wrote out all the assignments (or at least I hope I did). I've also included my alternative (wishful thinking) syntax. I hope this example communicates the work that I am trying (or, rather, desperate) to avoid. Thanks for any input!

function exFunc(dur,wv1,cmdCopWv,bEffWv,diodeWv,wv1IntByTimeX,wv1IntEndForSlope,wv1LastIntTimeX,effCopyOnOffThreshold,cmdStartP,outWv,durNum,repNum,doLbls)
    Variable dur,durNum,repNum
    WAVE/D wv1,cmdCopWv,bEffWv,diodeWv
    Variable wv1IntByTimeX,wv1IntEndForSlope,wv1LastIntTimeX,effCopyOnOffThreshold,cmdStartP        //start and end for line slope fit and baseline for diode
    wave outWv
    Variable doLbls
   
    Variable avgWins=0.001
   
    //get a bunch of parameters
    make/o/d/free/n=2 coefs
    CurveFit/M=2/W=0 line, kwCWave=coefs, wv1[x2pnt(wv1,wv1IntByTimex),x2pnt(wv1,wv1IntEndForSlope)]/D
    Double wv1lineOffset=coefs[0]   //need to store this...
    Double wv1lineSlope=coefs[1]//and this...
    Variable wv1AvgStartX=wv1LastIntTimeX-avgWins
   
    duplicate/o/r=(wv1AvgStartX,wv1LastIntTimeX) wv1,wv1Temp
    wv1Temp-=(wv1lineOffset+wv1lineSlope*X)
    Double wv1Avg=mean(wv1Temp,wv1AvgStartX,wv1LastIntTimeX)        //and this...
   
    Variable cmdTimingOK = cmdCopWv[cmdStartP] > 3
    Variable wv1AvgStartP=x2pnt(wv1, wv1AvgStartX)      //and this...
    Variable wv1AvgEndP=x2pnt(wv1, wv1LastIntTimeX)     //and this...
   
    Double effSum=sum(bEffWv,wv1AvgStartX,wv1LastIntTimeX)      //and so on...
    Double effArea=area(bEffWv,wv1AvgStartX,wv1LastIntTimeX)
    Variable effSumRange=wv1LastIntTimeX-wv1AvgStartX
    Variable effSumPnts=wv1AvgEndP-wv1AvgStartP
   
    make/o/d/n=0 risingLevels,fallingLevels
    Variable cmdStartX=pnt2x(bEffWv,cmdStartP)
    FindLevels/D=risingLevels/EDGE=1/Q/R=(cmdStartX,wv1LastIntTimeX) bEffWv, effCopyOnOffThreshold
    Variable effNoRising= V_flag==2
    FindLevels/D=fallingLevels/EDGE=2/Q/R=(cmdStartX,wv1LastIntTimeX) bEffWv, effCopyOnOffThreshold
    Variable effNoFalling = V_flag==2
   
    Variable effNumRiseLevels=dimsize(risingLevels,0)
    Variable effNumFallLEvels=dimsize(fallingLevels,0)
    Variable effnFallnRiseDiff=effNumFallLEvels-effNumRiseLevels
    Variable effnFallnRiseMismatch= effnFallnRiseDiff!=0
   
    Variable effStartX=nan,effEndX=nan
    if (!effNoRising)
        effStartX=risingLevels[0]
    endif
    if (!effNoFalling)
        effEndX=fallingLEvels[effNumFallLEvels-1]
    endif
   
    Variable diodeBaselineEndX=wv1IntEndForSlope
    VAriable diodeBaselineStartX=diodeBaselineEndX-avgWins
    Double diodeBaseline=mean(diodeWv,diodeBaselineEndX,diodeBaselineStartX)
    Variable diodeStartX=diodeBaselineEndX
    Variable diodeEndX=wv1AvgStartX
    Variable diodeStartP=x2pnt(diodeWv,diodeStartX)
    Variable diodeEndP=x2pnt(diodeWv,diodeEndX)
    duplicate/o/r=(diodeStartX,wv1AvgStartX) diodeWv,diodeTemp
    diodeTemp-=diodeBaseline
    Double diodeSum=sum(diodeTemp)
    Double diodeArea=area(diodeTemp)
    Variable diodeSumRange=diodeEndX-diodeStartX
    Variable diodeSumPnts=diodeEndP-diodeStartP
   
    wavestats/q diodeTemp
    Double diodePeakLocX=V_minloc
    Double diodePeakValX=V_min
    Double diodeHalfMaxVal = diodePeakValX * 0.5
    FindLevel/EDGE=2/Q diodeTemp, diodeHalfMaxVal      
    Variable diodeFindRisingFail=V_flag
    Double diodeRisingX=V_levelx
    FindLevel/EDGE=1/Q diodeTemp, diodeHalfMaxVal  
    Variable diodeFindFallingFail=V_flag
    Double diodeFallingX=V_levelx  
    Double diodeFWHM=diodeFallingX-diodeRisingX
    Variable diodeUnexpectedORder=diodeFWHM < 0
   
    //store the bunch of parameters...
   
    if (doLbls)
        string lbls="wv1lineOffset;wv1lineSlope;wv1AvgStartX;wv1Avg;cmdTimingOK;wv1AvgStartP;wv1AvgEndP;effSum;effArea;effSumRange;effSumPnts;cmdStartX;effNoRising;effNoFalling;"
        lbls+="effNumRiseLevels;effNumFallLEvels;effnFallnRiseDiff;effnFallnRiseMismatch;effStartX;effEndX;diodeBaselineEndX;diodeBaselineStartX;diodeBaseline;"
        lbls+="iodeStartX;diodeEndX;diodeStartP;diodeEndP;diodeSum;diodeArea;diodeSumRange;diodeSumPnts;diodePeakLocX;diodePeakValX;diodeHalfMaxVal;"
        lbls+="diodeFindRisingFail;diodeRisingX;diodeFindFallingFail;diodeFallingX;diodeFWHM;diodeUnexpectedORder;"
       
        dl_assignLblsFromList(outwv,0,0,lbls,"",0)      //loops through the list of labels and assigns that to the dimension of interest (rows)
    endif
   
    //DESIRED SYNTAX (will not compile, as expected)
    outwv[][repNum][durNum]=$stringfromlist(p,lbls)
   
    //even better desired syntax (would not require me to have to keep an up-to-date list of all the variables I might want to store
    String localVarList=variablelist("*",";",-1)    //let's imagine -1 variableTypeCode selects local variables
    outwv[][repNum][durNum]=$stringfromlist(p,localVarList)
   
    //current syntax
    variable i
    i=0;outwv[i][repNum][durNum]=wv1lineOffset
    i+=1;outwv[i][repNum][durNum] = wv1lineSlope
    i+=1;outwv[i][repNum][durNum] = wv1AvgStartX
    i+=1;outwv[i][repNum][durNum] = wv1Avg
    i+=1;outwv[i][repNum][durNum] = cmdTimingOK

    i+=1;outwv[i][repNum][durNum] = wv1AvgStartP
    i+=1;outwv[i][repNum][durNum] = wv1AvgEndP
    i+=1;outwv[i][repNum][durNum] = effSum
    i+=1;outwv[i][repNum][durNum] = effArea
    i+=1;outwv[i][repNum][durNum] = effSumRange
 
    i+=1;outwv[i][repNum][durNum] = effSumPnts
    i+=1;outwv[i][repNum][durNum] = cmdStartX
    i+=1;outwv[i][repNum][durNum] = effNoRising
    i+=1;outwv[i][repNum][durNum] = effNoFalling
    i+=1;outwv[i][repNum][durNum] = effNumRiseLevels
 
    i+=1;outwv[i][repNum][durNum] = effNumFallLEvels
    i+=1;outwv[i][repNum][durNum] = effnFallnRiseDiff
    i+=1;outwv[i][repNum][durNum] = effnFallnRiseMismatch
    i+=1;outwv[i][repNum][durNum] = effStartX
    i+=1;outwv[i][repNum][durNum] = effEndX
 
    i+=1;outwv[i][repNum][durNum] = diodeBaselineEndX
    i+=1;outwv[i][repNum][durNum] = diodeBaselineStartX
    i+=1;outwv[i][repNum][durNum] = diodeBaseline
    i+=1;outwv[i][repNum][durNum] = diodeStartX
    i+=1;outwv[i][repNum][durNum] = diodeEndX
 
    i+=1;outwv[i][repNum][durNum] = diodeStartP
    i+=1;outwv[i][repNum][durNum] = diodeEndP
    i+=1;outwv[i][repNum][durNum] = diodeSum
    i+=1;outwv[i][repNum][durNum] = diodeArea
    i+=1;outwv[i][repNum][durNum] = diodeSumRange
 
    i+=1;outwv[i][repNum][durNum] = diodeSumPnts
    i+=1;outwv[i][repNum][durNum] = diodePeakLocX
    i+=1;outwv[i][repNum][durNum] = diodePeakValX
    i+=1;outwv[i][repNum][durNum] = diodeHalfMaxVal
    i+=1;outwv[i][repNum][durNum] = diodeFindRisingFail
 
    i+=1;outwv[i][repNum][durNum] = diodeRisingX
    i+=1;outwv[i][repNum][durNum] = diodeFindFallingFail
    i+=1;outwv[i][repNum][durNum] = diodeFallingX
    i+=1;outwv[i][repNum][durNum] = diodeFWHM
    i+=1;outwv[i][repNum][durNum] = diodeUnexpectedORder
end
It's probably worth mentioning that I am aware of one solution for tracking all the local variables one has generated, but I really don't like it because it involves global variables and execute. Plus, double precision variables wouldn't be an option as far as I know.

function exFunc1()
    //ideally from a non-root folder made for this function (which feels like a lot of work!). In that case, there's no need for the pre-appending unique string "aaa_" before each variable name
    Variable/G aaa_aName=1,aaa_anotherName=10,aaa_oneMoreName=100,aaa_yetAnotherName=1000
   
    String variables=VariableList("aaa_*", ";", 4 ) //get the variables, using a match string to avoid variables not of interest
    print "variables",variables
    Variable i,numVars=itemsinlist(variables)
    make/o/d/n=(numVars) outwv
    String lbl,varN,cmd,assignFormat="outwv[%u]=%s",killFormat="killvariables/z %s"
    for (i=0;i<numVars;i+=1)
        varN=Stringfromlist(i,variables)
        sprintf cmd, assignFormat ,i,varN
        Execute cmd
        lbl=replacestring("aaa_",varN,"")
        SetDimLabel 0,i,$lbl,outwv
       
        sprintf cmd, killFormat,varN
        Execute cmd
    endfor
   
    edit/k=1 outwv.ld
end

March 9, 2018 at 06:41 am - Permalink

Have you considered using a structure with an internal wave? You can use the get/put methods to store the structure.

--
J. J. Weimer
Chemistry / Chemical & Materials Engineering, UAH

March 9, 2018 at 04:30 pm - Permalink

Thanks; I haven't considered that, though I am afraid I am not sure that I understand what you mean. I could iterate through predefined structure variables (like this http://www.igorexchange.com/node/8128#comment-16086), but the work of defining all the variables in a structure seems the same as assigning the variables to their appropriate index in a wave. I could also make an array of structures (like this http://www.igorexchange.com/node/8128#comment-16097). That has some appeal because the substructures could have a name (holding the identity of the variable, e.g., "fullWidthAtHalfMax") and hold the value. At best, that seems similar to the implementation in my long example in the original post -- iterate through a wave and assign to each index and, if necessary, do the same for the row labels (whose names can't be transcribed from the variable names but instead have to be typed out separately).

Even if there does turn out to be a workaround, it seems like it would be great to be able to get a reference even for pass-by-value local variables (String, Variable, Double)? One would need a nameofwave() like function for these and NVAR/SVAR for local variables. Perhaps the local variables are handled in a way that makes this problematic to implement?

March 10, 2018 at 09:28 am - Permalink

I admit that even with your example and description, I cannot appreciate the depth or seriousness of the problem you have, let alone make a viable recommendation to solve it. For my programming, local variables are just that. When I want to store variables between functions, I use globals or structures (with the latter preferred when at all possible). When I want to store variables between invocations of the same experiment, I use globals, although I have also used the method to get/put structures.

The extensive and exhaustive copy+paste+expand+re-do method to carry around local variables just seems to me to be an answer in search of a reasonable use-case.

--
J. J. Weimer
Chemistry / Chemical & Materials Engineering, UAH

March 10, 2018 at 10:18 am - Permalink

jjweimer wrote:
The extensive and exhaustive copy+paste+expand+re-do method to carry around local variables just seems to me to be an answer in search of a reasonable use-case.


For me, the action of interest is to loop through a bunch of local variables (to store them in an output wave for graphing or to pass to other functions). The copy+paste+exapnd+re-do method is one way to go about it (and perhaps the only way currently).

It so happens that if I were using global variables (or many individual waves), I would have a second option. I could loop through globals of interest with WaveList(...), StringList(...), VariableList(...) and WAVE, SVAR, and NVAR. I think it would be great if I could do the same with local variables. It's a small concern, but I think it would make my code easier to maintain and modify. At the very least it would be more concise.

March 10, 2018 at 01:40 pm - Permalink

Consider this example from your opening post.

FindLevel/EDGE=1/Q diodeTemp, diodeHalfMaxVal  
Variable diodeFindFallingFail=V_flag


Here are three alternatives.

* Use a pre-defined structure and store/retrieve the parameters using get/put

FindLevel/EDGE=1/Q diodeTemp, diodeHalfMaxVal  
mystructure.diodeFindFallingFail = v_flag


* Use a pre-defined wave storage system

FindLevel/EDGE=1/Q diodeTemp, diodeHalfMaxVal  
waveofvalues[...] = v_flag


* Store to a string key=value list

FindLevel/EDGE=1/Q diodeTemp, diodeHalfMaxVal  
sprintf mylistofvalues, "%sdiodeFindFallingFail = %d;", mylistofvalues, v_flag


Any one of these methods avoids the issues that you face.

I will leave to others who are wiser than me as to whether functions such as ListofLocalVariables and ListofLocalStings that you desire are desired overall and would indeed be robust enough to have.

--
J. J. Weimer
Chemistry / Chemical & Materials Engineering, UAH

March 11, 2018 at 07:19 am - Permalink

Maybe this will help?

function test()
    variable/g v_a = -500
    variable/g v_b = 500
   
    string s_list = variablelist("*",",",4)
    make/o/n=(itemsinlist(s_list,",")) w_out
   
    test_init_wave(w_out, s_list)
end

function test_init_wave(w, s_list)
    wave& w
    string s_list
   
    execute/q/p nameofwave(w) + " = {" + s_list[0,strlen(s_list)-2] + "}"
end


The thing is that you can instantiate a wave with the following syntax: make/o/n=3 wave = {1,2,3}.

From here on you just generate the comma-separated list of variables in the current path and you execute the instantiation of the wave.

best,
_sk

March 12, 2018 at 05:51 am - Permalink

I think that these are all good approaches, each more or less desirable depending on the application.

jjweimer wrote:
* Use a pre-defined wave storage system

FindLevel/EDGE=1/Q diodeTemp, diodeHalfMaxVal  
waveofvalues[...] = v_flag



I often prefer holding the variables in locals, then assigning them to an output wave at the end of a function. That way, it's easier to reuse the locals later on in the function and to keep track of the indexing of the output wave.

March 12, 2018 at 05:52 am - Permalink

Only now I figured out your code is for local variables. So what I posted earlier might not be relevant.

Let me ask you: what function and/or how do you use the output wave? Do you "unpack" the wave by following this process in reverse? i.e. (i=0;wv1lineOffset=outwv[i][repNum][durNum])

       variable i
    i=0;outwv[i][repNum][durNum]=wv1lineOffset
    i+=1;outwv[i][repNum][durNum] = wv1lineSlope
    i+=1;outwv[i][repNum][durNum] = wv1AvgStartX
    i+=1;outwv[i][repNum][durNum] = wv1Avg
    i+=1;outwv[i][repNum][durNum] = cmdTimingOK
 
    i+=1;outwv[i][repNum][durNum] = wv1AvgStartP
    i+=1;outwv[i][repNum][durNum] = wv1AvgEndP
    i+=1;outwv[i][repNum][durNum] = effSum
    i+=1;outwv[i][repNum][durNum] = effArea
    i+=1;outwv[i][repNum][durNum] = effSumRange
 
    i+=1;outwv[i][repNum][durNum] = effSumPnts
    i+=1;outwv[i][repNum][durNum] = cmdStartX
    i+=1;outwv[i][repNum][durNum] = effNoRising
    i+=1;outwv[i][repNum][durNum] = effNoFalling
    i+=1;outwv[i][repNum][durNum] = effNumRiseLevels
 
    i+=1;outwv[i][repNum][durNum] = effNumFallLEvels
    i+=1;outwv[i][repNum][durNum] = effnFallnRiseDiff
    i+=1;outwv[i][repNum][durNum] = effnFallnRiseMismatch
    i+=1;outwv[i][repNum][durNum] = effStartX
    i+=1;outwv[i][repNum][durNum] = effEndX
 
    i+=1;outwv[i][repNum][durNum] = diodeBaselineEndX
    i+=1;outwv[i][repNum][durNum] = diodeBaselineStartX
    i+=1;outwv[i][repNum][durNum] = diodeBaseline
    i+=1;outwv[i][repNum][durNum] = diodeStartX
    i+=1;outwv[i][repNum][durNum] = diodeEndX
 
    i+=1;outwv[i][repNum][durNum] = diodeStartP
    i+=1;outwv[i][repNum][durNum] = diodeEndP
    i+=1;outwv[i][repNum][durNum] = diodeSum
    i+=1;outwv[i][repNum][durNum] = diodeArea
    i+=1;outwv[i][repNum][durNum] = diodeSumRange
 
    i+=1;outwv[i][repNum][durNum] = diodeSumPnts
    i+=1;outwv[i][repNum][durNum] = diodePeakLocX
    i+=1;outwv[i][repNum][durNum] = diodePeakValX
    i+=1;outwv[i][repNum][durNum] = diodeHalfMaxVal
    i+=1;outwv[i][repNum][durNum] = diodeFindRisingFail
 
    i+=1;outwv[i][repNum][durNum] = diodeRisingX
    i+=1;outwv[i][repNum][durNum] = diodeFindFallingFail
    i+=1;outwv[i][repNum][durNum] = diodeFallingX
    i+=1;outwv[i][repNum][durNum] = diodeFWHM
    i+=1;outwv[i][repNum][durNum] = diodeUnexpectedORder



best,
_sk

March 12, 2018 at 07:22 am - Permalink

_sk wrote:
Only now I figured out your code is for local variables. So what I posted earlier might not be relevant.

I like the idea of using global variables and Execute, though it has some limitations that using local variables doesn't (e.g., conflicting names with other globals and no use of Double)

_sk wrote:
Let me ask you: what function and/or how do you use the output wave? Do you "unpack" the wave by following this process in reverse? i.e. (i=0;wv1lineOffset=outwv[i][repNum][durNum])


In this case, I might compute statistics across a dimension (e.g., average a specific row and layer across columns, which hold the value of one parameter across replicates). I would also plot things like: display/k=1 outwv[%parameterName0][][0] vs outwv[%parameterName1][][0]. The only requirement for the order of the output wave rows is that it has to be consistent with the row labels.

March 12, 2018 at 09:27 am - Permalink

aoa wrote:

I often prefer holding the variables in locals, then assigning them to an output wave at the end of a function. That way, it's easier to reuse the locals later on in the function and to keep track of the indexing of the output wave.


Using locals within a function is not the problem. The problem is that you want an automatic way to infer whether they exist or not (to avoid having to remember and/or enter again manually).

In the meantime, I have another approach as you mention already above. Use dimension labels on the wave.

FindLevel/EDGE=1/Q diodeTemp, diodeHalfMaxVal  
waveofvalues[%diodeFindFallingFail] = v_flag


--
J. J. Weimer
Chemistry / Chemical & Materials Engineering, UAH

March 12, 2018 at 11:07 am - Permalink

jjweimer wrote:
In the meantime, I have another approach as you mention already above. Use dimension labels on the wave.

FindLevel/EDGE=1/Q diodeTemp, diodeHalfMaxVal  
waveofvalues[%diodeFindFallingFail] = v_flag


@JJ
Given the above, what was the meaning of your previous suggestion? Is [...] shorthand for multiple dimensions for example, Waveofvalues[][][]?

jjweimer wrote:
* Use a pre-defined wave storage system

FindLevel/EDGE=1/Q diodeTemp, diodeHalfMaxVal  
waveofvalues[...] = v_flag

March 12, 2018 at 11:41 am - Permalink

jtigor wrote:

@JJ
Given the above, what was the meaning of your previous suggestion? Is [...] shorthand for multiple dimensions for example, Waveofvalues[][][]?


The [...] was shorthand for index location. I'd forgotten entirely about dimension labels until this morning.

--
J. J. Weimer
Chemistry / Chemical & Materials Engineering, UAH

March 12, 2018 at 12:33 pm - Permalink

I agree that dimension labels are a nice solution, but I have always hesitated to use them in functions that might be called from loops. In many cases, I want to calculate the same set of parameters over many trial runs and store them in different columns/layers of an output wave. It seems better to hard-code those output wave indexes because I would guess that finding the index of a label would take more time (as suggested by the help file for dimension labels). I was curios how bad it is, so I ran the following test. Using dimension labels appears to be 3-6 times slower for look-up waves 10-100 parameters long, growing increasingly slow for longer lookup waves.

Here's the test function I made:
function lblTest(testWvPnts,lblLens,numTests)
    Variable testWvPnts     //how long of a wave to generate
    Variable lblLens            //how long the labels will be
    Variable numTests       //how many indexes to access
   
    //make a wave, give it some values and labels
    Variable i,j,asciiMin=97,asciiMax=122,range=122-97,asciiVal
    Make/o/n=(testWvPnts)/free lblTestWv
    Make/o/n=(testWvPnts)/T/free lblTestList
    String lbl
    for (i=0;i<testWvPnts;i+=1)
        lbl=""
        for (j=0;j<lblLens;j+=1)
            asciiVal=asciiMin+enoise(range/2)+range/2
            lbl+=num2char(asciiVal)
        endfor
       
        SetDimLabel 0,i,$lbl,lblTestWv
        lblTestWv[i]=i
        lblTestList[i]=lbl
    endfor
   
    //pick out some labels and indices to collect
    Make/o/n=(numTests)/free testRows
    testRows=floor(enoise(testWvPnts/2) + testWvPnts/2)
    make/o/n=(numTests)/t/free testLbls
    testLbls=lblTestList[testRows[p]]
   
    variable mstimer,temp,directIndexTime,dimLabelTime
   
    //compare run time for direct indexing and dimension label indexing
    mstimer=startmstimer
    for (i=0;i<numTests;i+=1)
        temp=lblTestWv[testRows[i]]
    endfor
    directIndexTime=stopmstimer(mstimer)
   
    mstimer=startmstimer
    for (i=0;i<numTests;i+=1)
        temp=lblTestWv[%$testLbls[i]]
    endfor
    dimLabelTime=stopmstimer(mstimer)
   
    //Print "directIndexTime",directIndexTime,"dimLabelTime",dimLabelTime       //uncomment to pring every result
   
    return dimLabelTime/directIndexTime
end


And here are a few example runs:
•make/o/d/n=100 tests
•tests=lblTest(5,10,30);print mean(tests)
  3.04948
•tests=lblTest(10,10,30);print mean(tests)
  3.2401
•tests=lblTest(20,10,30);print mean(tests)
  3.57727
•tests=lblTest(50,10,30);print mean(tests)
  4.65896
•tests=lblTest(100,10,30);print mean(tests)
  6.19278
•tests=lblTest(200,10,30);print mean(tests)
  9.13127

March 12, 2018 at 01:57 pm - Permalink

aoa wrote:
In many cases, I want to calculate the same set of parameters over many trial runs and store them in different columns/layers of an output wave.


Alright, this is slowly becoming clearer.

You have an exhaustive list of parameters that are read from somewhere or calculated using something. You want to accumulate sets of the parameters over different experimental runs. You want to store the sets in a cohesive way. You want to analyze the sets within and across each other.

I see that you make the assumption that all variables are double precision.

Locally, I would set up a structure. Each variable in the structure would have the same size (double). I would accumulate the values into the structure, referencing them therefore by name. I would use FBinWrite to store the structure. When the time came to analyze the data, I would use FBinRead to read the sets of structures back into waves (or to columns in a matrix). I would pre-define dimension labels in the waves (or matrix) as need to make the administrative duties of the analysis easier. Or, I would just read only those parts of the stored structure that I want to analyze rather than carrying around everything.

--
J. J. Weimer
Chemistry / Chemical & Materials Engineering, UAH

March 12, 2018 at 01:25 pm - Permalink

jjweimer wrote:
Locally, I would set up a structure. Each variable in the structure would have the same size (double). I would accumulate the values into the structure, referencing them therefore by name. I would use FBinWrite to store the structure. When the time came to analyze the data, I would use FBinRead to read the sets of structures back into waves (or to columns in a matrix). I would pre-define dimension labels in the waves (or matrix) as need to make the administrative duties of the analysis easier. Or, I would just read only those parts of the stored structure that I want to analyze rather than carrying around everything.


Thanks for the input. I agree that this is a solid approach, though it still requires pre-defining the structure values by name, as you point out.

March 12, 2018 at 01:56 pm - Permalink

Quote:
I would guess that finding the index of a label would take more time


You can resolve the labels outside the loop using FindDimLabel.

March 12, 2018 at 02:11 pm - Permalink

hrodstein wrote:
Quote:
I would guess that finding the index of a label would take more time


You can resolve the labels outside the loop using FindDimLabel.

Absolutely. However, in that case, in the loop, you have to keep track of the parameters' index, which is arbitrary. I think it would be great if Igor could handle more of that arbitrary stuff for the programmer. Using local variables with a meaningful name and then having a way to iterate through them seems like a good way. The earlier examples with Execute show exactly what I imagine, but it's currently only applicable to global variables.

If it's even possible to implement this wish, I suppose Igor's iteration through the locals would have some overhead of its own. I am of course not sure how it would compare to directly indexing.

March 12, 2018 at 02:42 pm - Permalink

how about defining a bunch of constants as labels:
constant kMyFirstParameter=1
constant kMySecondParameter=2
...
constant kMyNthParameter=N

and replace your local variables with a wave, so that

Variable avgWins=0.001

is replaced by

w[kAvgWins]=0.001

then you can archive like this:

outwv[][repNum][durNum]=w[p]

March 14, 2018 at 07:57 am - Permalink

tony wrote:
how about defining a bunch of constants as labels:
constant kMyFirstParameter=1
constant kMySecondParameter=2
...
constant kMyNthParameter=N

and replace your local variables with a wave, so that

Variable avgWins=0.001

is replaced by

w[kAvgWins]=0.001

then you can archive like this:

outwv[][repNum][durNum]=w[p]


Thank you for the suggestion. The only missing part is that there's no automated way to label the dimensions of an output wave. If there were a ConstantList(...) function similar to VariableList(...) then one could loop through the constant names and assign them to labels. (I suppose the slight advantage of constants over global variables would be that they don't crowd the experiment space, though they could still lead to name conflicts, unlike local variables.)

March 30, 2018 at 08:07 am - Permalink

aoa wrote:
Thank you for the suggestion. The only missing part is that there's no automated way to label the dimensions of an output wave.


I'm not sure why you would need to use dimension labels. The constants become de facto dimension labels (each one ties your variable name to a dimension index). If your code is all within one procedure file you can use static constant definitions and you only have to list the names once at the top of the file. It's not required, but sticking to the WM convention of a k prefix for constants will help avoid name conflicts.

you would archive results like this:

outwv[][repNum][durNum]=w[p]

and retrieve archived data like this:

w=outwv[p][repNum][durNum],

no transfer of dimension labels required.

So, w[kVariableName] and outwv[kVariableName][repNum][durNum] can be used instead of a variable that would have been named VariableName.

April 1, 2018 at 12:48 pm - Permalink

tony wrote:
I'm not sure why you would need to use dimension labels. The constants become de facto dimension labels (each one ties your variable name to a dimension index). If your code is all within one procedure file you can use static constant definitions and you only have to list the names once at the top of the file. It's not required, but sticking to the WM convention of a k prefix for constants will help avoid name conflicts.


I think that's a good approach for many cases. However, I really like dimension labels and so I often want the names to end up stored in them. My short answer to why dimension labels is: keeping track of things and making graphs.

The slightly longer answer: They help me keep track of what is what across experiments and across time (as my code might at least potentially change across either one). Unlike constants, one can also index into a wave from the command line by referring to a dimension label e.g. Display wave1[%param1] vs wave1[%param2]. The dimension labels also work nicely with many built-in Igor interfaces. For example, when one plots by referring to a dimension label, the label can be seen in the Modify Trace Appearance dialogue, so one can come back to the trace later and easily recall what is plotted, and the Wave Subrange dialogue (from within the same Modify Trace Appearance Dialogue) also helpfully references dimension labels when present.

April 17, 2018 at 03:22 pm - Permalink