Read and extract values of variables and waves' name using multithread function from mass # of waves

I don't know that this is the *only* problem, but this:

        NVAR gnamewave= dfr:$namewave // NVAR is for (numeric) Variable/G

needs to be:

        SVAR gnamewave= dfr:$namewave  // SVAR for String/Gs

--Jim Prouty
Software Engineer, WaveMetrics, Inc.

Log in or register to post comments

January 17, 2014 at 09:19 am - Permalink

viralvector

[quote=JimProuty]I don't know that this is the *only* problem, but this:

        NVAR gnamewave= dfr:$namewave // NVAR is for (numeric) Variable/G

needs to be:

        SVAR gnamewave= dfr:$namewave  // SVAR for String/Gs

--Jim Prouty
Software Engineer, WaveMetrics, Inc.[/quote]

Thanks, I just learned the function of SVAR.
Here is what I added to:
In the function of ThreadSafe Function ThreadWorker(wIn, waveIndex)
I introduced
string NameWave =NameofWave(wIn)
String/G gNameWave=NameWave

Then, in the function of Function ThreadAtATimeDemo()
I added one column to the table

    // Make output wave
    Make/O/D/N=(numDatasetsToProcess,3)  outputDFR:results = NaN
    Wave results = outputDFR:results
    SetDimLabel 1,0,CenterOfMass,results
    SetDimLabel 1,1,Peak,results
      SetDimLabel 1,2,Name,results

And introduced SVAR for wave name

 DFREF dfr = ThreadGroupGetDFR(threadGroupID, 0)
        if (DataFolderRefStatus(dfr) != 0)
            // Process results
            NVAR gcenterOfMass = dfr:gcenterOfMass
            NVAR gxPeak = dfr:gxPeak
            SVAR gNameWave= dfr:gNameWave
            NVAR gWaveIndex = dfr:gWaveIndex
           
            results[gWaveIndex][0] = gcenterOfMass
            results[gWaveIndex][1] = gxPeak
              results[gWaveIndex][2] = gNameWave

But, I received error saying results[gWaveIndex][2] = gNameWave
unknown/inappropriate name or symbol...
with my current knowledge, I don't know where is the error and I need your help.

Thank you!

Here is the code:

#pragma rtGlobals=3     // Use modern global access method.
 
Menu "Macros"
    "Thread-at-a-Time Demo", ThreadAtATimeDemo()
End
 
// These control various troubleshooting items and other aspects of the demonstration
Constant kNumDatasetsToProcess = 400        // Number of sample input waves to create and process
Constant kSampleWaveNumPoints = 10000       // Number of points in each sample input wave
Constant kPrintStartThreadMessage = 1
Constant kPrintGotThreadDFMessage = 1
Constant kSimulateVaryingComputationTime = 1
 
ThreadSafe Function ThreadWorker(wIn, waveIndex)
    WAVE wIn
    Variable waveIndex
    String NameWave = NameofWave(wIn)
  Wavestats/q wIn
    variable xPeak = V_maxloc
    Variable i,n=numpnts(wIn), massTimesX=0
    for(i=0; i<n; i+=1)
        massTimesX += wIn[i]*pnt2x(wIn,i)
    endfor
    Variable totalMass=sum(wIn)
    Variable centerOfMass= massTimesX/totalMass
  //     printf "Center of Mass: %g", centerofMass
//return centerOfMass
 
 
    // This is used to simulate varying computation time
    if (kSimulateVaryingComputationTime > 0)
        Variable randomSecs = abs(enoise(kSimulateVaryingComputationTime))
        Sleep/S randomSecs
    endif
    
    //Variable wMin = WaveMin(wIn)
    //Variable wMax = WaveMax(wIn)
    
    // Create data folder to return to calling thread
    NewDataFolder/S output
    
    // Create output variables
    // Variable/G gMin = wMin
    Variable/G gcenterOfMass =centerOfMass
    Variable/G gxPeak = xPeak
      String/G gNameWave=NameWave
    
    // This tells the calling thread which wave was processed
    Variable/G gWaveIndex = waveIndex
    
    // Send data back to main thread
    ThreadGroupPutDF 0, :
    
    return 0        // Success
End
 
 
Function ThreadAtATimeDemo()
    if (IgorVersion() < 6.23)
        Abort "This demo requires Igor Pro 6.23 or later."
    endif
 
    Variable t0 = StopMSTimer(-2)       // Used to time the function
 
    DFREF originalDFR = GetDataFolderDFR()
 
    DFREF outputDFR = GetDataFolderDFR()
 
    Variable numDatasetsToProcess = kNumDatasetsToProcess
    Variable numDatasetsProcessed = 0
    
    // Make output wave
    Make/O/D/N=(numDatasetsToProcess,3)  outputDFR:results = NaN
    Wave results = outputDFR:results
    SetDimLabel 1,0,CenterOfMass,results
    SetDimLabel 1,1,Peak,results
      SetDimLabel 1,2,Name,results
      
    // Create output table
    DoWindow /F OutputTable
    if (V_flag == 0)
        Edit /N=OutputTable /W=(466,49,906,652) results.ld
    endif
 
    // Look for sample data
    Variable numPointsInSampleData = kSampleWaveNumPoints
//  NewDataFolder /O :SampleData
    DFREF inputDFR = :
    String name
    Variable i
    for(i=0; i<numDatasetsToProcess; i+=1)
    
    sprintf name, "(%d)", i     // wave0, wave1
    //  Make/O/D/N=(numPointsInSampleData) inputDFR:$name = gnoise(1)
        SetDimLabel 0,i,$name,results
    endfor
 
    // Create threads
    Variable numThreads = ThreadProcessorCount
    Variable threadGroupID = ThreadGroupCreate(numThreads)
    
    SetDataFolder inputDFR      // Needed for WaveRefIndexed
    
    // Dispatch threads and collect output
    Variable threadIndex
    Variable waveIndex = 0
    do
        // Start any free threads
        do
            if (waveIndex >= numDatasetsToProcess)
                break                               // No more input data to dispatch to a thread
            endif
            
            // Find a free thread
            threadIndex = ThreadGroupWait(threadGroupID, -2)        // Requires Igor Pro 6.23
            threadIndex -= 1    // Because ThreadGroupWait returns threadIndex+1
            if (threadIndex < 0)
                break           // No free threads
            endif
        
            // Start thread
            Wave wIn = WaveRefIndexed("", waveIndex, 4)
 
            ThreadStart threadGroupID, threadIndex, ThreadWorker(wIn, waveIndex)
            if (kPrintStartThreadMessage)
                Printf "Started thread %d\r", threadIndex       // For debugging only
            endif
            waveIndex += 1
        while(1)
        
        // See if thread output is available
        DFREF dfr = ThreadGroupGetDFR(threadGroupID, 0)
        if (DataFolderRefStatus(dfr) != 0)
            // Process results
            NVAR gcenterOfMass = dfr:gcenterOfMass
            NVAR gxPeak = dfr:gxPeak
            SVAR gNameWave= dfr:gNameWave
            NVAR gWaveIndex = dfr:gWaveIndex
           
        
            results[gWaveIndex][0] = gcenterOfMass
            results[gWaveIndex][1] = gxPeak
              results[gWaveIndex][2] = gNameWave
            // Update the output table
            DoUpdate
 
            // For debugging only
            if (kPrintGotThreadDFMessage)
                Printf "Got data for wave %d: CenterOfMass=%g, Peak=%g\r", gWaveIndex, gcenterOfMass, gxPeak
            endif
            
            numDatasetsProcessed += 1
            
            // This is a free data folder that would be killed automatically by Igor.
            // Nonetheless we explicitly kill it here and now.
            KillDataFolder dfr
        endif
    while(numDatasetsProcessed < numDatasetsToProcess)
        
    Variable dummy = ThreadGroupRelease(threadGroupID)
    
    Variable t1 = StopMSTimer(-2)
    Variable elapsedTime = (t1 - t0) / 1E6
    Printf "Finished ThreadAtATimeDemo in %.2f seconds.\r", elapsedTime
 
    SetDataFolder originalDFR
End
 
Function CleanupExperiment()    // We call this to prepare the experiment for shipping
    DoWindow /K OutputTable         // The table is recreated by the demo
    KillWaves/Z results                 // Output wave is recreated by the demo
    KillDataFolder /Z root:SampleData   // Sample data is recreated by the demo
End

Log in or register to post comments

January 17, 2014 at 10:25 am - Permalink

JimProuty

[quote]
Thanks, I just learned the function of SVAR.
Here is what I added to:
In the function of ThreadSafe Function ThreadWorker(wIn, waveIndex)
I introduced
string NameWave =NameofWave(wIn)
String/G gNameWave=NameWave

Then, in the function of Function ThreadAtATimeDemo()
I added one column to the table

    // Make output wave
    Make/O/D/N=(numDatasetsToProcess,3)  outputDFR:results = NaN
    Wave results = outputDFR:results
    SetDimLabel 1,0,CenterOfMass,results
    SetDimLabel 1,1,Peak,results
      SetDimLabel 1,2,Name,results

And introduced SVAR for wave name

 DFREF dfr = ThreadGroupGetDFR(threadGroupID, 0)
        if (DataFolderRefStatus(dfr) != 0)
            // Process results
            NVAR gcenterOfMass = dfr:gcenterOfMass
            NVAR gxPeak = dfr:gxPeak
            SVAR gNameWave= dfr:gNameWave
            NVAR gWaveIndex = dfr:gWaveIndex
           
            results[gWaveIndex][0] = gcenterOfMass
            results[gWaveIndex][1] = gxPeak
              results[gWaveIndex][2] = gNameWave

But, I received error saying results[gWaveIndex][2] = gNameWave
unknown/inappropriate name or symbol...
with my current knowledge, I don't know where is the error and I need your help.

[/quote]

Note that the results wave is numeric.
The string is not.
You can't assign a string to a number.

--Jim Prouty
Software Engineer, WaveMetrics, Inc.

Log in or register to post comments

January 17, 2014 at 11:58 am - Permalink

viralvector

[quote=JimProuty][quote]
Thanks, I just learned the function of SVAR.
Here is what I added to:
In the function of ThreadSafe Function ThreadWorker(wIn, waveIndex)
I introduced
string NameWave =NameofWave(wIn)
String/G gNameWave=NameWave

Then, in the function of Function ThreadAtATimeDemo()
I added one column to the table

    // Make output wave
    Make/O/D/N=(numDatasetsToProcess,3)  outputDFR:results = NaN
    Wave results = outputDFR:results
    SetDimLabel 1,0,CenterOfMass,results
    SetDimLabel 1,1,Peak,results
      SetDimLabel 1,2,Name,results

And introduced SVAR for wave name

 DFREF dfr = ThreadGroupGetDFR(threadGroupID, 0)
        if (DataFolderRefStatus(dfr) != 0)
            // Process results
            NVAR gcenterOfMass = dfr:gcenterOfMass
            NVAR gxPeak = dfr:gxPeak
            SVAR gNameWave= dfr:gNameWave
            NVAR gWaveIndex = dfr:gWaveIndex
           
            results[gWaveIndex][0] = gcenterOfMass
            results[gWaveIndex][1] = gxPeak
              results[gWaveIndex][2] = gNameWave

But, I received error saying results[gWaveIndex][2] = gNameWave
unknown/inappropriate name or symbol...
with my current knowledge, I don't know where is the error and I need your help.

[/quote]

Note that the results wave is numeric.
The string is not.
You can't assign a string to a number.

--Jim Prouty
Software Engineer, WaveMetrics, Inc.[/quote]

Thank you.
Are there any ways to extract wave name as well as the two variables to the table using this algorithm? If so, you are welcome to make changes.
I guess the problem is the gWaveIndex ? I noticed that gWaveIndex was assigned as variable. But I don't know how to convert it to string.
Please bear in mind, I am new to IGOR and I try my best to understand programming syntax as a biologist. I need your advices and suggestions.

Thank you again.

Log in or register to post comments

January 17, 2014 at 12:47 pm - Permalink

viralvector

[quote=JimProuty][quote]
Thanks, I just learned the function of SVAR.
Here is what I added to:
In the function of ThreadSafe Function ThreadWorker(wIn, waveIndex)
I introduced
string NameWave =NameofWave(wIn)
String/G gNameWave=NameWave

Then, in the function of Function ThreadAtATimeDemo()
I added one column to the table

    // Make output wave
    Make/O/D/N=(numDatasetsToProcess,3)  outputDFR:results = NaN
    Wave results = outputDFR:results
    SetDimLabel 1,0,CenterOfMass,results
    SetDimLabel 1,1,Peak,results
      SetDimLabel 1,2,Name,results

And introduced SVAR for wave name

 DFREF dfr = ThreadGroupGetDFR(threadGroupID, 0)
        if (DataFolderRefStatus(dfr) != 0)
            // Process results
            NVAR gcenterOfMass = dfr:gcenterOfMass
            NVAR gxPeak = dfr:gxPeak
            SVAR gNameWave= dfr:gNameWave
            NVAR gWaveIndex = dfr:gWaveIndex
           
            results[gWaveIndex][0] = gcenterOfMass
            results[gWaveIndex][1] = gxPeak
              results[gWaveIndex][2] = gNameWave

But, I received error saying results[gWaveIndex][2] = gNameWave
unknown/inappropriate name or symbol...
with my current knowledge, I don't know where is the error and I need your help.

[/quote]

Note that the results wave is numeric.
The string is not.
You can't assign a string to a number.

--Jim Prouty
Software Engineer, WaveMetrics, Inc.[/quote]

Hello Jim,

Thank you for pointing out the error again. I have fixed the incomparability (text vs. numeric)
I got what I want with the minimal knowledge of programming.
Here is the code, just in case someone is interested in trimming it.

#pragma rtGlobals=3     // Use modern global access method.
 
Menu "Macros"
    "Read_CM_Peak_MultiThreads", ThreadAtATimeDemo()
End
 
// These control various troubleshooting items and other aspects of the demonstration
Constant kNumDatasetsToProcess = 10     // Number of sample input waves to create and process
Constant kSampleWaveNumPoints = 10000       // Number of points in each sample input wave
Constant kPrintStartThreadMessage = 1
Constant kPrintGotThreadDFMessage = 1
Constant kSimulateVaryingComputationTime = 1
 
ThreadSafe Function ThreadWorker(wIn, waveIndex)
    WAVE wIn
    Variable waveIndex
    String NameWave = NameofWave(wIn)
  Wavestats/q/z wIn
    variable xPeak = V_maxloc
    Variable i,n=numpnts(wIn), massTimesX=0
    for(i=0; i<n; i+=1)
        massTimesX += wIn[i]*pnt2x(wIn,i)
    endfor
    Variable totalMass=sum(wIn)
    Variable centerOfMass= massTimesX/totalMass
  //     printf "Center of Mass: %g", centerofMass
//return centerOfMass
 
 
    // This is used to simulate varying computation time
    if (kSimulateVaryingComputationTime > 0)
        Variable randomSecs = abs(enoise(kSimulateVaryingComputationTime))
        Sleep/S randomSecs
    endif
    
    //Variable wMin = WaveMin(wIn)
    //Variable wMax = WaveMax(wIn)
    
    // Create data folder to return to calling thread
    NewDataFolder/S output
    
    // Create output variables
    // Variable/G gMin = wMin
    Variable/G gcenterOfMass =centerOfMass
    Variable/G gxPeak = xPeak
      String/G gNameWave=NameWave
    
    // This tells the calling thread which wave was processed
    Variable/G gWaveIndex = waveIndex
    
    // Send data back to main thread
    ThreadGroupPutDF 0, :
    
    return 0        // Success
End
 
 
Function ThreadAtATimeDemo()
    if (IgorVersion() < 6.23)
        Abort "This demo requires Igor Pro 6.23 or later."
    endif
 
    Variable t0 = StopMSTimer(-2)       // Used to time the function
 
    DFREF originalDFR = GetDataFolderDFR()
 
    DFREF outputDFR = GetDataFolderDFR()
 
    Variable numDatasetsToProcess = kNumDatasetsToProcess
    Variable numDatasetsProcessed = 0
    
    // Make output wave
    Make/O/D/N=(numDatasetsToProcess,2)  outputDFR:results = NaN
    wave results= outputDFR:results
    SetDimLabel 1,0,CenterOfMass,results
    SetDimLabel 1,1,Peak,results
    
    make/O /T/N=(numDatasetsToProcess)  outputDFR:resultstext
    Wave/t resultstext = outputDFR:resultstext
     // SetDimLabel 0,0,name,resultstext
            
     // Make/T/N=(numpnts(numWave)) textWave = num2str(numWave)
     // str2num
      
    // Create output table
    DoWindow /F OutputTable
    if (V_flag == 0)
        Edit /N=OutputTable /W=(466,49,906,652) results.ld
        Edit/N=OutputTableText resultstext
    endif
 
    // Look for sample data
    Variable numPointsInSampleData = kSampleWaveNumPoints
//  NewDataFolder /O :SampleData
 
    DFREF inputDFR = :
    String name 
    Variable i
    for(i=0; i<numDatasetsToProcess; i+=1)
    
    sprintf name, "(%d)", i     // wave0, wave1
    //  Make/O/D/N=(numPointsInSampleData) inputDFR:$name = gnoise(1)
        SetDimLabel 0,i,$name,results
    endfor
 
 
    // Create threads
    Variable numThreads = ThreadProcessorCount
    Variable threadGroupID = ThreadGroupCreate(numThreads)
    
    SetDataFolder inputDFR      // Needed for WaveRefIndexed
    
    // Dispatch threads and collect output
    Variable threadIndex
    Variable waveIndex = 0
    do
        // Start any free threads
        do
            if (waveIndex >= numDatasetsToProcess)
                break                               // No more input data to dispatch to a thread
            endif
            
            // Find a free thread
            threadIndex = ThreadGroupWait(threadGroupID, -2)        // Requires Igor Pro 6.23
            threadIndex -= 1    // Because ThreadGroupWait returns threadIndex+1
            if (threadIndex < 0)
                break           // No free threads
            endif
        
            // Start thread
            Wave wIn = WaveRefIndexed("", waveIndex, 4)
 
            ThreadStart threadGroupID, threadIndex, ThreadWorker(wIn, waveIndex)
            if (kPrintStartThreadMessage)
                Printf "Started thread %d\r", threadIndex       // For debugging only
            endif
            waveIndex += 1
        while(1)
        
        // See if thread output is available
        DFREF dfr = ThreadGroupGetDFR(threadGroupID, 0)
        if (DataFolderRefStatus(dfr) != 0)
            // Process results
            NVAR gcenterOfMass = dfr:gcenterOfMass
            NVAR gxPeak = dfr:gxPeak
            SVAR gNameWave= dfr:gNameWave
            NVAR gWaveIndex = dfr:gWaveIndex
           
        
            results[gWaveIndex][0] = gcenterOfMass
            results[gWaveIndex][1] = gxPeak
          resultstext[gWaveIndex][0] =gNameWave
            // Update the output table
            DoUpdate
 
            // For debugging only
            if (kPrintGotThreadDFMessage)
                Printf "Got data for wave %d: CenterOfMass=%g, Peak=%g\r", gWaveIndex, gcenterOfMass, gxPeak
            endif
            
            numDatasetsProcessed += 1
            
            // This is a free data folder that would be killed automatically by Igor.
            // Nonetheless we explicitly kill it here and now.
            KillDataFolder dfr
        endif
    while(numDatasetsProcessed < numDatasetsToProcess)
        
    Variable dummy = ThreadGroupRelease(threadGroupID)
    
    Variable t1 = StopMSTimer(-2)
    Variable elapsedTime = (t1 - t0) / 1E6
    Printf "Finished ThreadAtATimeDemo in %.2f seconds.\r", elapsedTime
 
    SetDataFolder originalDFR
End
 
Function CleanupExperiment()    // We call this to prepare the experiment for shipping
    DoWindow /K OutputTable         // The table is recreated by the demo
    DoWindow/K OutputTabletext
    KillWaves/Z results                 // Output wave is recreated by the demo
    KillDataFolder /Z root:SampleData   // Sample data is recreated by the demo
End

Log in or register to post comments

January 17, 2014 at 03:28 pm - Permalink

jtigor

Another approach (instead of using two waves one for numeric results and one for text) is to create your results wave as a text wave and write the numeric values to it using either sprintf or num2str. How you will use the results determines whether this approach is appropriate. If you need the values for further calculations, then this will probably end up being more trouble than it's worth.

Log in or register to post comments

January 17, 2014 at 05:06 pm - Permalink

viralvector

[quote=jtigor]Another approach (instead of using two waves one for numeric results and one for text) is to create your results wave as a text wave and write the numeric values to it using either sprintf or num2str. How you will use the results determines whether this approach is appropriate. If you need the values for further calculations, then this will probably end up being more trouble than it's worth.[/quote]

Hi jtigor,
Obtaining such values have made my life so much easier!!!
I see your point. I will try to change the output into string using num2str. If I got stuck in doing such changes, I will seek help here.

Thanks!

Log in or register to post comments

January 17, 2014 at 08:08 pm - Permalink

johnweeks

[quote=jtigor]Another approach (instead of using two waves one for numeric results and one for text) is to create your results wave as a text wave and write the numeric values to it using either sprintf or num2str. How you will use the results determines whether this approach is appropriate. If you need the values for further calculations, then this will probably end up being more trouble than it's worth.[/quote]

This can be a good technique. I think it's worth pointing out that when transmitting values via text you have to be *very* careful that you preserve the necessary precision of the numbers. A round-trip through num2str/str2num will truncate the values to 6 digits.

The num-string-num conversions are also slow. Since you are going to the trouble of writing threaded code, I presume speed is important to you.

As jtigor says, how you use the results is a crucial consideration.

John Weeks
WaveMetrics, Inc.
support@wavemetrics.com

Log in or register to post comments

January 20, 2014 at 09:37 am - Permalink

jjweimer

Perhaps you could replace the text wave of names with a string variable. This ...

if (DataFolderRefStatus(dfr) != 0)
// Process results
    NVAR gcenterOfMass = dfr:gcenterOfMass
    NVAR gxPeak = dfr:gxPeak
    SVAR gNameWave= dfr:gNameWave
    NVAR gWaveIndex = dfr:gWaveIndex
 
 
    results[gWaveIndex][0] = gcenterOfMass
    results[gWaveIndex][1] = gxPeak
    resultstext[gWaveIndex][0] =gNameWave

... becomes this ...

// create results as a 2 column matrix
Make/O/D/N=(numDatasetsToProcess,2)  outputDFR:results = NaN
// create gListInfo to store content in the same place that you create gCenterOfMass
// make sure it is initialized to be blank
String/G gListInfo = ""
 
if (DataFolderRefStatus(dfr) != 0)
// Process results
    NVAR/DFREF=dfr gcenterOfMass, gxPeak, gWaveIndex
    SVAR/DFREF=dfr gNameWave, gListInfo   // create gListInfo when you create gNameWave
    results[gWaveIndex][0] = gcenterOfMass
    results[gWaveIndex][1] = gxPeak
    gListInfo += num2str(gWaveIndex) + "=" + gNameWave + ";" // one option with num2str (use this or below, not both!)
    sprintf("%s%d=%s;",gListInfo,gWaveIndex,gNameWave) // alternative with sprint (use this or above, not both!)

Your processing afterward is then a case of extracting the keyword=value set from gListInfo.

--
J. J. Weimer
Chemistry / Chemical & Materials Engineering, UAHuntsville

Log in or register to post comments

January 20, 2014 at 10:57 am - Permalink