Multithreading wave read issues

Share icon

Hello,

   I have a genetic algorithm program that works well, but I want to make it more efficient using multithreading and I rewrote the code to include multithreading, but I am running into problems reading waves when using multiple threads. To simplify what is going on I wrote a simpler program to demonstrate the problem. This simple code is supposed to pick two random waves (named "indX" where X is an integer) that exist and write them into temporary waves for further processing. Here is the code:

function BreedTest()
 
 
    variable tgID=ThreadGroupCreate(4)
 
    make /o/d/n=500 indA0,indA1,indA2,indA3,indB0,indB1,indB2,indb3 //create temp waves
 
 
    ThreadStart tgID,0, CrossoverThreadTest(indA0,indB0)
    ThreadStart tgID,1, CrossoverThreadTest(indA1,indB1)
    ThreadStart tgID,2, CrossoverThreadTest(indA2,indB2)
    ThreadStart tgID,3, CrossoverThreadTest(indA3,indB3)
        
    // Wait for all threads to finish before starting another batch of threads
    do
        Variable threadGroupStatus = ThreadGroupWait(tgID,100)
    while (threadGroupStatus != 0)
    
end
 
threadsafe function CrossoverThreadTest(indA,indB)
    wave indA,indB //temp waves for each thread
    
    string loopstr1,loopstr2
    variable a,b
    
    a=floor(20+enoise(20))
    b=floor(20+enoise(20))
    
    print a
    print b
    
    loopstr1="ind"+num2str(a)
    loopstr2="ind"+num2str(b)
    
    Duplicate /o $loopstr1,indA
    Duplicate /o $loopstr2,indB
 
end

when I run "crossoverthreadtest" by itself it functions perfectly. However, when I run it in the multithreading function ("breedtest") I  get the error: "While executing Duplicate, the following error occurred: expected wave name".

It seems that when multithreading the program isn't able to see the "indX" waves, but doesn't have any issues when it is single thread. I am assuming this has something to do with how multithreading handles directories and waves but couldn't find anything in the help files. Does anyone know how to make this work?

Also, I am creating and passing separate temp files to each thread to be written too. Is there a more efficient method? Thanks.

 

 

 

Please execute the following command:

DisplayHelpTopic "Thread Data Environment"

Note this paragraph:

Except for waves passed to the thread worker function as parameters and the thread worker's return value, the input and output queues are the only way for a thread to share data with the main thread. Examples below illustrate the use of these queues.

 

Your CrossoverThreadTest would only have access to the two waves passed as input parameters.

You might be able to pass a wave reference wave (Make/WAVE ...) to the threadsafe worker function, but as you've written the code even if the Duplicate worked the destination would be the private data folder of the preemptive thread but since you are not using ThreadGroupPutDF and ThreadGroupGetDFR I think you'll just otherwise lose that wave you create.

 

September 12, 2019 at 03:39 pm - Permalink

Aclight, Thank you for the information. Based on your comments it sounds like I should use ThreadGroupPutDF and ThreadGroupGetDFR. I have been looking at the Help topics on these and it isn't totally clear how to best implement it for my goal. Can you provide a code snippet showing how I can do this?

September 12, 2019 at 04:34 pm - Permalink

I'm not sure if you saw this topic or not, but it does have an example:

DisplayHelpTopic "Input/Output Queues"

I don't have a clear idea about what you're trying to do, since the example you gave doesn't run and doesn't make sense to me. For example, you're not actually using what would be the output waves of the worker function (the result of Duplicate) and your worker thread doesn't actually do any analysis.

I'm not sure that using the input/output queue is actually necessary in your situation.

September 12, 2019 at 04:48 pm - Permalink

function BreedTest()
 
 
  variable tgID=ThreadGroupCreate(4)
 
  string ListA = "indA0;indA1;indA2;indA3", ListB = "indB0;indB1;indB2;indb3"
 
 
  string loopstr1,loopstr2
  variable a,b, i
  
  for(i = 0;i < 4;i += 1)
   make /o/d/n=500 $StringFromList(i, ListA),$StringFromList(i, ListB) //create temp waves
   
   a=floor(20+enoise(20))
   b=floor(20+enoise(20))
   
   print a
   print b
   
   loopstr1="ind"+num2str(a)
   loopstr2="ind"+num2str(b)
    
   ThreadStart tgID,i, CrossoverThreadTest($StringFromList(i, ListA),$StringFromList(i, ListB), $loopstr1, $loopstr2)
  endfor
  
  // Wait for all threads to finish before starting another batch of threads
  do
    Variable threadGroupStatus = ThreadGroupWait(tgID,100)
  while (threadGroupStatus != 0)
  
end
 
threadsafe function CrossoverThreadTest(indA,indB, sourceA, sourceB)
  wave indA,indB //temp waves for each thread
  wave sourceA, sourceB
  
  Duplicate /o sourceA,indA
  Duplicate /o sourceB,indB
  
  
end

 

September 13, 2019 at 02:18 am - Permalink

Here is an example that I believe does pretty much the same thing as your original example, but uses the (arguably?) simpler MultiThread syntax. It also preserves the waves created by the Duplicate operation in the worker function.

 

function BreedTest()
 
    Make/O/D/N=40/WAVE indWaves // equivalent to ind0, ind1, etc.
    indWaves = NewFreeWave(4, 500)      // double precision waves, 500 points
 
    Make/O/D/N=(4,2)/WAVE indABWaves    // equivalent to indA0, indA1... in col 0 and indB0, indB1... in col 1
   
    Make/O/FREE/N=(4) junkWave  // Only needed to make it possible to use MultiThread instead of the more complicated ThreadGroup stuff
    MultiThread junkWave = CrossoverThreadTest(indWaves, indABWaves, p)
    
    // At this point, we don't need junkWave
    WaveClear junkWave
    
    // Do something with the waves in indABWaves
   
end
 
 ThreadSafe function CrossoverThreadTest(sourceWaveRefWave, destWaveRefWave, destRowNumber)
    WAVE/WAVE sourceWaveRefWave
    WAVE/WAVE destWaveRefWave
    Variable destRowNumber
   
    variable a,b
   
    a=floor(20+enoise(20))
    b=floor(20+enoise(20))
   
    WAVE loopWave1 = sourceWaveRefWave[a]
    WAVE loopWave2 = sourceWaveRefWave[b]
   
   // NOTE: If we don't use the /FREE flag with Duplicate below,
   // when this function is called in a preemptive thread the waves
   // created by Duplicate are automatically killed when the thread
   // returns. But if they are free waves, they aren't killed until
   // the last reference to the wave is lost. Since we store the
   // wave reference in destWaveRefWave, the wave isn't killed
   // when the thread returns.
    Duplicate /o/FREE loopWave1, indA
    destWaveRefWave[destRowNumber][0] = indA
    Duplicate /o/FREE loopWave2, indB
    destWaveRefWave[destRowNumber][1] = indB
 
end

 

September 13, 2019 at 09:10 am - Permalink