Issues in multi-threading

Share icon

Hi everyone. I am trying to speed up my code using multi-threading the loops, but the generated data show artifacts for more than one thread. Please see the attached image with the 2d waves generated using 1, 3 and 8 threads. I have also attached the part of my code involving multi threading. I pass many variables and waves to the worker function as parameters. Has anyone faced similar issues or may be I am doing stupid mistakes somewhere. I really appreciate any help. Thank you.

 

​
//Creat the tread group for parallel processing
threadGroupID=ThreadGroupCreate(nthreads)
for(it=0; it<NBand; it+=1) // different bands
    i=0 // k points        
    do
        // Thread-at-a-time
        threadIndex = ThreadGroupWait(threadGroupID,-2) - 1
        if (threadIndex < 0)
            DummyThread = ThreadGroupWait(threadGroupID, 10)    // Let threads run a while
            continue                // No free threads yet
        else
            ThreadStart threadGroupID, threadIndex, calEDC(Im,N, i,it,2,mu, kt, spacecharge, A, Width, Bg, IndBand, FDCut, E_raw, BCSse, ME_c, ME_e, ME_e2, ME_k, ME_k2, Option, W_c, W_e, W_k, generalparameters)
            i+=1
        endif
    while(i<=numkpnts-1)
    // Wait for all threads to finish only for thread-at-a-time
    do
        Variable threadGroupStatus = ThreadGroupWait(threadGroupID,10)
    while(threadGroupStatus != 0)
       
endfor //band loop
       
// Delete the thread group
dummyThread = ThreadGroupRelease(threadGroupID) //Delete the thread group

​

 

 2D waves calculated using 1, 3 and 8 threads.

Are you sure a thread does not overwrite the data from other threads? How have you splitted your task into multiple threads?

I'm using threads a lot in Igor Pro and they are pretty robust.

October 7, 2019 at 01:40 am - Permalink

To expand on what Thomas said, you are passing a *lot* of parameters to your thread groups:

ThreadStart threadGroupID, threadIndex, calEDC(Im,N, i,it,2,mu, kt, spacecharge, A, Width, Bg, IndBand, FDCut, E_raw, BCSse, ME_c, ME_e, ME_e2, ME_k, ME_k2, Option, W_c, W_e, W_k, generalparameters)

Since you didn't post the calEDC function it's hard to say what might be going wrong, but my guess is that calEDC modifies the value of one or more waves that are included in this parameter list. While that is safe to do from the perspective that it won't cause Igor to crash, it's usually not safe to do if you want the right answer.

Typically you want to partition your task into threads in such a way that the result of the calculation of any single thread is not needed by any other thread.

October 7, 2019 at 08:29 am - Permalink

I am passing all the parameters and waves to the worker function calEDC. Also the final output 2D wave indBand. Then each thread calculates separate columns of the output wave.

October 7, 2019 at 10:11 am - Permalink

The threads are independent in a sense that they doesn't depend on each other. Only wave they write to -- different columns of indBand. Rest of the parameters are the inputs. Most of them are same for all the threads. Parameter i and it are the column and the layers of indBand where that particular thread should write the output.

I will try to see if partitioning the output wave for each threads and combine them in the main programme helps to get proper result.

October 7, 2019 at 10:24 am - Permalink

Select an example which runs fast and is small. I would then add some debug output into the worker function which prints the indizes it uses for writing into the output wave. And then have a look if they overlap.

The size of the output wave has to be constant as well.

October 7, 2019 at 10:28 am - Permalink

It working now. Looks like in one of the parameter I was writing from all the threads. Thanks for the advice.

October 7, 2019 at 12:19 pm - Permalink

Forum

Support

Gallery

Igor Pro 9

Learn More

Igor XOP Toolkit

Learn More

Igor NIDAQ Tools MX

Learn More