Questions about multithreaded loops and FuncFits

The FuncFit operation is threadsafe- is that what you're asking? The W_coef and W_sigma waves (and any others that might be created) will be created in the context of the thread that runs FuncFit. Those waves will be lost when the thread terminates unless you do something to pack them up and send them back to the main thread. That something might involve ThreadGroupPutDF and ThreadGroupGetDF(R).

If your fitting function is already threadsafe and your data set is big enough, Igor will use threads for a portion of the fitting process. If FuncFit runs in a preemptive thread, the automatic within-FuncFit threading is disabled to avoid contention for threads, which can seriously degrade performance.

Having said that, if you have many fits to perform, running each fit in a thread will get  you a better speed-up than the automatic threading because more of the fitting process will run in parallel.

I tried to split the loop to the individual cores.

The f_test function passes certain areas of the matrix to each thread.

The data for the fit is then created in the temporary directory for each thread. Afterwards the fit is performed and the parameters are entered into the globally available wave.

Here is an example of the fit function:

But when I look at the results and compared it with the single thread fit, I get different results. I can see the six threads in the result - it looks like that they have different offsets and I get this error message:

Did I do something fundamentally wrong or is the problem that all threads want to access the same waves while fitting?

One thing I see is the use of constraints. Passing a text constraint wave to FuncFit in a thread is not supported (I probably should have mentioned that previously). The constraint code uses Igor's interpreter to parse the constraint expressions, and the interpreter is not threadsafe. For the work-around, DisplayHelpTopic "Constraint Matrix and Vector" and DisplayHelpTopic "Constraints and ThreadSafe Functions".

Perhaps being naive but ... Would it be feasible and beneficial instead to split your matrix into as many "beam chunks" as you have cores and passed those "beam chunks through a threadsafe fitting routine? IOW, try to optimize the spread of the data fitting in the two loops across all cores rather than putting all the thread safe load in the fitting function.

For example, given a hypothetical 10 core system using a 10 x 10 x C matrix, create 10 sub-matrices (e.g. split to ten 10 x 1 x C matrices) that are then each sent off to their own individual core for fitting.

Just a few things that I noticed in your f_test2 function :

None of these waves are threadsafe because they are all located in the current datafolder. Each thread will therefore overwite the waves of the other threads. A simple fix would be to create free waves instead.

That way the waves becomes unique to each thread and they will no longer overwrite each other.

A free waves is like a local variable. It only exists within the function that created it. A normal wave is like a global variable. It has a location in Igor datafolder structure and is shared between all functions.

There may be other issues with your code, but the shared waves was what caught my eye.

olelytken wrote:

Just a few things that I noticed in your f_test2 function :

Make/D/O/N=2/O W_coef                      
Make/D/O/N=(1,2) M_FitConstraint=0
Make/D/O/N=1 W_FitConstraint=0
Make/D/N=2/O W_sigma
make/N=(v_layers_max)/O tempwave
MatrixOP/O tempwave=beam(w_matrix,row_index,column_index)

None of these waves are threadsafe because they are all located in the current datafolder. Each thread will therefore overwite the waves of the other threads. A simple fix would be to create free waves instead.

That's actually not true. From this help topic:

"When threadsafe functions execute in the main thread, they have normal access to data folders, waves, and variables. But when running in a preemptive thread, threadsafe functions use their own private data folders, waves, and variables."

So in a preemptive thread, those waves are created in a thread local data folder. This makes them more or less the same as a free wave. But I agree with your recommendation to explicitly create them as free waves, because I think it makes the code a little more explanatory and it ensures that the behavior when run in a preemptive thread is the same as when run from the main thread.

Many thanks for the suggestions.

@johnweeks: I had already adjusted the constraints using this help topics. I created the necessary waves for it. Without the threadsafe Waves you can't compile the code at all, because an error message will be displayed.

@jjweimer: My attempt is to assign matrix sections to a thread. (e.g. row 0 to 9 -> core 1, row 10 to 19 -> core 2 ... results -> one wave for all cores).

@olelytken & aclight: I have implemented the /FREE flag.

Additionally I tried to duplicate the reference wave into each thread and then fit it.

Then I rewrote the fit function to take the wave0 from the current datafolder. Do I correctly assume that this datafolder is the temporary folder from the thread that calls the FuncFit? (With the normal fit via the menu the fit works).

I could solve the offset / stripe pattern by recreating the w_coef wave before each fit and not using the results of the last fit as start value:

Unfortunately I have the problem that the wave W_sigma is not read out at all and the Wave W_coef is apparently not accessible properly. (Creation of the wave w_sigma and the readout was inserted of course.)

> My attempt is to assign matrix sections to a thread. (e.g. row 0 to 9 -> core 1, row 10 to 19 -> core 2 ... results -> one wave for all cores).

I would propose to use Duplicate/FREE to split the matrix and then assign the newly created FREE sub matrices each to their own core. The trade off is to increase the RAM needed for the code with the hope that the speed increase in running the code compensates.

I have tested the function which is to be processed by several threads in parallel with single inputs (certain data and one processor core). If this function is not threadsafe, it works as aspected (uses core #1 for beam and multiple threads for the fitting). As soon as this function is threadsafe by the command threadsafe, this function behaves strange:
Although I specified the flag /Q, the result (of each fit) is shown in a separate window. The following text is written to the history:

Are the results possibly saved differently when threadsafe is active compared to threadsafe is not active?

Is it possible that the thread's temporary folder is not output as string with the

command?

(W_coef and W_sigma seems to be used not correctly with make/FREE. But since the waves are temporary anyway, that's not too bad.)

Should splitting into multiple threads work for my problem? I have inserted the creation of a difference of two points on the "tempwave" (its generated using beam) and this is done without errors by splitting different regions of rows to several threads. Only the fit seems not to work, although it works without ThreadSafe.

jjweimer wrote:

I would propose to use Duplicate/FREE to split the matrix and then assign the newly created FREE sub matrices each to their own core. The trade off is to increase the RAM needed for the code with the hope that the speed increase in running the code compensates.

This is a good idea. The Waves are not so large that there could be problems with the RAM.

Edit: Code for generation of an exampe:

f_generation_matrix creates the necessary waves. f_test2 is the function that is not threadsafe (in this case v_row_per_thread =v_row_max). f_test is the function to start the evaluation over all threads. This does not work because the threadsafe function f_test3 (f_test3 is a copy of f_test2) does not work.

The result of the fit is correct if w_export1=tempwave2 and w_export2=tempwave3.

Sorry about my earlier error with respect to your use of the constraint waves. Old eyes, I guess:)

Maybe you could provide short walk-through of the code? I don't see how the waves reference1 and reference2 are transmitted to a FuncFit running in a thread.

In case John's last comment isn't clear, your problem is here:

This is not going to work when it is called from a preemptive thread. As I quoted above:

"When threadsafe functions execute in the main thread, they have normal access to data folders, waves, and variables. But when running in a preemptive thread, threadsafe functions use their own private data folders, waves, and variables."

I think the wording of the statement is a bit vague, but that statement means that functions called in a preemptive thread do not have access to global waves or variables. So you cannot look up w_refwave1 and w_refwave2 since those are global waves.

If you modify your function a bit, like this, and then call your test function that uses threads for the fit, you'll see plenty of messages printed out:

Edit2: Ok, thank you very much. It works. :)

I have updated the source code below so that others who will have the same problem in the future will find the solution better.

@johnweeks: In this case the compilation is very helpful, because it complains about a wrong contraints wave.

I have added notes for the tasks of each section.

But in itself the procedure would be as follows:
1) f_generation_matrix()

2) ThreadSafe: f_test(test_matrix)
or
3a) for the variant that is not threadsafe, create the w_export waves: 

3b) f_test2(test_matrix2,w_export1,w_export2,w_export3,w_export4,w_export5,40,40,800,40,0)