Is it possible to use recursive expression in user-defined fitting function?

I am curious whether it would cause trouble to use recursive expression in the fitting function. For example,

Function fitting(Coefs, x) : FitFunc

Wave Coefs
Variable x

Variable y

Wave temp = fit_data

y = Coefs[0] + Coefs[1]*x^2 + temp[x-1]

return y

End

FuncFit fitting, Coefs, data, /X=x_data /D
That function won't work because fit_data isn't created until the end of the iteration.

What you need is an all-at-once function, which allows you to compute all the model values at once. Read about them by executing this command on Igor's command line:

DisplayHelpTopic "All-At-Once Fitting Functions"

The function will look something like this:

Function fitting(coefs, yw, xw) : fitfunc
    Wave coefs, yw, xw

    yw[0] = coefs[0] + coefs[1]*xw[0]^2
    yw[1,] = coefs[0] + coefs[1]*xw[p]^2 + yw[p-1]
end

Note how I made a special case of yw[0] so that you don't try to access yw[-1].

John Weeks
WaveMetrics, Inc.
support@wavemetrics.com
Thanks a lot! John.

The trick for the boundary treatment is especially impressive. I once considered to put a control structure in the function to achieve the same thing but hesitated to implement for possible performance penalty.
I have a further question to use the All-at-once fitting function. Is it correct to fit the function by FuncFit operation with the syntax:

FuncFit fitting, coefs, data /D=yw /X=xw

where coefs stands for the coefficient wave, yw is the wave created in the working data folder to store x and y values and xw and data are the data for x and y axis?
John,
I'm guessing that your fit function may or may not be threadsafe, depending on funcfit splits calculations between threads.

YHLien wrote:
I have a further question to use the All-at-once fitting function. Is it correct to fit the function by FuncFit operation with the syntax:

FuncFit fitting, coefs, data /D=yw /X=xw

where coefs stands for the coefficient wave, yw is the wave created in the working data folder to store x and y values and xw and data are the data for x and y axis?

Well...
The syntax you show is correct, but the names you used suggest you may have a misconception.

xw and yw in the parameter list for the all-at-once function are names that stand in for temporary waves created by funcfit at run-time. They have the values of your input Y data and X data.

The /D flag is used to designate a wave to receive model values corresponding to the solution found by funcfit.

In your command line the wave coefs will (usually) be passed directly to the the fitting function via the pw input parameter.

John Weeks
WaveMetrics, Inc.
support@wavemetrics.com
andyfaff wrote:
John,
I'm guessing that your fit function may or may not be threadsafe, depending on funcfit splits calculations between threads.

The function itself, if marked with the Threadsafe keyword, will be threadsafe. What you're probably talking about is the fact that all-at-once fit functions do not use multiple threads within FuncFit, whereas ordinary user fit functions do.

If you mark your all-at-once fit function as threadsafe, then you can call FuncFit from an Igor preemptive thread.

John Weeks
WaveMetrics, Inc.
support@wavemetrics.com
johnweeks wrote:

Function fitting(coefs, yw, xw) : fitfunc
    Wave coefs, yw, xw

    yw[0] = coefs[0] + coefs[1]*xw[0]^2
    yw[1,] = coefs[0] + coefs[1]*xw[p]^2 + yw[p-1]
end



What I was thinking about was the funcfit could still multithread the all-at-once function if it sent half the points to one thread and half the points to the other. At which point the thing would break because you're doing a yw[p-1] assignment (the next point depends on the previous point.
johnweeks wrote:
YHLien wrote:
I have a further question to use the All-at-once fitting function. Is it correct to fit the function by FuncFit operation with the syntax:

FuncFit fitting, coefs, data /D=yw /X=xw

where coefs stands for the coefficient wave, yw is the wave created in the working data folder to store x and y values and xw and data are the data for x and y axis?

Well...
The syntax you show is correct, but the names you used suggest you may have a misconception.

xw and yw in the parameter list for the all-at-once function are names that stand in for temporary waves created by funcfit at run-time. They have the values of your input Y data and X data.

The /D flag is used to designate a wave to receive model values corresponding to the solution found by funcfit.

In your command line the wave coefs will (usually) be passed directly to the the fitting function via the pw input parameter.

John Weeks
WaveMetrics, Inc.
support@wavemetrics.com


Did you mean the correct syntax for the issue should be:

   FuncFit fitting, coefs, data /D /X=data_x

in which the original data composes the wave "data" for y axis and the wave "data_x" for x axis, and the wave yw and xw will be automatically created by FuncFit operation

I also have another question for curiosity. Did the algorithm of FuncFit determine the fitting function types, i.e., basic, multivariate and all-at-once by checking the pattern of parameter list? For example, the parameter list of basic type is (pw, x), the parameter list of multivariate is (pw, x1, x2....) and the parameter list of all-at-once would be (pw, yw, xw).

I am also wondering whether it is possible to combine to multivariate and all-at-once to have the fitting function like

    Function fitting(pw, w_y, w_x1, w_x2...)

        Wave pw, w_y, w_x1, w_x2

        (expression of the practical function)

    End

andyfaff wrote:
johnweeks wrote:

Function fitting(coefs, yw, xw) : fitfunc
    Wave coefs, yw, xw

    yw[0] = coefs[0] + coefs[1]*xw[0]^2
    yw[1,] = coefs[0] + coefs[1]*xw[p]^2 + yw[p-1]
end



What I was thinking about was the funcfit could still multithread the all-at-once function if it sent half the points to one thread and half the points to the other. At which point the thing would break because you're doing a yw[p-1] assignment (the next point depends on the previous point.

Ah, yes. You're right about that. And that is just one of the many things you can do in a user-defined function, and one of the reasons that all-at-once functions still aren't threaded. Plain user-defined fit functions are threaded, and they do it by sending the various models required for an iteration to different threads. That way, each thread computes all the points for a given model, so it's OK for the function to access stuff out of order.

Since the derivatives require a model for each coefficient (with each model based on having one of the coefficients slightly perturbed) each model is computed by a different thread. That, in turn, means that you get at most N+1 threads, where N is the number of coefficients.

In addition, since the derivatives require access to the unperturbed model, everything has to wait on a mutex that is freed when the thread doing the unperturbed model finishes before it goes on to compute the derivatives. The whole thing is so complex and took so long to get right that I haven't had the courage to tackle threading all-at-once functions.

John Weeks
WaveMetrics, Inc.
support@wavemetrics.com
YHLien wrote:
Did you mean the correct syntax for the issue should be:

   FuncFit fitting, coefs, data /D /X=data_x

in which the original data composes the wave "data" for y axis and the wave "data_x" for x axis, and the wave yw and xw will be automatically created by FuncFit operation

Yes, that's what I meant. If you use /D alone, the fit_ wave will have by default 200 evenly-spaced points. That may not work well with your fitting function, since each point depends on previous points. If you do this, be sure to read the part of the all-at-once fit function documentation that starts with "Some things to be aware of with regard to this function". Be sure you understand the more complicated example function.
Quote:
I also have another question for curiosity. Did the algorithm of FuncFit determine the fitting function types, i.e., basic, multivariate and all-at-once by checking the pattern of parameter list? For example, the parameter list of basic type is (pw, x), the parameter list of multivariate is (pw, x1, x2....) and the parameter list of all-at-once would be (pw, yw, xw).

Yes, that's correct. The ": fitfunc" after the function declaration makes it possible to eliminate non-fitting functions that just happen to have a parameter list the matches a fitting function.
Quote:
I am also wondering whether it is possible to combine to multivariate and all-at-once to have the fitting function like

    Function fitting(pw, w_y, w_x1, w_x2...)

        Wave pw, w_y, w_x1, w_x2

        (expression of the practical function)

    End

Yes, that is correct. I have still forgotten to add this to the documentation after several years! I will do that today.

John Weeks
WaveMetrics, Inc.
support@wavemetrics.com