evaluate goodness of fit for a set of data fit with a linear function compared to a power function

Share icon

Hi there,

i have a few sets data similar to the one below. i have fit the same set of data with both a linear function and a power function. i want to find out which of the two functions have a better goodness of fit. would this be possible with code already in Igor or would i have to write my own?

Blur level |Values | standard deviation

0 | 0 | 0.0777825
0.5 | 0.107972 | 0.0929795
1 | 0.325398 | 0.106902
2 | 0.309486 | 0.107013
4 | 0.610247 | 0.0993707

i have trying to work this out for far too long now and have hit the proverbial brick wall! so i would be very grateful for any help or advice anyone can give me. i have only really used Igor to fit functions to graphs for publication, i.e., have not programmed anything.

thank you very much.
Usually one would use the Reduced Chi-squared value as a measure of the goodness of fit - you can look this up on wikipedia, for example. This is the Chi-squared value (as calculated by Igor in doing the fits) divided by the number of degrees of freedom.

I have written some code to illustrate this. Paste the following into your procedure window.
Function MakeData()
    Make/O/N=(5) DataX
    DataX[]={0,0.5,1,2,4}
    Make/O/N=(5) DataY
    DataY[]={0,0.107972,0.325398,0.309486,0.610247}
    Make/O/N=(5) DataErr
    DataErr[]={0.0777825,0.0929795,0.106902,0.107013,0.0993707}
End
Function MakeGraph()
    Wave DataX,DataY,DataErr
    Display/N=MyGraph DataY vs DataX
    ModifyGraph mode=3,marker=1
    ErrorBars DataY Y,wave=(DataErr,DataErr)
End
Function DoLineFit()
    Wave DataX,DataY,DataErr
    CurveFit/NTHR=0/TBOX=768 line  DataY /X=DataX /W=DataErr /I=1 /D
    printf "Reduced Chi-sq (Line): %g",V_chisq/(DimSize(DataX,0)-2) // there are two fit parameters
End
Function DoPowerFit()
    Wave DataX,DataY,DataErr
    CurveFit/NTHR=0/TBOX=768 Power  DataY /X=DataX /W=DataErr /I=1 /D
    printf "Reduced Chi-sq (Power): %g",V_chisq/(DimSize(DataX,0)-3) // there are three fit parameters
End

Then in the command line run each function in turn.

I get the following results for the reduced Chi-squared:
Line: 0.68
Power: 0.64

One could tentatively say that the power curve is a better fit to this data. BUT, these values of the reduced Chi-squared are somewhat less than unity, indicating that you are overestimating the standard deviations in the data. Also the values are quite close to each other, so one (well me anyway) would be somewhat hesitant to draw any robust conclusion from this data. It is not surprising that a fit-function with more parameters is apparently a (slightly) better fit than the more simple function.
This coupled with looking at the data with the fit line superimposed, would suggest (to me) that there is no evidence or justification to say that the data is non-linear (Occam's razor!).

Hope this helps,
Regards,
Kurt

September 24, 2013 at 04:01 am - Permalink

Hi Kurt,

thanks for your swift response! i was secretly hoping that linear fits would be ok and the rest of the data suggest that they are.

with respect to the SDs, they are quite large but i really just need the slope values.

thanks so much for the help, i now have lots of faith in this forum!

kind regards

Akash

September 24, 2013 at 07:01 am - Permalink

And having computed the reduced chi-square for each of the fits, the ratio of the reduced chi-square values follows an F distribution. From that you can compute a probability that the power law fit is significantly better than the line fit. Read any good statistical text on the F test.

Oh- and be sure you understand that all this statistical mumbo jumbo assumes normal distributions and linearity. The power law fit is nonlinear, so the result is an approximation. You might also want to learn about the bootstrap technique, which is supported by the StatsResample operation in Igor. More work required, but statistically more meaningful.

John Weeks
WaveMetrics, Inc.
support@wavemetrics.com

September 24, 2013 at 09:27 am - Permalink