Fast calculation of linear correlations
Peeyush Khare
Hello,
I have to calculate linear correlation coefficients for a 2D matrix with large dimensions (e.g. 30000 rows and 2000 or more columns). This can take a while to run and I have a couple such matrices . I am wondering if there is a faster way to run correlations in IGOR. Please advise.
Best,
Peeyush
I don't fully understand your problem; when you say "correlation coefficients" of a matrix do you mean covariance?
If so,
DisplayHelpTopic "MatrixOP"and look under Functions for covariance(w).
August 14, 2024 at 09:28 am - Permalink
Hi s.r.chinn,
Yep, I mean Pearson correlation coefficients to check how well the individual columns linearly correlate. Thanks for the tips!
Best,
Peeyush
August 15, 2024 at 02:25 pm - Permalink
I have a form of Pearson correlation in my Image Profile tools to align rows or columns. I use Correlate/AUTO for that, which seems to be reasonably fast. I also implemented fitting to determine substep offsets. Here is my code from that tool. in1 and in2 at the two columns to compare (I loop over all columns to compare against the first):
Duplicate/Free in2, work
Correlate/AUTO in1, work
WaveStats/Q work
variable error = 1, result = 0
variable optLoc = sub ? V_maxLoc : (V_maxRowLoc - x2pnt(work, 0)) // if sub-shift is active, return in scaled wave units
if (V_sum != 0)
if (V_maxRowLoc >= 3 && sub)
try
CurveFit/M=0/Q gauss, work[V_maxRowLoc-3,V_maxRowLoc+3]; AbortOnRTE
wave W_coef
if (numtype(W_coef[2]) == 0 && W_coef[2] > optLoc-3 && W_Coef[2] < optLoc+3)
result = W_coef[2]
error = 0
endif
catch
if (V_AbortCode == -4)
variable cfError = GetRTError(1)
endif
endtry
endif
if (error)
result = optLoc
endif
endif
return result
end
August 15, 2024 at 08:23 pm - Permalink