Differentiate - irregular spaced data
Mike German
Just a quick question - I have two 1D waves, with measurements (say distance and time) at irregular times. Does the differentiate function operate on irregular spaced data such as this. I would use cetral differencing.
While I am not in the position to answer your question with absolute confidence, I would say, yes, you can do that. Just select an x-wave in the Differentiate menu dialogue or use the /X= flag with the Differentiate function. Central differencing is the default here and does not need further options.
January 23, 2022 at 04:40 am - Permalink
In reply to While I am not in the… by chozo
Thanks Chozo
I have found from my experiments, the Differentiate operation doe not work with irregularly spaced points, unless I am doing something wrong. I used
Differentiate /METH=0 myVariable /X=myDateTime /D=myDiffferentiatedVariable
In the plot blue points are wave to be differentiated (left axis) plotted against Date/Time and red is differentiated wave.
February 2, 2022 at 12:53 am - Permalink
In reply to Thanks Chozo I have found… by Mike German
February 2, 2022 at 12:56 am - Permalink
I see. Could you maybe provide some example experiment file, so that we can test things? I don't immediately see what might be wrong here, but I could imagine that Differentiate may have problems with Date/Time values. Or this is just the way it looks with just a few points to work with...
February 2, 2022 at 01:14 am - Permalink
looks about right to me.
this
Differentiate /METH=0 myVariable /X=myDateTime /D=myDiffferentiatedVariable
is the same as
myDiffferentiatedVariable = (myVariable[p+1] - myVariable[p-1]) / (myDateTime[p+1] - myDateTime[p-1])
It can work only within the precision of the data :)
Edit: looks about right, except that the magnitude and sign of the differentiated data in your plot don't tally with the data. Is there some additional scale factor, like -1000?
February 2, 2022 at 02:54 am - Permalink
In reply to I see. Could you maybe… by chozo
Sample data in experiment attached
February 2, 2022 at 03:14 am - Permalink
In reply to looks about right to me. … by tony
Tony,
well spotted - see differentiate.pxp which contains code.
Perhaps the central diffencing is doing what it is supposed to do - I was expecting the differentiated curve to be smoother like the original.
February 2, 2022 at 03:17 am - Permalink
As tony already mentioned, the large jitteriness in the derivative most probably comes from the slight variations in your input data. You could cheat a bit here by smoothing out these variations if this is can be justified. Your data seems to follow a very smooth trend which can be very closely approximated with a polynomial fit. This gives you a very smooth derivative (see the attachment). Of course you would need to think about what errors you are introducing by this approach. But if you just need the trend (slope) of the derivative or something, then this may work.
February 2, 2022 at 03:38 am - Permalink
In reply to As tony already mentioned,… by chozo
As you say chozo, a very good fit to the data and smooth diffentiated wave. This may well be fit for purpose :-)
I should get a decent value for the slope of the Doppler2 which is better, I think, than smoothing. These data are one set of many so I need to see if this works in general
CurveFit poly_XOffset 3, mySRangeTot /X=myUTD /D
Out of interest at the command prompt you used
ReplaceWave /X trace=Doppler2, $""
How does this work? i.e. the $""
February 2, 2022 at 04:49 am - Permalink
Good that this seems to be an option for now. Yes, the fit should work in general. You could also have the fit result as a transient wave (either delete or have a free wave) just for the differentiation if you don't want to clutter up your file.
The ReplaceWave command just got rid of the x-wave in the graph, since Dopper2 has its own (linear) scaling from the fit and does not rely on UTD. The $"" part is an empty assignment for the x-wave which effectively removes the dependence on UTD.
Many people actually struggle with understanding the $ operator. It is basically a way to convert a bunch of characters in to an object reference. You can try to read this to get a grasp on it:
DisplayHelpTopic "String Substitution Using $"
February 2, 2022 at 06:15 am - Permalink
In reply to Good that this seems to be… by chozo
Many thanks Chozo and Tony for finding a practical approach to my differention problem. The purpose of the differentiation of my measured data was two-fold; to find the slope of the curve and to see whether it passed through zero. The fit approach does provide this. In general the measured data pass slowly through a minimum value and hence provides more points and weighting to the fit. The V_Chisq value provides a useful indication of any problems with measured data.
February 7, 2022 at 04:27 am - Permalink