Glitch in FindValue Function

Share icon

FallAmpl = {1,0.9,0.8,0.7,0.6,0.5,0.4,0.3,0.2,0.1}
template_fall = {0.9,0.8,0.7,0.6,0.5,0.4,0.3,0.2,0.1}
However the findvalue returns V_value = -1 in the case where FallAmpl[4] = 0.6, eventhough template_fall[3] = 0.6. What is the reason for this glitch?
Decimal fractions cannot be exactly represented as floating-point numbers. You need to use the /T flag to establish a tolerance for your search. To see what's happening, execute this command:

Print/D FallAmp[4], template_fall[3]

Read more than you ever wanted to know about about floating-point truncation:
http://en.wikipedia.org/wiki/Floating_point

It's an important topic; anyone doing computation on a computer needs to be aware of the issue.

John Weeks
WaveMetrics, Inc.
support@wavemetrics.com

June 11, 2014 at 09:15 am - Permalink