Issues with ZERO on SetVariable Controls
jjweimer
I have this ongoing problem with SetVariable controls in panels. Perhaps I am missing an easy answer or one that has already been posted.
I have SetVariable controls that are designed to step between -N to N in increments. The increments should go identically through 0 (zero). Invariably, as I step through the values, the ZERO becomes something akin to 2.7 x 10^(-17) rather than EXACTLY ZERO.
What am I missing here?
My guess is that you're running into floating point precision errors, depending on what N is and what your increment is. See https://www.itu.dk/~sestoft/bachelor/IEEE754_article.pdf for full details
September 24, 2018 at 09:17 am - Permalink
OK. I would think that a range -2 to 2 with increments 0.1 would not have this problem. Here is the setvariable control extracted from the panel. The value _should_ be exactly zero. It increments up/down to exact values above/below 0.1 or -0.1 with no issues.
Here is the value with a format code in the panel display. The value shows 0.0 but is not set at zero exactly.
SetVariable setvarA,limits={-2,2,0.1},value= _NUM:2.77555756156289e-17
Here is the statement with an increment at 0.1.
When I update to 0 manually, the value is _NUM:0 (exactly).
The precision offset is not critical in my calculations. But it could be for someone. Should there be a standard way to avoid this? Something akin to a checkbox: [ ] round number values to format precision.
Igor Pro 7.08 on macOS 10.13.6
September 24, 2018 at 03:31 pm - Permalink
0.1 has no exact representation in ANY IEEE floating point format: it is a repeating bicimal:
https://www.exploringbinary.com/why-0-point-1-does-not-exist-in-floatin…
You should never use an increment like this and expect to get exactly 0 after repeated increments and decrements.
Use rounding before using the value:
Variable val= round(inputVal*10)/10
September 24, 2018 at 04:33 pm - Permalink
Well! I learned something that I probably did know vaguely but never really had hit home properly.
I have a feature request in the works.
September 24, 2018 at 04:46 pm - Permalink