Consistent mapping from special double values to integers

Share icon

The guy sitting in front of my computer introduced a bug due to passing a NaN where an integer should have been.
And that NaN mapped to a -1 which was kind of hard to track down.

It would be nice to have a fixed way of converting from special double values to integers.
Currently that seems a bit unspecified.

Function DoStuff()

    int a
    int64 b
    uint64 c

    print "Mapping NaN to int"

    a = NaN
    b = NaN
    c = NaN

    printf "%x, ,%x, %x\r", a, b, c

    Make/FREE/B a1 = {NaN}
    Make/FREE/B/U a2 = {NaN}
    Make/FREE/W b1 = {NaN}
    Make/FREE/W/U b2 = {NaN}
    Make/FREE/I c1 = {NaN}
    Make/FREE/I/U c2 = {NaN}
    Make/FREE/L d1 = {NaN}
    Make/FREE/L/U d2 = {NaN}

    print/D a1[0], a2[0], b1[0], b2[0], c1[0], c2[0], d1[0], d2[0]

    print "Mapping inf to int"

    a = Inf
    b = Inf
    c = Inf

    printf "%x, ,%x, %x\r", a, b, c

    Make/FREE/B a1 = {Inf}
    Make/FREE/B/U a2 = {Inf}
    Make/FREE/W b1 = {Inf}
    Make/FREE/W/U b2 = {Inf}
    Make/FREE/I c1 = {Inf}
    Make/FREE/I/U c2 = {Inf}
    Make/FREE/L d1 = {Inf}
    Make/FREE/L/U d2 = {Inf}

    print/D a1[0], a2[0], b1[0], b2[0], c1[0], c2[0], d1[0], d2[0]
   
    print "Mapping -inf to int"

    a = -inf
    b = -inf
    c = -inf

    printf "%x, ,%x, %x\r", a, b, c

    Make/FREE/B a1 = {-inf}
    Make/FREE/B/U a2 = {-inf}
    Make/FREE/W b1 = {-inf}
    Make/FREE/W/U b2 = {-inf}
    Make/FREE/I c1 = {-inf}
    Make/FREE/I/U c2 = {-inf}
    Make/FREE/L d1 = {-inf}
    Make/FREE/L/U d2 = {-inf}

    print/D a1[0], a2[0], b1[0], b2[0], c1[0], c2[0], d1[0], d2[0]
End


•dostuff() Mapping NaN to int 8000000000000000, ,8000000000000000, 8000000000000000 -1 255 -1 65535 -1 4294967295 -9223372036854775808 -9223372036854775808 Mapping inf to int 8000000000000000, ,8000000000000000, 8000000000000000 -1 255 -1 65535 -1 4294967295 -9223372036854775808 -9223372036854775808 Mapping -inf to int 8000000000000000, ,8000000000000000, 8000000000000000 0 0 0 0 0 0 -9223372036854775808 -9223372036854775808

IMHO the best solution would be to throw a RTE with rtFunctionErrors=1 but I would expect that this is too costly performance-wise.