Spelling suggestions: "subject:"floating point calues"" "subject:"floating point 5values""
1 |
CREATING FLOATING POINT VALUES IN MIL-STD-1750A 32 AND 48 BIT FORMATS: ISSUES AND ALGORITHMSMitchell, Jeffrey B. 10 1900 (has links)
International Telemetering Conference Proceedings / October 25-28, 1999 / Riviera Hotel and Convention Center, Las Vegas, Nevada / Experimentation with various routines that create floating point values in MIL-STD-1750A 32 and 48 bit formats has uncovered several flaws that result in loss of precision
in approximation and/or incorrect results. This paper will discuss approximation and key
computational conditions in the creation of values in these formats, and will describe
algorithms that create values correctly and to the closest possible approximation. Test
cases for determining behavior of routines of this type will also be supplied.
|
Page generated in 0.0673 seconds