International Telemetering Conference Proceedings / October 25-28, 1999 / Riviera Hotel and Convention Center, Las Vegas, Nevada / Experimentation with various routines that create floating point values in MIL-STD-1750A 32 and 48 bit formats has uncovered several flaws that result in loss of precision
in approximation and/or incorrect results. This paper will discuss approximation and key
computational conditions in the creation of values in these formats, and will describe
algorithms that create values correctly and to the closest possible approximation. Test
cases for determining behavior of routines of this type will also be supplied.
| Identifer | oai:union.ndltd.org:arizona.edu/oai:arizona.openrepository.com:10150/607271 |
| Date | 10 1900 |
| Creators | Mitchell, Jeffrey B. |
| Contributors | L3 Communications |
| Publisher | International Foundation for Telemetering |
| Source Sets | University of Arizona |
| Language | en_US |
| Detected Language | English |
| Type | text, Proceedings |
| Rights | Copyright © International Foundation for Telemetering |
| Relation | http://www.telemetry.org/ |
Page generated in 0.0018 seconds