Single-precision numeric format option

ben_speight

Number-formatted line items and properties use a double-precision representation internally, and at least double precision during calculations. They are specified in the IEEE754 standard as a signed binary floating point format with 52 bit significand and 11-bit exponent, allowing a range of 2.2*10^-308 .. 1.8*10^308 and a precision of 15-17 significant decimal digits, and for integers up to an absolute magnitude of 9,007,199,254,740,992 to be exactly represented.

Amongst others, the IEEE754 standard also defines a single-precision representation which is similar but has a 23-bit significand and 8-bit exponent. This allows a range of 1.2*10^-38 .. 3.4*10^38 and precision of 6-9 significant decimal digits, and for integers up to an absolute magnitude of 16,777,216 to be exactly represented.

A single-precision number occupies four bytes of storage, compared to eight bytes for a double-precision number. For cases where only a limited range/precision is required, narrowing cell values can allow model builders to halve the footprint of a numeric line item or property, enabling more efficient use of workspace allocation, and for much larger numeric datasets to be manipulated.

Note that intermediate results within a formula that are not stored in a line item or property cell will continue to use (at least) double precision, as well as any calculations performed, regardless of whether or not they operate on or target single-precision data. Conversion would only be performed when a value is stored and/or retrieved.

The option would be provided as an additional radio box selection on the numeric format dialog. Transitions from single to double precision would preserve all data, and in the other direction preserve as much as the representation permits - values too large will be converted to an infinity of the same sign, whilst values too small will be converted to a zero of the same sign. "Not a Number" values (NaNs) will remain as such. To guard against data loss, transitioning an existing line item or property with detail data from double to single precision will elicit a warning prompt. Double precision will remain the default for new line items and properties.

 

Miran

Status changed to: New