iOS Question Reading data from a table with FLOAT datatype fields

[Alex_197]

Hi all
I have a table in my database with fields datatype FLOAT.

I'm reading values from these fields like this

Dim rs as ResultSet
Mytxt.Text=rs.GetDouble("BP1")

MyTxt is B4XView

If the value in this field is lower than 90 MyTxt.text shows whatever we have in this field. For example 45.3 looks like 45.3 but if it's 96.6 it looks like 96.599999999

Why it's like that?

 

[Erel]

Never display non-integer numbers without NumberFormat.

 

[emexes]

If the value in this field is lower than 90 MyTxt.text shows whatever we have in this field. For example 45.3 looks like 45.3 but if it's 96.6 it looks like 96.599999999
Why it's like that?

It happens because the 53 bit mantissa (precision/resolution) of (double) FLOAT is log10(2^53) decimal digits ie 15.95 significant digits, and so for randomish values shown to 16 significant digits, some of them are going to be out by +/- 1 least-significant digit column value.

If default conversion to decimal was capped at a (safe) 15 significant digits rather than the (optimistic) 16 significant digits, then the decimal representation problem would go away... or, at least, not be visible... although you could still occasionally have a similar issue with such errors accumulating through multiple operations.

@Erel's advice is also in that direction - use NumberFormat to limit the significant digits displayed to 15 or fewer (for Double) or to 6 or fewer (for Single Float).