Question

I’m importing a text file containing a field with numbers up to the thousandth decimal as a number field. I’m then running and update query to fix the SSNs (preceding zeros) and subtract 500 from the number field.

UPDATE CODImportFile 
SET CODImportFile.[Original SSN] = Format([Original SSN],"000000000")
,   CODImportFile.[Lifetime Eligibility Used] = [Lifetime Eligibility Used]-500;

I have no clue why, but some of the results look like this:

Original:500.016
Result: 1.60000000000196E-02 

Any idea what could be causing this?

Answer 1

Since the data type of [Lifetime Eligibility Used] is double, the calculation in your query is similar to this:

SELECT
    CDbl(500.016) - 500 AS difference,
    TypeName(CDbl(500.016)-500) AS data_type;

… which returns this:

difference           data_type
1.60000000000196E-02 Double

Those unexpected decimal places are an artifact of the imprecision inherent with decimal math on a binary system. You could convert the field type to single (or cast double to single in the calculation), but then you would still get “extra” decimal places … just a different set of values for those decimal places.

I think you should discard any decimal places you don’t want to store.

UPDATE CODImportFile
SET [Lifetime Eligibility Used] =
    Round([Lifetime Eligibility Used]-500, 3);

Then you only need decide which rounding method you want.

? Round(0.1245, 3)
 0.124 

? CDbl(Format(0.1245, "0.000"))
 0.125