Question

I have a very strange issue with GZIPOutputStream when compressing an array of doubles. At the 57th element, I get a small discrepancy when I reload the data:

57 > 3.003727492141554E7 3.0037273913440887E7
1900: false
57 > -6.110783629228158E7 -6.110783629076558E7
2000: false

1900 and 2000 are two different set of doubles. The left value is the original value.

When I use a simple FileOutputStream without GZIP, I don’t get the issue. Why? Is GZIP output stream known to loose information?

EDIT

Here is how I read and write the data:

public static final double[] coefficients = new double[1161289];

...

public static void dump(File f) throws FileNotFoundException, IOException {

    OutputStream os = FileUtils.zipContent(f);

    byte[] ba = new byte[8];
    ByteBuffer BF = ByteBuffer.wrap(ba);
    BF.order(ByteOrder.BIG_ENDIAN);

    for (int i=0;i<coefficients.length;i++) {
        BF.putDouble(0, coefficients[i]);
        os.write(ba,0,8);
    }

    os.close();

}

public static void load(File f) throws FileNotFoundException, IOException {

    InputStream is = FileUtils.readZippedContent(f);

    byte[] ba = new byte[8];
    final ByteBuffer BF = ByteBuffer.wrap(ba);
    BF.order(ByteOrder.BIG_ENDIAN);

    for (int i=0;i<coefficients.length;i++) {
        is.read(ba,0,8);
        coefficients[i] = BF.getDouble(0);
    }

}

...

public static GZIPOutputStream zipContent(File f)
        throws FileNotFoundException, IOException {

    return new GZIPOutputStream(new FileOutputStream(f));

}

public static GZIPInputStream readZippedContent(File f)
        throws FileNotFoundException, IOException {

    return new GZIPInputStream(new FileInputStream(f));

}

Answer 1

gzip is lossless compression. any missing data would have to be accounted for by some other problem.

if the 57th element is the final element, I’d suspect failure to close the output file before re-opening for reading.