Question

so we’re on the same page: i’m using NSData+GZIP, JSONKit, and NSData+Base64

my goal here, as the title suggests, is to take a json object, compress it, base64 encode it, and send it up to a .net server as a form post which in turn expects to take this data and reverse the process and end with a json object.

i need to do this on both android and iOS. android is done, it works well. for reference, here is what i’m doing there:

public String compress(String string) throws IOException {
    byte[] blockcopy = ByteBuffer.allocate(4).order(java.nio.ByteOrder.LITTLE_ENDIAN).putInt(string.length()).array();
    ByteArrayOutputStream os = new ByteArrayOutputStream(string.length());
    GZIPOutputStream gos = new GZIPOutputStream(os);
    gos.write(string.getBytes());
    gos.close();
    os.close();
    byte[] compressed = new byte[4 + os.toByteArray().length];
    System.arraycopy(blockcopy, 0, compressed, 0, 4);
    System.arraycopy(os.toByteArray(), 0, compressed, 4, os.toByteArray().length);

    StringBuffer hexString = new StringBuffer();
    for (int i=0; i<compressed.length; i++)
        hexString.append(Integer.toHexString(0xFF & compressed[i]));

    return Base64.encodeBytes(compressed);
}

here is a test on iOS that succeeds:

    NSLog(@"test? %@", [[NSString alloc]initWithData:[[NSData dataFromBase64String:[[[@"test" dataUsingEncoding:NSUTF8StringEncoding] gzippedData] base64EncodedString]] gunzippedData] encoding:NSUTF8StringEncoding]);

this prints:

"test? test"

the area of concern:

    NSStringEncoding encoding = NSUTF8StringEncoding;

    NSString* string = [[NSArray arrayWithArray:mutableData] JSONString];
    NSData* data = [string dataUsingEncoding:encoding];

    NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL: [NSURL URLWithString:[NSString stringWithFormat:@"http://www.google.com/myendpoint.aspx?deviceid=%@&accesstoken=%@", [defaults objectForKey:@"deviceId"], [defaults objectForKey:@"ggAccessToken"]]]];
    [request setHTTPMethod:@"POST"];
    NSMutableData *requestBodyData = nil;

    requestBodyData = [NSMutableData data];

    // Create boundry
    NSString *boundary = [NSString stringWithFormat:@"--%@--", [[NSProcessInfo processInfo] globallyUniqueString]];
    [request setValue:[NSString stringWithFormat:@"multipart/form-data; boundary=%@", boundary] forHTTPHeaderField:@"Content-Type"];
    NSData *boundaryData = [[[@"--" stringByAppendingString:boundary] stringByAppendingString:@"\r\n"] dataUsingEncoding:encoding];

    // Add Data to POST
    [requestBodyData appendData:boundaryData];
    [requestBodyData appendData:[[NSString stringWithFormat:@"Content-Disposition: multipart/form-data; "] dataUsingEncoding:encoding]];
    [requestBodyData appendData:[[NSString stringWithFormat:@"name=\"%@\"; ", @"contacts"] dataUsingEncoding:encoding]];

    [requestBodyData appendData:[[NSString stringWithFormat:@"\r\n\r\n"] dataUsingEncoding:encoding]];

    NSString* finalString = [[data gzipDeflate] base64EncodedString];
    //an example of finalString at this point: H4sIAAAAAAAAA6WOwQrCMAyGX0VyUqjHOtk7CN5HGekWMdB2knansXc3FcSDKIiHhBC+5P+6BWYJGdrOGbhdp0R1XiCgpwAtnCbPgcBAmqMn0c3WWrvbaNtrWVjNC+ZzffARPsCqIRSRn4EXllz6hJGULcK5YNJzHEehnB8qrgb8o9j8onj8rjigCNObobsD+2WV/kYBAAA=

    NSData* finalData = [finalString dataUsingEncoding:NSUTF8StringEncoding];

    [requestBodyData appendData:finalData];
    [requestBodyData appendData:[[NSString stringWithFormat:@"\r\n"] dataUsingEncoding:encoding]];
    [requestBodyData appendData:boundaryData];

    [request setHTTPBody:requestBodyData];

    [NSURLConnection sendAsynchronousRequest:request queue:[NSOperationQueue mainQueue] completionHandler:^(NSURLResponse *response, NSData *data, NSError *error) {
        NSLog(@"%@", [data objectFromJSONData]);
    }];

all seems well at this point, but the result from the (.net)server:

ERROR TYPE: System.IO.InvalidDataException - ERROR MESSAGE: The magic number in GZip header is not correct. Make sure you are passing in a GZip stream. -

i realize there isn’t much of a question here just yet, but heres something of one:
what could i be doing at this point that would yield incorrectly compressed data? (lets assume at this time at all is correct server-side given that android is fully functional and the guy who built that side of it is pretty damn experienced)…

Answer 1

There are 2 ways to use gzip, as a stream, or as a data file. The compression itself doesn’t require a header, but when saved to a file, it is standard to include the header and footer bytes as defined in the RFC.

So the problem is one library is looking for the header, while the other library that wrote the data didn’t include it, it’s probably just a raw stream of gzip data. It can be decompressed, but the library needs to know to not look for the headers.

When saving to a file, there are additional benefits of the file format, mainly due to the CRC check that is included at the end of the file to help verify that the data integrity of the file has not been corrupted.