Question

I have a low level caching mechanism which receives a json array from a server and caches it in a file.

The actual caching mechanism is just saving large streams to a file without awareness that it is json. Therefore when I would like to append a stream to an existing file cache by aggregating streams into another file I end up with something like this:

[{"id":3144,"created_at":"1322064201"}][{"id":3144,"created_at":"1322064201"}] 

where obviously what I desire is something like this:

[{"id":3144,"created_at":"1322064201"},{"id":3144,"created_at":"1322064201"}]

What is the most efficient/effective way of doing this?

I have looked into FilterReader but seen as I know that all I actually need to do is remove the last char ] of the existing cache and first char of new content [ and add a , I thought there may be a better way than checking every char in these big streams.

For context my code does something like this:

    ... input stream passed with new content

    File newCache = new File("JamesBluntHatersClub")
    FileOutputStream tempFileOutputStream = new FileOutputStream(newCache);
    FileInputStream fileInputStream = new FileInputStream(existingCache);
    copyStream(fileInputStream, tempFileOutputStream);
    copyStream(inputStream, tempFileOutputStream);

    ... clean up

UPDATE:

Having implemented a FilterReader which checks chars one at a time like so:

@Override
public int read() throws IOException {
    int content = super.read();
    // replace open square brackets with comma
    switch (content) {
        case SQUARE_BRACKETS_OPEN:
            return super.read();
        case SQUARE_BRACKETS_CLOSE:
            return super.read();
        default:
            return content;
    }
}

the processing time is unacceptably slow so I am looking for another option. I was thinking about using the file size to determine the size of the file and removing the tail square bracket this way

Answer 1

This method did the trick

/**
 * Copys the input streams in order to the output stream and retains json array
 * format
 * 
 * @param inputStreamA
 * @param inputStreamB
 * @param outputStream
 * @throws IOException
 */
private void copyStreamsToOutput(InputStream inputStreamA, InputStream inputStreamB,
        FileOutputStream outputStream) throws IOException {
    copyStream(inputStreamA, outputStream);
    // truncate file to remove trailing ']'
    outputStream.getChannel().truncate(outputStream.getChannel().size() - 1);
    // add comma between json objects
    outputStream.write(COMMA);
    // skip '['
    inputStreamB.skip(1);
    // and copy rest of streamas normal
    copyStream(inputStreamB, outputStream);
}

Would be very interested to here if this is bad practice, I am guessing there may be encoding issues.

UPDATE

/**
 * Copys the input streams in order to output stream and retains json array
 * format
 * 
 * @param inputStreamA
 * @param inputStreamB
 * @param outputStream
 * @throws IOException
 */
private void copyStreamsToOutput(InputStream inputStreamA, InputStream inputStreamB,
        FileOutputStream outputStream) throws IOException {
    copyStream(inputStreamA, outputStream);
    long channelSize = outputStream.getChannel().size();
    // truncate file to remove trailing ']'
    outputStream.getChannel().truncate(channelSize - 1);
    // check to see if array was empty (2 = [])
    if (channelSize > 2) {
        // add comma between json objects
        outputStream.write(COMMA);
    }
    // skip '['
    inputStreamB.skip(1);
    // and copy rest of streams normal
    copyStream(inputStreamB, outputStream);
    long newChannelSize = outputStream.getChannel().size();
    // check if we haven't just added a empty array
    if(newChannelSize - channelSize < 2){
        // if so truncate to remove comma 
        outputStream.getChannel().truncate(channelSize - 1);
        outputStream.write(CLOSE_SQUARE_BRACKET);
    }
}

Added ability to handle an empty json array in either stream