Question

I’m implementing a cache server with MongoDB and ConcurrentHashMap java class. When there are available space to put object in memory, it will put at. Otherwise, the object will be saved in a mongodb database. Is allowed that user specify a size limit in memory (this should not exceed heap size limit obviously!) for the memory cache. The clients can use the cache service connecting through RMI. I need to know the size of each object to verify if a new incoming object can be put into memory. I searched over internet and i got this solution to get size:

  public long getObjectSize(Object o){
    try {

        ByteArrayOutputStream bos = new ByteArrayOutputStream();
        ObjectOutputStream oos = new ObjectOutputStream(bos);
        oos.writeObject(o);
        oos.close();

        return bos.size();      
    } catch (Exception e) {
        return Long.MAX_VALUE;
    }
} 

This solution works very well. But, in terms of memory use doesn’t solve my problem. 🙁 If many clients are verifying the object size at same time this will cause stack overflow, right? Well… some people can say: Why you don’t get the specific object size and store it in memory and when another object is need to put in memory check the object size? This is not possible because the objects are variable in size. 🙁

Someone can help me? I was thinking in get socket from RMI communication, but I don’t know how to do this…

Answer 1

You can solve the problem of limiting the size of serialized objects using a custom FilterOutputStream that:

1. counts the bytes that are written by the write method calls, and
2. throws a custom IOException subclass when the count exceeds your limit.

Then put this filter between the ByteArrayOutputStream and the ObjectOutputStream.

This is what the code would look like (not tested!):

    public LimitExceededException extends IOException { ... }

    public class LimitingOutputStream extends FilterOutputStream {
        private int limit;
        private int count;

        public LimitingOutputStream(OutputStream out, int limit) {
            super(out);
            this.limit = limit;
        }

        @Override
        public void write(byte b) throws IOException {
            if (count++ > limit) {
                throw LimitExceededException(...);
            }
            super.write(b);
        }

        @Override
        // (This override is not strictly necessary, but it makes it faster)
        public void write(byte[] bytes, int from, int size) throws IOException {
            if (count += size > limit) {
                throw LimitExceededException(...);
            }
            super.write(bytes, from, size);
        }
    }

    /**
     * Return the serialization of `o` in a byte array, provided that it is
     * less than `limit` bytes.  If it is too big, return `null`.
     */
    public byte[] serializeWithLimit(Object o, int limit) {
        try {
            ByteArrayOutputStream bos = new ByteArrayOutputStream();
            LimitingOutputStream los = new LimitingOutputStream(bos, limit);
            ObjectOutputStream oos = new ObjectOutputStream(los);
            oos.writeObject(o);
            oos.close();
            return bos.toByteArray(); 
        } catch (LimitExceededException e) {
            return null;
        }
    }

^{Yes, this uses exceptions to “get out” when the limit is exceeded, but this is IMO a good use of exceptions. And I challenge anyone who disagrees with that to come up with a better solution. Put it in another Answer.}

---

By the way, this is REALLY BAD code:

} catch (Exception e) {
    return Long.MAX_VALUE;
}

In addition to the IOExceptions that you might expect to be thrown, you are also catching all manner of unchecked exceptions, most of which would be caused by bugs … that you need to know about:

1. It is bad practice to catch Exception except when you are trying to do a last-ditch diagnosis.

2. Whenever you catch unexpected exceptions, be sure to log them so that the stack trace can be recorded (depending on the logger configurations). Or if your application doesn’t use a logging framework, then have it call e.printStackTrace().

^{(And if you wouldn’t do this in production code, don’t do it into a StackOverflow Question either … ‘cos some copy-and-paste coder might just copy it.)}