Question

I’m pushing data from my Java application over a USB connection to an Arduino in the other end.

The Arduino is only capable of buffering 64 bytes of data at its end, so I have to limit how many bytes are sent in each ‘gulp’ from my Java app (excess bytes will be lost). When the Arduino code is prepared to receive more bytes, it will send a simple ping up the wire.

So, I’ve extended BufferedOutputStream in a class ArduinoBufferedOutputStream, which wrappes the actual output stream. From different parts of the Java app an arbitrary number of bytes are written to the stream (with write(byte b)), and the stream is occationally flushed.

What I need (I guess) is to override BufferedOutputStreams flush method, so that it will not send more than 64 bytes, before receiving the ping from Arduino, at which time the stream should send 64 more bytes (or less).

     static class ArduinoBufferedOutputStream extends BufferedOutputStream {

        public static final int WIRE_CAPACITY = 25;
        private byte[] waiting = new byte[0];
        private int onWire = 0;

        public ArduinoBufferedOutputStream(final OutputStream wrapped) throws IOException {
            super(wrapped, 500);
        }

        public void ping() {
            this.onWire = 0;
            this.flush();
        }

        @Override
        public void flush() throws IOException {
            if (this.onWire >= WIRE_CAPACITY) {
                return; // we're exceeding capacity, don't to anything before the next ping
            }
            if (this.count > WIRE_CAPACITY) {
                this.waiting = new byte[this.count - WIRE_CAPACITY];
                System.arraycopy(this.buf, WIRE_CAPACITY, waiting, 0, this.count - WIRE_CAPACITY);
                this.buf = Arrays.copyOfRange(this.buf, 0, WIRE_CAPACITY);
                this.count = WIRE_CAPACITY;
            } else {
                this.waiting = new byte[0];
            }
            onWire += this.count;
            super.flush();
            if (this.waiting.length > 0) {
                System.arraycopy(this.waiting, 0, this.buf, 0, Math.min(this.waiting.length, WIRE_CAPACITY));
                this.count = Math.min(this.waiting.length, WIRE_CAPACITY);
            }
        }
    }

However, this doesn’t work properly. Bytes are lost if the buffer contains more than WIRE_CAPACITY bytes, as demonstrated by the following testcase:

@Test
public void testStream() throws IOException {
    final ArduinoBufferedOutputStream stream = new ArduinoDisplayOutputStream.ArduinoBufferedOutputStream(System.out);
    stream.write("This is a very, very long string, which must be made even longer to demonstrate the problem".getBytes());
    stream.flush();
    stream.ping();
    stream.ping();
    stream.ping();
    stream.ping();
    stream.ping();
    stream.ping();
    stream.ping();
    stream.ping();
    stream.ping();
}

The following string is printed: This is a very, very long string, which must be ma, while I obviously would like the whole string to be printed.

Can anyone see what I’m doing wrong here?
Or even better, does anyone know of an existing library that does what I want?

Answer 1

OK, I think I got it right, finally. Extending BufferedOutputStream was a dead end. Instead, I use a separate buffer, like @ben-lawry suggested. And by using a ConcurrentLinkedQueue as that buffer, I can easily poll WIRE CAPACITY number of bytes on each flush().

Not terribly efficient, one might say, but the stuff going on on the Java side will still be speedy compared to the Arduino processing.

Here’s what I got:

   static class ArduinoBufferedOutputStream extends OutputStream {

        public static final int WIRE_CAPACITY = 25;
        private final ConcurrentLinkedQueue<Byte> buffer = new ConcurrentLinkedQueue<Byte>();
        private int onWire = 0;
        private OutputStream wrapped;

        public ArduinoBufferedOutputStream(final OutputStream wrapped) throws IOException {
            super();
            this.wrapped = wrapped;
        }

        @Override
        public void write(int i) throws IOException {
            this.buffer.add((byte) i);
        }

        @Override
        public void write(byte[] bytes) throws IOException {
            for (byte aByte : bytes) {
                this.buffer.add(aByte);
            }
        }

        @Override
        public void write(byte[] bytes, int off, int len) throws IOException {
            for (int i = off; i < len; i++) {
                this.buffer.add(bytes[i]);
            }
        }

        public void ping() throws IOException {
            synchronized (this.buffer) {
                this.onWire = 0;
            }
            this.flush();
        }

        @Override
        public void flush() throws IOException {
            synchronized (this.buffer) {
                for (; this.onWire < WIRE_CAPACITY && buffer.size() > 0; this.onWire++) {
                    wrapped.write(buffer.poll());
                }
            }
        }
    }

Thanks, @gpeche and @ben-lawry – it certainly helped getting a few more eyes on the problem.