Question

I’m trying to write 2 different buffers (buffer A and B) multithreaded with SourceDataLine to play the sounds at the same time. But it keeps switching between buffer A and buffer B, do I need to merge the buffers together before writing them to my SourceDataLine or is there a way to play them synchronized?

class PlayThread extends Thread {
    byte[] buffer = new byte[2 * 1024];

    @Override
    public void run() {
        try {
            while (true) {
                DatagramPacket receive = new DatagramPacket(buffer, buffer.length);
                mDatagramSocket.receive(receive);
                mSourceDataLine.write(receive.getData(), 0, receive.getData().length);

                System.out.println("Received!");
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

I have 2 PlayThread instances with a different incoming buffer. Below is the function where the SourceDataLine is initialized.

private void init() {
    try {
        DataLine.Info sourceDataLineInfo = new DataLine.Info(
                SourceDataLine.class, audioFormat);
        DataLine.Info targetDataLineInfo = new DataLine.Info(
                TargetDataLine.class, audioFormat);

        Mixer.Info[] mixerInfo = AudioSystem.getMixerInfo();

        Mixer mixer = AudioSystem.getMixer(mixerInfo[3]);
        mSourceDataLine = (SourceDataLine) AudioSystem
                .getLine(sourceDataLineInfo);
        mTargetDataLine = (TargetDataLine) mixer.getLine(targetDataLineInfo);

        mSourceDataLine.open(audioFormat, 2 * 1024);
        mSourceDataLine.start();

        mTargetDataLine.open(audioFormat, 2 * 1024);
        mTargetDataLine.start();
    } catch (LineUnavailableException ex) {
        ex.printStackTrace();
    }
}

Thank you.

Answer 1

You absolutely do have to merge them. Imagine writing numbers to a file from two threads:

123456...
123456...

might become

11234235656...

Which is what’s happening to you.

Another issue is that you need to buffer your data as it comes in from the network, or you will likely drop it. You need at least two threads — one for reading and one for playing for this to work. However, in your case, you will probably have better luck with one reader thread for each input packet stream. (See my talk slides: http://blog.bjornroche.com/2011/11/slides-from-fundamentals-of-audio.html I specifically have a slide about streaming from http which is also relevant here)

So, Instead of multiple PlayThreads, make multiple ReaderThreads, which wait for data and then write to a buffer of some sort (PipedInput and PipedOutputStream work well for Java). Then you need another thread to read the data from the buffers and then write the COMBINED data to the stream.

This leaves your original question of how to combine the data. The answer is that there’s no single answer, but usually the easiest correct way is to average the data on a sample-by-sample basis. However, exactly how you do so depends on your data format, which your code doesn’t include. Assuming it’s big-endian 16-bit integer, you need to convert the incoming raw data to shorts, average the shorts, and convert the averaged short back to bytes.

The byte to short conversion is most easily accomplished using DataInputStream and DataOutputStream.