I am writing a small app that capture the audio from the android MIC, performs a FFT on the input and then graphs the chart to the user. I am trying to do the recording and graphing concurrently (obviously with a small delay from being recorded to being graphed). I am attempting to launch two threads, one to read and one to process. However, I am having synchronization issues when I process it seems to only be receiving (or not) zeros. Any advise would be greatly appreciated. 🙂
public class Plotter extends Activity {
/* plotting objects */
private static GraphicalView mView;
private LineGraph line = new LineGraph();
private boolean recordAudio = true; // record?
private AudioRecord mRecorder = null; // audio object
private Menu mMenu; // app menu
private static final String LOG_TAG = "Frequency Plotter"; // debug tag
private Mfft mfft = null; // FFT class
private static final int BUF_SIZE = 8192; // amount to read in
private Thread listener = null;
private Thread processor = null;
Stack<Float> items = new Stack<Float>();
/* colors for line */
private int[] colors = {Color.BLUE,Color.CYAN,Color.DKGRAY,Color.GRAY,
Color.GREEN,Color.LTGRAY,Color.MAGENTA,Color.RED,Color.WHITE,Color.YELLOW};
private void processAudio(){
ArrayList<Double> real = new ArrayList<Double>();
try{
Random randomGenerator = new Random();
float[] in = new float[2048];
Arrays.fill(in,1);
while(true){
synchronized(items){
while(items.size() < 2048)
items.wait();
items.notifyAll();
for(int i=0; i < 2048; i++){
in[i] = items.pop();
}
}
double[] ret = mfft.fft(2048,44100,in); // get FFT of data
TimeSeries dataset = new TimeSeries( (real.size()+1)/2048 + "" );
XYSeriesRenderer renderer = new XYSeriesRenderer(); // customized renderer
// Customization time
renderer.setColor(colors[randomGenerator.nextInt(10)]);
renderer.setPointStyle(PointStyle.SQUARE);
renderer.setFillPoints(true);
line.addRenderer(renderer); // add custom renderer
for(int i = 0; i < 2048; i++){
real.add(ret[i]);
dataset.add(real.size()-1,ret[i]); // Add it to our graph
}
line.addDataset(dataset); // add data to line
mView.repaint(); // render lines
}
}catch(Exception e){
Log.e(LOG_TAG, e + " ");
}
}
private void writeToBuffer(short[] in) {
synchronized(items){
for(int i = 0; i < BUF_SIZE; i++){ // copy to create float
items.push((float)in[i]);
}
items.notifyAll();
}
}
private void listen(){
final short[] in = new short[BUF_SIZE];
mRecorder = new AudioRecord(
MediaRecorder.AudioSource.MIC, // source
44100, // frequency (HERTZ)
AudioFormat.CHANNEL_IN_MONO, // channel
AudioFormat.ENCODING_PCM_16BIT, // format
BUF_SIZE // size data packet
);
mRecorder.startRecording();
while(recordAudio){
try{
/* read next part */
mRecorder.read(in,0,BUF_SIZE); // read from device
writeToBuffer(in);
}catch(Exception t){
/* something went horribly wrong!!!*/
recordAudio = false;
Log.e(LOG_TAG, "Failure reading" + t.getMessage());
}
}
}
private void startRecording(){
/* create a new thread that will run the recording in the background */
listener = new Thread(
new Runnable(){
public void run(){
listen();
}
});
listener.start();
/* small delay to produce */
try {
Thread.sleep(100);
} catch (InterruptedException e1) {
e1.printStackTrace();
}
/* create a thread to process the audio */
processor = new Thread(
new Runnable(){
public void run(){
processAudio();
}
});
processor.start();
}
private void stopRecording(){
recordAudio = false;
mRecorder.stop();
mRecorder.release();
mRecorder = null;
}
/** clear the current chart */
private void clearChart(){
line = new LineGraph();
this.onStart();
}
/** Called when the activity is first created. */
@Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.main);
}
@Override
protected void onStart() {
super.onStart();
/* instantiate */
mfft = new Mfft(); // instance of the FFT class
mView = line.getView(this); // get the chart view
/* new horizontal layout */
LinearLayout ll = new LinearLayout(this);
ll.setOrientation(LinearLayout.HORIZONTAL);
ll.addView(mView); // add chart to layout
setContentView(ll); // set layout
}
@Override
public boolean onOptionsItemSelected(MenuItem item) {
// Handle item selection
switch (item.getItemId()) {
case R.id.record:
startRecording();
item.setEnabled(false); // disable start
mMenu.findItem(R.id.stop).setEnabled(true); // enable stop
return true;
case R.id.stop:
stopRecording();
item.setEnabled(false); // disable stop
mMenu.findItem(R.id.clear).setEnabled(true); // enable stop
return true;
case R.id.clear:
clearChart(); // clear chart
item.setEnabled(false); // disable clear
mMenu.findItem(R.id.record).setEnabled(true); // enable stop
return true;
default:
return super.onOptionsItemSelected(item);
}
}
@Override
public boolean onCreateOptionsMenu(Menu menu) {
mMenu = menu;
MenuInflater inflater = getMenuInflater();
inflater.inflate(R.menu.my_menu, menu);
return true;
}
}
Edit: Added full definitions.
similar example code, Audalyzer
Unfortunately the author has stopped development on this project, but the source tarball is still available online. In particular note:
org.hermit.android.io.AudioReader.java. You read the audio and pass it via aStackobject, this author usesshort []arrays. (still that does not seem like it should be your problem source…)http://code.google.com/p/moonblink/downloads/detail?name=SourceTarball.zip
BUF_SIZE thoughts
Your audio buffer (BUF_SIZE = 8192) feels a bit small. How does that relate to
AudioRecord.getMinBufferSize()? I used 2x minBufferSize, and that’s without doing any calculations on it (only read/write).HandlerthoughtsI’m still reviewing your code, unclear how your threads communicate. But, your problem sounds like it needs a way for the threads to communicate a
Handler.Below are the links I’ve been reviewing to grasp how to use
Handlers and communicate between threads effectively:threads – nice overview of handlers (withOUT looper). w/ code example:
com.indy.testing.TestMain.java.MyThread.javahttp://indyvision.net/2010/02/android-threads-tutorial-part-3/
threads – ok overview of handlers and loopers
http://techtej.blogspot.com/2011/02/android-passing-data-between-main.html
threads w/ 2way comm. w/ code example: sample.thread.messaging.ThreadMessaging.java
http://codinghard.wordpress.com/2009/05/16/android-thread-messaging/