Question

I have a problem regarding an actual project using Java, Kinect (OpenNI) and Processing.
If I use just Processing and Java everything works fine nothing is stumbling and I get no exceptions.
But if I jail the processing Applet in a JFrame (to solve some problems with the Applet Style of Processing) I got the following problem:

• Every 3 seconds the kinect images hangs shortly (looks like Java is clearing anything out of the memory using the garbage collector)

• after 20 seconds the application stops and I get the following error:

  Exception in thread "Animation Thread" java.lang.OutOfMemoryError: Java heap space
  at java.awt.image.DataBufferInt.<init>(Unknown Source)
  at java.awt.image.Raster.createPackedRaster(Unknown Source)
  at java.awt.image.DirectColorModel.createCompatibleWritableRaster(Unknown Source)
  at java.awt.image.BufferedImage.<init>(Unknown Source)
  

---

Here is my relevant Code regarding the visualization:

public boolean drawGrayscaleImage(){    

    //init PApplet and build JFrame
    GrayscalePApplet grayscalePApplet = new  GrayscalePApplet ();
    grayscalePApplet.init();        
    this.grayscaleJFrame = this.initFrame(grayscalePApplet);

    //Set Uplink for PApplet and begin drawing
    grayscalePApplet.setGraphicP(this);     

    return false;
}

---

Here the drawing function from the Processing PApplet Class

public void draw(){

    if(graphicP != null){
        //creat the relevant image Buffers for java and Processing
        BufferedImage image = new BufferedImage(imageWidth, imageHeight, BufferedImage.TYPE_BYTE_GRAY);           
        PImage pimage = new PImage(image.getWidth(),image.getHeight(),PConstants.ARGB);

        //fill up the databuffer using a converted Kinect Grayscale Image
        DataBufferByte dataBuffer = new DataBufferByte(graphicP.getImage(ImageType.GRAYSCALE), this.imageWidth * this.imageHeight);
        Raster raster = Raster.createPackedRaster(dataBuffer,imageWidth, imageHeight, 8, null);
        image.setData(raster);

        //draw image to Processing
        image.getRGB(0, 0, pimage.width, pimage.height, pimage.pixels, 0, pimage.width);
        pimage.updatePixels();
        image(pimage, 0, 0);

        // null everything to get Garbagecollection to work (?)
        image = null;
        pimage = null;
        dataBuffer = null;
    }
}

How can I prevent that OutOfMemory Exception?
What may causes that exception?

Answer 1

Since I can’t see the full code, I can’t make an accurate assumption but my guess is the problem is here:

BufferedImage image = new BufferedImage(imageWidth, imageHeight, BufferedImage.TYPE_BYTE_GRAY);           
        PImage pimage = new PImage(image.getWidth(),image.getHeight(),PConstants.ARGB);

        //fill up the databuffer using a converted Kinect Grayscale Image
        DataBufferByte dataBuffer = new DataBufferByte(graphicP.getImage(ImageType.GRAYSCALE), this.imageWidth * this.imageHeight);
        Raster raster = Raster.createPackedRaster(dataBuffer,imageWidth, imageHeight, 8, null);
        image.setData(raster);

I’m not sure what Kinect API you’re using (OpenNI/libfreenect), but regardless, you probably shouldn’t create a new Image multiple times per frame( as you do in draw()). Images fill the memory quite quickly and you don’t need multiple images all the time, you simply need two images that you keep updating.
For example you initialize the images once in setup() and then update the pixels of those images in draw()

Also, if you’re using Processing I recommend having a look at Kinect Processing wrapper libraries like SimpleOpenNI for OpenNI or dLibs(Win)/OpenKinect for libfreenect. It’s a lot simpler to display the depth image with any of these:

With SimpleOpenNI:

import SimpleOpenNI.*;

SimpleOpenNI  ni;

void setup(){
  ni = new SimpleOpenNI(this);
  ni.enableDepth()
  size(ni.depthWidth(), ni.depthHeight()); 
}

void draw(){
  ni.update();
  image(context.depthImage(),0,0);
}

With dLibs freenect:

import dLibs.freenect.toolbox.*;
import dLibs.freenect.constants.*;
import dLibs.freenect.interfaces.*;
import dLibs.freenect.*;

Kinect kinect;                     // main kinect-object
KinectFrameDepth kinectDepth;     // depth frame
PImage depthFrame;

void setup(){
  size(640,480);
  kinect = new Kinect(0);
  kinectDepth = new KinectFrameDepth(DEPTH_FORMAT._11BIT_);// create a depth instance
  kinectDepth.connect(kinect);  //connect the created depth instance to the main kinect
  depthFrame = createImage(DEPTH_FORMAT._11BIT_.getWidth(), DEPTH_FORMAT._11BIT_.getHeight(), RGB);
}
void draw(){
  assignPixels(depthFrame, kinectDepth);
  image(depthFrame, 0, 0);
}
void assignPixels(PImage img, Pixelable kinectDev){
  img.loadPixels();
  img.pixels = kinectDev.getPixels();  // assign pixels of the kinect device to the image
  img.updatePixels();
}
void dispose(){
  Kinect.shutDown(); 
  super.dispose();
}

With OpenKinect P5:

import org.openkinect.*;
import org.openkinect.processing.*;

Kinect kinect;

void setup() {
  size(640,480);
  kinect = new Kinect(this);
  kinect.start();
  kinect.enableDepth(true);
}

void draw() {
  background(0);
  image(kinect.getDepthImage(),0,0);
}

void stop() {
  kinect.quit();
  super.stop();
}

It’s up to what OS and what features you need to pick the right Processing wrapper for you.

SimpleOpenNI sample

dLibs_freenect

OpenKinect P5