Question

As part of a University project, I am working on a Java swing program for annotating images. One of the features of this program is to be able to draw polygons around certain areas of an image which can then be labeled with a caption.

When drawing a polygon, each click draws a new green vertex on the image and links this vertex to the previous one by drawing a line. There is also a preview line that is drawn as the user is moving their mouse so they can see what the next click will add to the shape of the polygon.

The problem I am having is that once the user has drawn one polygon, there is a significant slowdown in the overall program performance. The drawing of the preview line becomes incredibly jittery, to the point where is is difficult to use.

The code which is responsible for this part of the program is in the file ImagePanel.java:

package hci;

import javax.imageio.ImageIO;
import javax.swing.JPanel;
import javax.swing.JOptionPane;

import java.awt.Color;
import java.awt.BasicStroke;
import java.awt.Stroke;
import java.awt.Dimension;
import java.awt.Graphics;
import java.awt.Graphics2D;
import java.awt.Image;
import java.awt.Polygon;
import java.awt.event.MouseEvent;
import java.awt.event.MouseListener;
import java.awt.event.MouseMotionListener;
import java.awt.image.BufferedImage;
import java.awt.geom.Point2D;
import java.io.File;
import java.util.ArrayList;

import hci.utils.*;

public class ImagePanel extends JPanel implements MouseListener, MouseMotionListener {

    private static final long serialVersionUID = 1L;
    BufferedImage image = null;
    CaptionedPolygon currentPolygon = null;
    ArrayList<CaptionedPolygon> polygonsList = null;
    Point mousePos;
    public static final int FIRST_NODE_SIZE = 15;

    public ImagePanel() {
        currentPolygon = new CaptionedPolygon();
        polygonsList = new ArrayList<CaptionedPolygon>();
        mousePos = new Point(0,0);

        this.setVisible(true);

        Dimension panelSize = new Dimension(800, 600);
        this.setSize(panelSize);
        this.setMinimumSize(panelSize);
        this.setPreferredSize(panelSize);
        this.setMaximumSize(panelSize);

        addMouseListener(this);
        addMouseMotionListener(this);
    }

    public ImagePanel(String imageName) throws Exception{
        this();
        image = ImageIO.read(new File(imageName));
        if (image.getWidth() > 800 || image.getHeight() > 600) {
            int newWidth = image.getWidth() > 800 ? 800 : (image.getWidth() * 600)/image.getHeight();
            int newHeight = image.getHeight() > 600 ? 600 : (image.getHeight() * 800)/image.getWidth();
            System.out.println("SCALING TO " + newWidth + "x" + newHeight );
            Image scaledImage = image.getScaledInstance(newWidth, newHeight, Image.SCALE_FAST);
            image = new BufferedImage(newWidth, newHeight, BufferedImage.TYPE_INT_RGB);
            image.getGraphics().drawImage(scaledImage, 0, 0, this);
        }
    }

    public void ShowImage(Graphics g) {

        if (image != null) {
            g.drawImage(
                    image, 0, 0, null);
        }
    }

    @Override
    public void paintComponent(Graphics g) {
        super.paintComponent(g);

        //display image
        ShowImage(g);
        drawPreviewLine(g);

        //display all the completed polygons
        for(CaptionedPolygon polygon : polygonsList) {
            fillPolygon(polygon, g);
            drawPolygon(polygon, g);
            finishPolygon(polygon, g);
        }

        //display current polygon
        drawPolygon(currentPolygon, g);


    }

    public void drawPreviewLine(Graphics g){
        if (currentPolygon.points.size() > 0 && mousePos != null){
            Point currentPoint = currentPolygon.points.get(currentPolygon.points.size() - 1);
            g.setColor(Color.GREEN);
            g.drawLine(currentPoint.getX(), currentPoint.getY(), mousePos.getX(), mousePos.getY());
        }
    }

    public void fillPolygon(CaptionedPolygon polygon, Graphics g){
        Color fillColor = new Color((float)0.0,(float)1.0,(float)0.0, (float)0.3);
        Polygon polyToDraw =  new Polygon();
        for (Point point : polygon.points){
            polyToDraw.addPoint(point.getX(), point.getY());
        }
        g.setColor(fillColor);
        g.fillPolygon(polyToDraw);
    }

    public void drawPolygon(CaptionedPolygon polygon, Graphics g) {
        for(int i = 0; i < polygon.points.size(); i++) {
            int sizeModifier = 0;
            Graphics2D g2 = (Graphics2D)g;
            g2.setColor(Color.GREEN);
            g2.setStroke(new BasicStroke(1));
            Point currentVertex = polygon.points.get(i);


            if(currentPolygon.equals(polygon) && i == 0){ //First point of the current polygon

                //Enlarge circle drawn if mouse hovers over point
                if(pointWithinCircle(mousePos.getX(), mousePos.getY(), currentVertex.getX(), currentVertex.getY(), (FIRST_NODE_SIZE + 2)/2)){
                    sizeModifier = 3;
                }

                int nodeSize = FIRST_NODE_SIZE + sizeModifier;

                g2.setColor(Color.WHITE);
                g2.fillOval(currentVertex.getX() - nodeSize/2 , currentVertex.getY() - nodeSize/2, nodeSize, nodeSize);
                g2.setStroke(new BasicStroke(2));
                g2.setColor(Color.GREEN);
                g2.drawOval(currentVertex.getX() - nodeSize/2 , currentVertex.getY() - nodeSize/2, nodeSize, nodeSize);
            }
            else if (i != 0){ //Some arbitary middle point
                Point prevVertex = polygon.points.get(i - 1);
                g2.drawLine(prevVertex.getX(), prevVertex.getY(), currentVertex.getX(), currentVertex.getY());
                g2.fillOval(currentVertex.getX() - 5, currentVertex.getY() - 5, 10, 10);
            }
            else{ //First point of some non current polygon
                g2.fillOval(currentVertex.getX() - 5, currentVertex.getY() - 5, 10, 10);
            }
        }
    }

    public void finishPolygon(CaptionedPolygon polygon, Graphics g) {
        //if there are less than 3 vertices than nothing to be completed
        if (polygon.points.size() >= 3) {
            Point firstVertex = polygon.points.get(0);
            Point lastVertex = polygon.points.get(polygon.points.size() - 1);

            g.setColor(Color.GREEN);
            g.drawLine(firstVertex.getX(), firstVertex.getY(), lastVertex.getX(), lastVertex.getY());
        }
    }

    public void addNewPolygon() {
        //finish the current polygon if any
        if (currentPolygon.points.size() > 0 ) {
            currentPolygon.caption = JOptionPane.showInputDialog(this, "Please enter a caption for this area") ;
            polygonsList.add(currentPolygon);
        }

        currentPolygon = new CaptionedPolygon();
        repaint();
    }

    public boolean pointWithinCircle(int targetX, int targetY, int circleCentX, int circleCentY, double circleRadius){
        Point2D.Double mousePoint = new Point2D.Double(targetX,targetY);
        Point2D.Double firstNodePoint = new Point2D.Double(circleCentX, circleCentY);
        return (mousePoint.distance(firstNodePoint) <= circleRadius);
    }

    @Override
    public void mouseClicked(MouseEvent e) {
    }

    @Override
    public void mouseEntered(MouseEvent arg0) {
    }

    @Override
    public void mouseExited(MouseEvent arg0) {
    }

    @Override
    public void mousePressed(MouseEvent e) {
        int x = e.getX();
        int y = e.getY();

        //check if the cursor is within image area
        if (x > image.getWidth() || y > image.getHeight()) {
            return;
        }

        //Clicking the left button will either add a new vertex or finish off a polygon
        if (e.getButton() == MouseEvent.BUTTON1) {
            if (currentPolygon.points.size() > 0 ){

                if(pointWithinCircle(x, y, currentPolygon.points.get(0).getX(), currentPolygon.points.get(0).getY(), FIRST_NODE_SIZE + 2)){
                    addNewPolygon();
                }
                else{
                    currentPolygon.points.add(new Point(x,y));
                    System.out.println(x + " " + y);
                    repaint();
                }
            }
            else{
                currentPolygon.points.add(new Point(x,y));
                System.out.println(x + " " + y);
                repaint();
            }
        } 
    }

    @Override
    public void mouseReleased(MouseEvent arg0) {
    }

    public void mouseDragged(MouseEvent e){
    }

    public void mouseMoved(MouseEvent e){
        mousePos.setX(e.getX());
        mousePos.setY(e.getY());
        repaint();
    }

}

I have tried running this program under several different operating systems and laptops now, and the slow-down is noticeable on all of them. This suggests it is a problem with my code rather than just what is running it.

I have a feeling my issue has something to do with the excessive amount of times I call the repaint() method. I haven’t really seen many good resources online on the best way to implement drawing features such as these using Java’s swing and graphics libraries, so advice on general practices I am messing up as well as direct solutions to this problem would be desirable.

Answer 1

It looks like you are drawing directly to the frame buffer. This is uber slow as each time you draw anything, the JVM has to make a system call to update the image on screen.

You will see much better rendering speed if you do all your drawing operations on a single frame in JVM memory, and only output to the system when the whole image is ready. (This is kind of what you’re already doing with your background-image image, infact you can just re-use the graphics object you already create, but don’t use)

So, you need to create a canvas to draw to;

BufferedImage canvas = new BufferedImage(width, height, BufferedImage.TYPE_3BYTE_BGR);

Then to grab the Graphics object that you will use to draw to the canvas;

Graphics cg = canvas.getGraphics();

Do all your drawing operations on cg, then in your paintComponent(Graphics g) function, just draw canvas to the component in a single call using;

g.drawImage(canvas, 0, 0, null);

In order to get even better performance, you should be drawing to a VolatileImage instead of a BufferedImage. But the BufferedImage is much easier to use, and will perform just fine for your purposes.