Question

Is there a good Java library for taking the legwork out of writing good micro-benchmarks? I’m thinking something which can provide (with a minimum of hassle) provide text (CSV or HTML, take your pick) output of results and maybe graphs summarizing results. Ideally, it should be something that plays nicely with JUnit or equivalent, and should be simple to configure benchmarks with variable parameters.

I’ve looked at japex, but found it too heavyweight (25 MB of libraries to include?!) and frankly it was just a pain to work with. Virtually nonexistent documentation, mucking about with ant, XML, and paths… etc.

Answer 1

A few of us from the Google Collections team are in the early days of building something that satisfies your needs. Here’s the code to measure how long foo() takes:

public class Benchmark1 extends SimpleBenchmark {
  public void timeFoo(int reps) {
    for (int i = 0; i < reps; i++) {
      foo();
    }
  }
}

Neither the API nor the tool itself is particularly stable. We aren’t even ready to receive bug reports or feature requests! If I haven’t scared you off yet, I invite you to take Caliper for a spin.