Question

I am using google caliper to run a very simple benchmark on a method. I am getting the following exception.

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    at com.code4refernce.caliper.SimpleCaliperTest.timeStringLen(SimpleCaliperTest.java:24)
    at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:616)
    at com.google.caliper.SimpleBenchmark$1.run(SimpleBenchmark.java:125)
    at com.google.caliper.TimeMeasurer.measureReps(TimeMeasurer.java:174)
    at com.google.caliper.TimeMeasurer.warmUp(TimeMeasurer.java:62)
    at com.google.caliper.TimeMeasurer.run(TimeMeasurer.java:122)
    at com.google.caliper.InProcessRunner.run(InProcessRunner.java:74)
    at com.google.caliper.InProcessRunner.run(InProcessRunner.java:49)
    at com.google.caliper.InProcessRunner.main(InProcessRunner.java:103)

I don’t understand why I am getting this exception. I tried to increase the JVM memory then also I get this exception.

The benchmark code is as following.

package com.code4refernce.caliper;
import java.util.Random;
import java.util.regex.Pattern;

import com.google.caliper.Param;
import com.google.caliper.SimpleBenchmark;

public class SimpleCaliperTest extends SimpleBenchmark {
    String regex = "(\\d{3}-\\d{3}-\\d{4})|(\\d{10})";
    Pattern REGEX_PATTERN = Pattern.compile(regex);

    String mdn[];
    Random random;
    @Param
    int index;

    @Override 
    protected void setUp() {
         random = new Random(0);
        mdn = new String[4];
            mdn[0] = "098412sdfasdf8000";
            mdn[1] = "11345";
            mdn[2] = "1423567890";
            mdn[3] = "123-456-7890";
    }


    public Boolean[] timeStringLen(int reps){
        Boolean results[] = new Boolean[reps];

        for(int i = 0; i<reps; i ++){

            results [i]= mdnCheckRegularMethod(mdn[index]);
        }
        return results;
    }
    public Boolean[] timeRegex(int reps){
        Boolean results[] = new Boolean[reps];
        for(int i = 0; i<reps; i ++){

            results[i]=mdnCheckRegEx(mdn[index]);
        }
        return results;
    }
    private boolean mdnCheckRegularMethod(String mdn){
        boolean result = false;

        if(mdn.length()==10){
            try{
                Integer.parseInt(mdn);
                result = true;
            }catch(Exception e){
                result = false;
            }
        }
        else if(mdn.length() == 13){
            byte[] bmdn = mdn.getBytes();
            for(int i = 0; i<bmdn.length; i++){
                if((i == 3 || i == 6) && bmdn[i] == '-'){}
                else if(bmdn[i] >= '0' && bmdn[i]<='9'){}
                else{
                    result = false;
                    break;
                }
            }
        }
        else{
            result = false;
        }
        return result;
    }

    private boolean mdnCheckRegEx(String mdn){
        return REGEX_PATTERN.matcher(mdn).matches();
    }
}

and the main class which runs the caliper benchmark.

package com.code4refernce.caliper;
import com.google.caliper.Runner;

    public class CaliperRunner {
        public static void main(String[] args) {
            String myargs[] = new String[2];
            myargs[0] = new String("-Dindex=0,1,2,3");
            myargs[1] = new String("-Jmemory=-Xms1024m ");
            Runner.main(SimpleCaliperTest.class, myargs);
        }
    }

I don’t understand whats going wrong here. Can someone point out?

Answer 1

Your method timeStringLen(int reps) may be called with reps up to Integer.MAX_VALUE. You could give your VM more memory via the -Xmx switch, but you’d better not allocate needlessly big arrays, as it’s not needed at all:

public boolean timeStringLen(int reps){
    boolean result = false;

    for(int i = 0; i<reps; i ++){
        result ^= mdnCheckRegularMethod(mdn[index]);
    }
    return result;
}

This does the same job, the only reason for returning a value is to prevent the JIT from optimizing it all away. Xoring is enough here, counting the cases when the method returned true is another possibility.

---

But there’s most probably another problem with your benchmark: The high values of reps used show that it’s running faster then I’d expect. The result of each iteration seems to be the same and I’d guess that you loops gets optimized to something like

result[i] = Boolean.TRUE

which I don’t think you want to measure. Using something like

Random random = new Random(0);
mdn = new String[4][1<<16];
for (int i=0; i<mdn.length; ++i) {
    mdn[0][i] = String.format("%03ddsfasdf00000", random.nextInt(1000));
    mdn[1][i] = String.format("%04d", random.nextInt(10000));
    mdn[2][i] = String.format("%10d", random.nextInt((int) 1e10));
    mdn[3][i] = String.format("%03d-%03d-%03d", random.nextInt(1000), random.nextInt(1000), random.nextInt(1000));
}

would help.