Question

I have created an application to process log files but am having some bottle neck when the amount of files = ~20

The issue comes from a particular method which takes on average a second or so to complete roughly and as you can imagime this isn’t practical when it needs to be done > 50 times

private String getIdFromLine(String line){
    String[] values = line.split("\t");
    String newLine = substringBetween(values[4], "Some String : ", "Value=");
     String[] split = newLine.split(" ");
     return split[1].substring(4, split[1].length());
}



private String substringBetween(String str, String open, String close) {
      if (str == null || open == null || close == null) {
          return null;
      }
      int start = str.indexOf(open);
      if (start != -1) {
          int end = str.indexOf(close, start + open.length());
          if (end != -1) {
              return str.substring(start + open.length(), end);
          }
      }
      return null;
  }

A line comes from the reading of a file which is very efficient so I don’t feel a need to post that code unless someone asks.

Is there anyway to improve perofmrance of this at all?

Thanks for your time

Answer 1

A few things are likely problematic:

1. Whether or not you realized, you are using regular expressions. The argument to String.split() is a treated as a regex. Using String.indexOf() will almost certainly be a faster way to find the particular portion of the String that you want. As HRgiger points out, Guava’s splitter is a good choice because it does just that.

2. You’re allocating a bunch of stuff you don’t need. Depending on how long your lines are, you could be creating a ton of extra Strings and String[]s that you don’t need (and the garbage collecting them). Another reason to avoid String.split().

3. I also recommend using String.startsWith() and String.endsWith() rather that all of this stuff that you’re doing with the indexOf() if only for the fact that it’d be easier to read.