Question

Currently, I have a script that loops over System.in for data processing. I am passing data to it from several files with cat.

cat myfiles*.txt | java MyDataProcessor 

Based on the idea that cat adds some inefficiency vs. Java opening the files directly, I’d like to optimize this to where Java opens the files directly:

java MyDataProcessor myfiles*.txt

Are there any Java libraries that make this fairly easy (i.e. that handle the translation of posix wildcards into file handlers)?

Answer 1

In case this isn’t obvious to someone, as it wasn’t to me at first, if the files are local, then you can let Posix do the parsing for you, and the files will be passed to main(String[] args) as arguments. In my case, I had a few other parameters, so just moved the wildcard argument as the last one.

// USAGE: java MyProcessor arg1 arg2 myfiles*.txt

public static void main(String[] args) throws Exception {
  String arg1 = args[0];
  String arg2 = args[1];

  // looping over all input files
  for (int i = 2; i < args.length; i++) {
    File inputFile = new File(args[i]).getCanonicalFile();
    BufferedReader in = new BufferedReader(new FileReader(inputFile)); 
    // ...
  }
}