Question

Is there some library or generator that I can use to generate multiple templated java classes from a single template?

Obviously Java does have a generics implementation itself, but since it uses type-erasure, there are lots of situations where it is less than adequate.

For example, if I want to make a self-growing array like this:

 class EasyArray {
      T[] backingarray;
 }

(where T is a primitive type), then this isn’t possible.

This is true for anything which needs an array, for example high-performance templated matrix and vector classes.

It should probably be possible to write a code generator which takes a templated class and generates multiple instantiations, for different types, eg for ‘double’ and ‘float’ and ‘int’ and ‘String’. Is there something that already exists that does this?

Edit: note that using an array of Object is not what I’m looking for, since it’s no longer an array of primitives. An array of primitives is very fast, and uses only as much space a sizeof(primitive) * length-of-array. An array of object is an array of pointers/references, that points to Double objects, or similar, which could be scattered all over the place in memory, require garbage collection, allocation, and imply a double-indirection for access.

Edit3:

Here is code to show the difference in performance between primitive and boxed arrays:

int N = 10*1000*1000;
double[]primArray = new double[N];
for( int i = 0; i < N; i++ ) {
    primArray[i] = 123.0;
}
Object[] objArray = new Double[N];
for( int i = 0; i < N; i++ ) {
    objArray[i] = 123.0;
}
tic();
primArray = new double[N];
for( int i = 0; i < N; i++ ) {
    primArray[i] = 123.0;
}
toc();
tic();
objArray = new Double[N];
for( int i = 0; i < N; i++ ) {
    objArray[i] = 123.0;
}
toc();

Results:

double[] array: 148 ms
Double[] array: 4614 ms

Not even close!

Answer 1

Came up with the following draft way of doing this:

We create a class, using the placeholder type TemplateArgOne, and add an annotation that is used to facilitate parsing:

@GenerateFrom(EndlessArray.class)
public class EndlessArray {
    TemplateArgOne[] values;
    int size = 0;
    int capacity;
    public EndlessArray() {
        capacity = 10;
        values = new TemplateArgOne[capacity];
    }
    public void reserve(int newcapacity ) {
        if( newcapacity > capacity ) {
            TemplateArgOne[] newvalues = new TemplateArgOne[newcapacity];
            for( int i = 0; i < size; i++ ) {
                newvalues[i] = values[i];
            }
            values = newvalues;
            capacity = newcapacity;
        }
    }
    public void add( TemplateArgOne value ) {
        if( size >= capacity - 1 ) {
            reserve( capacity * 2);
        }
        values[size] = value;
    }
    public void set( int i, TemplateArgOne value ) {
        values[i] = value;
    }
    public TemplateArgOne get( int i ) {
        return values[i];
    }
}

We define TemplateArgOne as an empty class, so the above code compiles just fine, can edit it with no problems in Eclipse:

public class TemplateArgOne {
}

Now, we create a new class to instantiate the template class, and we add a couple of annotations to tell the generator which class we want to instantiate, and what type we want for TemplateArgOne:

@GenerateFrom(root.javalanguage.realtemplates.EndlessArray.class)
@GenerateArg(from=TemplateArgOne.class,to=double.class)
public class EndlessArrayDouble {

}

Then we run the generator, passing in the directory of our source code, and the name of the instantiation class, ie EndlessArrayDouble:

public static void main(String[] args ) throws Exception {
    new RealTemplateGenerator().go("/data/dev/machinelearning/MlPrototyping", EndlessArrayDouble.class);
}

And presto! When we go to the EndlessArrayDouble class, it’s been populated with our code! :

package root.javalanguage.realtemplates;

import root.javalanguage.realtemplates.RealTemplateGenerator.*;

@GenerateArg(from=TemplateArgOne.class,to=double.class)
@GenerateFrom(root.javalanguage.realtemplates.EndlessArray.class)
public class EndlessArrayDouble {
    double[] values;
    int size = 0;
    int capacity;
    public EndlessArrayDouble() {
        capacity = 10;
        values = new double[capacity];
    }
    public void reserve(int newcapacity ) {
        if( newcapacity > capacity ) {
            double[] newvalues = new double[newcapacity];
            for( int i = 0; i < size; i++ ) {
                newvalues[i] = values[i];
            }
            values = newvalues;
            capacity = newcapacity;
        }
    }
    public void add( double value ) {
        if( size >= capacity - 1 ) {
            reserve( capacity * 2);
        }
        values[size] = value;
    }
    public void set( int i, double value ) {
        values[i] = value;
    }
    public double get( int i ) {
        return values[i];
    }
}

Note that if you run it multiple times, without modifying the template class, then the instantiation class java file will not be modified, ie it wont trigger a redundant file reload in Eclipse.

The generator code:

//Copyright Hugh Perkins 2012, hughperkins -at- gmail
//
//This Source Code Form is subject to the terms of the Mozilla Public License, 
//v. 2.0. If a copy of the MPL was not distributed with this file, You can 
//obtain one at http://mozilla.org/MPL/2.0/.

package root.javalanguage.realtemplates;

import java.lang.annotation.*;
import java.util.*;

public class RealTemplateGenerator {
    @Target(ElementType.TYPE)
    @Retention(RetentionPolicy.SOURCE)
    public @interface GenerateFrom {
        Class<?> value();
    }
    @Target(ElementType.TYPE)
    @Retention(RetentionPolicy.SOURCE)
    public @interface GenerateArg {
        Class<?> from();
        Class<?> to();
    }
public final static BufferedReader newBufferedReader(String filepath ) throws Exception {
    FileInputStream fileInputStream = new FileInputStream( filepath );
    InputStreamReader inputStreamReader = new InputStreamReader(fileInputStream);
    BufferedReader bufferedReader = new BufferedReader(inputStreamReader);
    return bufferedReader;
}

public final static BufferedWriter newBufferedWriter( String filepath ) throws Exception {
    FileOutputStream fileOutputStream = new FileOutputStream(filepath);
    OutputStreamWriter outputStreamWriter = new OutputStreamWriter(fileOutputStream);
    BufferedWriter bufferedWriter = new BufferedWriter(outputStreamWriter);
    return bufferedWriter;
}

public final static ArrayList<String> readFileAsLines(String filepath ) throws Exception {
    ArrayList<String> lines = new ArrayList<String>(40000);

    BufferedReader bufferedReader = newBufferedReader(filepath);

    String line = bufferedReader.readLine();
    int numlines = 0;
    while( line != null ) {
        lines.add(line.substring(0, line.length() ));
        line = bufferedReader.readLine();
        numlines++;
    }
    bufferedReader.close();
    return lines;
}

public final static void writeFileFromLines( String filepath, ArrayList<String> lines ) throws Exception {
    BufferedWriter bufferedWriter = newBufferedWriter(filepath);
    for( String line : lines ) {
        bufferedWriter.write(line + "\n");
    }
    bufferedWriter.close();
}

    void go( String sourcedirectory, Class<?> targetclass ) throws Exception {
        String targetclassfilepath = sourcedirectory + "/" + targetclass.getCanonicalName().replace(".","/") + ".java";
        ArrayList<String> initialLines = FileHelper.readFileAsLines(targetclassfilepath);
        String fromclassname = "";
        HashMap<String,String> argOldToNew = new HashMap<String, String>();
        ArrayList<String> generatelines = new ArrayList<String>();
        for( String line : initialLines ) {
            if( line.startsWith("@GenerateFrom")){
                fromclassname = line.split("\\(")[1].split("\\.class")[0];
            }
            if( line.startsWith("@GenerateArg")) {
                String fromclass= line.split("from=")[1].split("\\.")[0];
                String toclass = line.split("to=")[1].split("\\.")[0];
                argOldToNew.put(fromclass,toclass);
                generatelines.add(line);
            }   
        }
        Class<?> targettype = this.getClass().forName(fromclassname); 
        String fromclassfilepath = sourcedirectory + "/" + targettype.getCanonicalName().replace(".","/") + ".java";
        ArrayList<String> templateLines = FileHelper.readFileAsLines(fromclassfilepath );
        ArrayList<String> generatedLines = new ArrayList<String>();
        for( int i = 0; i < templateLines.size(); i++ ){
            String line = templateLines.get(i);
            if( !line.startsWith("@GenerateFrom") && !line.startsWith("@GenerateArg")) {
                for( String oldarg : argOldToNew.keySet() ) {
                    line = line.replace(oldarg, argOldToNew.get(oldarg));
                }
                line = line.replace(targettype.getSimpleName(), targetclass.getSimpleName());
            } else if( line.startsWith("@GenerateFrom") ) {
                for( String generateline : generatelines ) {
                    generatedLines.add(generateline);
                }
            }
            generatedLines.add(line);
        }
        boolean isModified = false;
        if( initialLines.size() != generatedLines.size() ) {
            isModified = true;
        } else {
            for(int i = 0; i < initialLines.size(); i++ ) {
                if( !initialLines.get(i).equals(generatedLines.get(i))) {
                    isModified = true;
                    break;
                }
            }
        }
        if( isModified ) {
            FileHelper.writeFileFromLines(targetclassfilepath, generatedLines);
            System.out.println("Generated " + targetclassfilepath );
        } else {
            System.out.println("No change to " + targetclassfilepath );         
        }
    }
    public static void main(String[] args ) throws Exception {
        new RealTemplateGenerator().go("/data/dev/machinelearning/MlPrototyping", EndlessArrayDouble.class);
    }
}