Question

I have a bunch of SQL statements that were originally written to support Oracle and use a lot of specific Oracle syntax. Most of this was pretty easy to address using simple regex style find and replace or through the use of JDBC functions. This case, however, is causing me a bit more trouble.

There are a bunch of instances where concatenation is used for values and I’m trying to replace col1 || col2 || col3 to be replaced with CONCAT(col1,col2,col3) style syntax in the case that the database engine is not Oracle.

I know it would be ideal to use some sort of ORM, but that isn’t practical in this case for various reasons.

I have a test app setup to test out some of the cases I am most likely to come into contact with but I am having trouble figuring out how to get it to replace all instance and also a variable number of matches after the ||. The solution as I have it now only matches the last occurence of col||col. Maybe it’s not possible, but I would like to try and exhaust this solution path.

import java.util.regex.Matcher;
import java.util.regex.Pattern;

public class regex {
    public static void main(String args[]){
        String sSource = "SELECT col1, col2||word||default_col||another field1, col3 || ' quote test ' || default_1 field2 FROM table order by 1";
        try{
            String pattern ="((\\b\\[a-zA-Z0-9_]+|'.*')\\s*(\\|\\|)\\s*(\\w+))";
            Pattern p = Pattern.compile(pattern, Pattern.CASE_INSENSITIVE|Pattern.MULTILINE|Pattern.DOTALL);
            Matcher m = p.matcher(sSource);
            while(m.find()){
                sSource = m.replaceAll("CONCAT($2 , $4)");
                System.out.println("found match");
            }
            System.out.println(sSource);
        } catch( Exception e) {
            System.out.println("Bad things:" + e.getMessage());
        }
    }
}

Answer 1

You cannot really do concat replacement for all || with regular expressions as they aren’t strong enough to handle the SQL-grammar. You can concatenate anything and everything using || including character literals containing ||.

However, regular expression might be enough in some special cases. It might be easier to divide and conquer the problem e.g.

1. convert (SELECT|,) <content> || to (SELECT|,) concat(<content> ||

2. convert || <content> <field> (FROM|,) to , <content>) <field> (FROM|,

3. convert all remaining || to ,

Store the result after each phase and apply the rules to that intermediate conversion.

Something like

private static String identifierOrString = "[a-zA-Z0-9_\\.\\(\\),]+";

public static void main(String[] args) {

  String testCase = "SELECT col1, col2||word||default_col||another field1, col3 || ' quote test ' || default_1 field2 FROM table where 'abc' = col4 || col5 || col6 GROUP BY col7 || col8";

  testCase = convertBeginnings(testCase);
  System.out.println("Phase 1: " + testCase);

  testCase = convertEndings(testCase);
  System.out.println("Phase 2: " + testCase);

  testCase = convertRemainingOperators(testCase);
  System.out.println("Finished: " + testCase);    
}

private static String convertBeginnings(String testCase) {
  return replace("(SELECT|WHERE|=|<>|like|GROUP BY|\\,)(\\s+)(%s|'[^']*')\\s*\\|\\|", testCase, "%s %s concat(%s ||");
}

private static String convertEndings(String testCase) { 
  return replace("\\|\\|\\s*(%1$s|'[^']*')\\s*(\\s%1$s)?\\s*((\\,|FROM|GROUP BY|ORDER BY|=|<>|like|$))", testCase,",%s) %s %s");
}

private static String replace(String regexp, String source , String target) {
  Matcher m = match(regexp, source);
  while(m.find()) {
    source = source.replace(m.group(0), String.format(target, nvl(m.group(1)), nvl(m.group(2)), nvl(m.group(3))));
  }
  return source;
}

private static String nvl(String value) {
  return null == value ? "" : value;
}

private static String convertRemainingOperators(String testCase) {
  return testCase.replaceAll("\\|\\|", ",");
}

private static Matcher match(String regexp, String target ) {
  Pattern p = Pattern.compile(String.format(regexp, identifierOrString), Pattern.CASE_INSENSITIVE|Pattern.MULTILINE|Pattern.DOTALL);
  return p.matcher(target); 
}