Question

I have a spring batch program which reads from a file and writes to a database.

Data is delimited by || (double pipes) and there is no delimiter at end of line.

one||two||three||four
foo||bar||foo1||bar1

Issue is the end of each line is having hundreds of trailing spaces before end of line character.

Using ctrl + Q, I in text pad shows:

one||two||three||four.......................................
foo||bar||foo1||bar1...............
car||bike||tango||charlie..........................

This is throwing parsing error at each line in ItemReader (org.springframework.batch.item.file.FlatFileItemReader)

I can not remove the spaces manually and need some automation so that before the file is read by spring batch program, the spaces are gone.

How to do this ?

Note: The error occurs only when I run the program over unix server. It does not happen on windows.

<beans:bean id="myBufferedReaderFactory" class="com.mypackage.MyBufferedReaderFactory"/>

<beans:bean id="FileToDBItemReader" class="org.springframework.batch.item.file.FlatFileItemReader">
                 <beans:property name="BufferedReaderFactory" ref="myBufferedReaderFactory"/>
        <beans:property name="resource" ref="MyInputFileResource" />
        <beans:property name="lineMapper">
            <beans:bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
                <beans:property name="lineTokenizer">
                    <beans:bean class="com.mypackage.MyCustomLineTokenizer">
                        <beans:property name="delimiter" value="||"/>
                        <beans:property name="names" value="one,two,three,four" />
                    </beans:bean>
                </beans:property>
                <beans:property name="fieldSetMapper">
                    <beans:bean class="com.mypackage.MyFieldSetMapper" />
                </beans:property>
            </beans:bean>
        </beans:property>
    </beans:bean>

Progressed a bit. But still not hitting the answer.

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.UnsupportedEncodingException;

import org.springframework.batch.item.file.BufferedReaderFactory;
import org.springframework.core.io.Resource;


public class MyBufferedReaderFactory implements BufferedReaderFactory {


    public BufferedReader create(final Resource resource, final String encoding) throws UnsupportedEncodingException, IOException {

        InputStream sourceStream = null;
        sourceStream = resource.getInputStream();

        //code to remove trailing spaces end

        //code to remove trailing spaces start

        return new BufferedReader(new InputStreamReader(sourceStream, encoding));

    }

}

How can I remove the trailing spaces using InputStream object in above class ??
Thanks for reading!!

Answer 1

Extend BufferedReader by implementing BufferedReaderFactory and add line-trimming functionality to the reader. See FlatFileItemReader.setBufferedReaderFactory().

public class LineTrimmingBufferedReader extends BufferedReader
{
    public LineTrimmingBufferedReader(Reader in)
    {
        super(in);
    }

    @Override
    public String readLine() throws IOException
    {
        String result = super.readLine();
        return result != null ? result.trim() : null;
    }
}

Implement proper trailing space trimming of the line if String.trim() isn’t acceptable.