Question

I’m a newbie in Spring Batch. I have inherited a batch process implemented with Spring Batch.
This works well, except for one thing I’ll try to describe.

I launch parseJob and, when it’s reading XML to process in bean parsingStepReader,
read() method is been invoking always.

The directory *path_to_xml* contains only one XML, invoke read() and return XML parsed, which is processed OK. Then, read() method is invoked again, return a null object, and is invoked again, return null… and so on.

When debugging, MultiResourceItemReader read method try to read, does not read anything (all resources has already been readed), increment currentResources and return null.

I have readed something about the job stops when the reader return a null object, but that read method returns null and reads again and again…
I changed restartable to false, but does not work.

The job is launched in Linux, batch mode, with org.springframework.batch.core.launch.support.CommandLineJobRunner
Because of this problem, the .sh that launch the job does not finish, and resources are busy.

How can I avoid this, or stop the job when resources (XML) input directory have already been processed?
Any help would be very appreciated. Best regards.

Beans file and Java class pieces are attached

<batch:job id="parseJob" restartable="true" incrementer="jobParametersIncrementer">
    <batch:flow parent="parseFlow"/>
    <batch:flow .../>
    <batch:flow .../>
</batch:job>

<batch:flow id="parseFlow">
    <batch:step id="parsingStep">
        <batch:tasklet start-limit="100" allow-start-if-complete="true" transaction-manager="..." task-executor="taskExecutor" throttle-limit="$...">
            <batch:chunk reader="parsingStepReader" writer="..." processor="..." commit-interval="..." skip-limit="10000000">
                <batch:skippable-exception-classes>
                    <batch:include class="java.lang.Exception" />
                </batch:skippable-exception-classes>
            </batch:chunk>
            <batch:listeners>
                <batch:listener ref="iwListener" />
                <batch:listener ref="mySkipListener" />
                <batch:listener ref="myStep1Listener" />
            </batch:listeners>
            <batch:no-rollback-exception-classes>
                <batch:include class="java.lang.Exception" />
            </batch:no-rollback-exception-classes>
        </batch:tasklet>
    </batch:step>
</batch:flow>

<!-- -->

<bean id="bpfReader" class="org.springframework.batch.item.xml.StaxEventItemReader" scope="prototype">
    <property name="fragmentRootElementName" value="..." />
    <property name="unmarshaller" ref="..." />
    <property name="strict" value="false" />
</bean>

<bean id="multiresourceItemReader" class="...SyncMultiResourceItemReader" abstract="true">
    <property name="strict" value="false" />
    <property name="delegate" ref="bpfReader" />
</bean>

<bean id="parsingStepReader" parent="multiresourceItemReader" scope="step">
    <property name="resources" value="<path_to_xml>" />
</bean>

And the reader class is:

public class SyncMultiResourceItemReader<T> extends MultiResourceItemReader<T> {
    . . .

    @Override
    public T read() throws Exception, UnexpectedInputException, ParseException {
        synchronized (this) {
            return super.read();
        }
    }

    . . .
}

UPDATE: Solution suggested by @vsingh works perfectly. Once an input element is chosen, it must be removed from the input. I don’t know why, but class org.springframework.batch.item.file.MultiResourceItemReader does not work as I expected, especially in an input error.

I hope this helps. Best regards

Answer 1

The read method will read the data , store at class level and pass it to the write method.
I will give you an example of how we did it

for eg

@Override
public Long read() throws Exception, UnexpectedInputException,
        ParseException, NonTransientResourceException {
    synchronized (this.myIds) {
        if (!this.myIds.isEmpty()) {
            return this.myIds.remove(0);
        }
        return null;
    }
}

myIds is a List at class level

This list is populated at before step method

@Override
public void beforeStep(final StepExecution stepExec) {
    this.stepExecution = stepExec;
            // read the ids from service and set at class level  

}