Question

I have a Spring Batch job consisting of two steps (so far).

The first step in the job is implemented in a tasklet. It takes the comma-separated value (CSV) files that need to be processed (using a good chunk of business logic to determine which ones those are), and copies them into a “drop zone” directory.

The second step is configured for chunk-oriented processing, with a reader and writer. The reader is a MultiResourceItemReader… which looks for all CSV files in the drop zone directory, and delegates each to the “real” reader (which parses the CSV).

My problem is that even though the first step succeeds, the second step fails to find any CSV files in the drop zone directory. Interestingly, if I immediately run the batch job again… then the second step does find and process the files!

I am speculating, but it looks like Spring Batch resolves the second step’s wildcard pattern at the outset… rather than waiting until it’s time for the second step to run. Even though the first step copies the files that it is supposed to, the second step has already decided that there are no files there.

I am fairly new to Spring Batch, and still learning my way around. Is there something obvious with context or scope that I am missing here? The relevant portions of my job definition are below. Thanks!

...
<!-- JOB DEFINITION -->
<job id="notificationJob" xmlns="http://www.springframework.org/schema/batch">
    <step id="copyFilesToLocal">
        <tasklet transaction-manager="jobRepositoryTransactionManager" ref="getFilesTasklet" />
        <next on="COMPLETED" to="processFiles"/>
    </step>
    <step id="processFiles">
        <tasklet transaction-manager="ecommerceTransactionManager">
            <chunk reader="multiFileReader" writer="notificationEmailWriter" commit-interval="1" />
        </tasklet>
    </step>
</job>

<!-- FIRST STEP -->
<bean id="getFilesTasklet" class="com.mypackage.FileMovingTasklet">
    <property name="localDao">
        <bean class="com.mypackage.BatchLocalDao">
            <property name="dataSource" ref="jobRepositoryDataSource" />
        </bean>
    </property>
    <property name="sourceDirectory">
        <bean id="sourceDirectory" class="org.springframework.core.io.FileSystemResource">
            <constructor-arg value="/mnt/source-directory" />
        </bean>
    </property>
    <property name="destinationDirectory">
        <bean id="destinationDirectory" class="org.springframework.core.io.FileSystemResource">
            <constructor-arg value="/home/myuser/drop-zone" />
        </bean>
    </property>
</bean>


<!-- SECOND STEP -->
<bean id="multiFileReader" class="org.springframework.batch.item.file.MultiResourceItemReader">
    <property name="resources" value="file://home/myuser/drop-zone/*.csv" />
    <property name="delegate" ref="myFileReader" />
</bean>
...

Answer 1

Your “multiFileReader” needs to be scope=”step”, so it defers the pattern expansion (initialization of bean properties) until the step is executed.