Question

I need advise regarding the pattern to use for the following problem.

There are many rows — let us call it messages(identified by MSG_ID in DB)– in a table which corresponds to a file. Means, the file has been split into many pieces and put into database.

So parts corresponding to file can be identified using a GROUP_ID column, and MSG_ID corresponds to individual message. The primary key is a combination of GROUP_ID and MSG_ID.

Now, each message consists of n number of logical records(which are typically payment instructions(k x 128 bytes of data)). Where current reading payment instruction ends can be said only after reading the next 128 characters. Also parts of a payment instruction can be in consecutive messages. Which means a complete payment instruction can be spread across end of MSG_ID n and start of MSG_ID n+1.

We are using spring batch to do the processing.

I have tried querying the table and writing all the records to a flat file one by one and start the spring batch from there.

I would like to know whether there’s any pattern which I can use to achieve the requirement without using a flat file.

Like,

1. Read MSG_ID 1 and GROUP_ID “ABC” from db

2. Seperate the payment instructions and give each instruction to the processor.

3. When end of MSG_ID 1 is reached check whether the final record in hand form a Payment Instruction, if not read MSG_ID 2 and GROUP_ID “ABC” and append to the previous left over record.

4. Read till MSG_ID==N where N is known before starting the read process.

5. Is there any ItemReaders in Spring or Iterator patterns in java which I can use ?

To be more clear, there are patterns for handling, “IF your logical record is spread in multiple rows”. Is there any pattern for “IF one row in DB contains ‘M’ number logical records, where M may not be an integer, use this type of Iterator or ItemReaders”

Thanks.

Answer 1

I believe you should write a custom Reader (processing logic) for your scenario. It certainly doesn’t seem as a common case.

The algorithm you proposed seems OK. You should have no trouble writing a Reader which reads a complete payment instruction and hands it off for further processing.