Question

I have a flat file that is pipe delimited and looks something like this as example

ColA|ColB|3*|Note1|Note2|Note3|2**|A1|A2|A3|B1|B2|B3

The first two columns are set and will always be there.

* denotes a count for how many repeating fields there will be following that count so Notes 1 2 3

** denotes a count for how many times a block of fields are repeated and there are always 3 fields in a block.

This is per row, so each row may have a different number of fields.

Hope that makes sense so far.

I’m trying to find the best way to parse this file, any suggestions would be great.

The goal at the end is to map all these fields into a few different files – data transformation. I’m actually doing all this within SSIS but figured the default components won’t be good enough so need to write own code.

UPDATE I’m essentially trying to read this like a source file and do some lookups and string manipulation to some of the fields in between and spit out several different files like in any normal file to file transformation SSIS package.

Using the above example, I may want to create a new file that ends up looking like this

“ColA”,”HardcodedString”,”Note1CRLFNote2CRLF”,”ColB”

And then another file

Row1: “ColA”,”A1″,”A2″,”A3″

Row2: “ColA”,”B1″,”B2″,”B3″

So I guess I’m after some ideas on how to parse this as well as storing the data in either Stacks or Lists or?? to play with and spit out later.

Answer 1

One possibility would be to use a stack. First you split the line by the pipes.

var stack = new Stack<string>(line.Split('|'));

Then you pop the first two from the stack to get them out of the way.

stack.Pop();
stack.Pop();

Then you parse the next element: 3* . For that you pop the next 3 items on the stack. With 2** you pop the next 2 x 3 = 6 items from the stack, and so on. You can stop as soon as the stack is empty.

while (stack.Count > 0)
{
    // Parse elements like 3*
}

Hope this is clear enough. I find this article very useful when it comes to String.Split().