Question

I have a file with format like :

[PATTERN]
line1
line2
line3
.
.
.
line
[PATTERN]
line1
line2
line3
.
.
.
line
[PATTERN]
line1
line2
line3
.
.
.
line

I want to extract the following blocks from above file :

[PATTERN]
line1
line2
line3
.
.
.
line

Note: Number of lines between 2 [PATTERN] may varies, so can’t rely on number of lines.

Basically, I want to store each pattern and the lines following it to Database, so I wil have to iterate all such blocks in my file.

How do this with Shell Scripting ?

Answer 1

This assumes you are using bash as your shell. For other shells, the actual solution can be different.

Assuming your data is in data:

i=0 ; cat data  | while read line ; do \
  if [ "$line" == "[PATTERN]" ] ; then \
    i=$(($i + 1)) ; touch file.$i ; continue ; \
  fi ; echo "$line" >> file.$i ; \
done

Change [PATTERN] by your actual separation pattern.

This will create files file.1, file.2, etc.

Edit: responding to request about an awk solution:

awk '/^\[PATTERN\]$/{close("file"f);f++;next}{print $0 > "file"f}' data

The idea is to open a new file each time the [PATTERN] is found (skipping that line – next command), and writing all successive lines to that file. If you need to include [PATTERN] in your generated files, delete the next command.

Notice the escaping of the [ and ], which have special meaning for regular expressions. If your pattern does not contain those, you do not need the escaping. The ^ and $ are advisable, since they tie your pattern to the beginning and end of line, which you will usually need.