Question

Let’s say we want to do some substitutions only between some patterns, let them be <a> and </a> for clarity… (all right, all right, they’re start and end!.. Jeez!)

So I know what to do if start and end always occur on the same line: just design a proper regex.

I also know what to do if they’re guaranteed to be on different lines and I don’t care about anything in the line containing end and I’m also OK with applying all the commands in the line containing start before start: just specify the address range as /start/,/end/.

This, however, doesn’t sound very useful. What if I need to do a smarter job, for instance, introduce changes inside a {...} block?

One thing I can think of is breaking the input on { and } before processing and putting it back together afterwards:

sed 's/{\|}/\n/g' input | sed 'main stuff' | sed ':a $!{N;ba}; s/\n\(}\|{\)\n/\1/g'

Another option is the opposite:

cat input | tr '\n' '#' | sed 'whatever; s/#/\n/g'

Both of these are ugly, mainly because the operations are not confined within a single command. The second one is even worse because one has to use some character or substring as a “newline holder” assuming it isn’t present in the original text.

So the question is: are there better ways or can the above-mentioned ones be optimized? This is quite a regular task from what I read in recent SO questions, so I’d like to choose the best practice once and for all.

P.S. I’m mostly interested in pure sed solutions: can the job be do with one invocation of sed and nothing else? Please no awk, Perl, etc.: this is more of a theoretical question, not a “need the job done asap” one.

Answer 1

This might work for you:

# create multiline test data
cat <<\! >/tmp/a
> this
> this { this needs
> changing to
> that } that
> that
> !
sed '/{/!b;:a;/}/!{$q;N;ba};h;s/[^{]*{//;s/}.*//;s/this\|that/\U&/g;x;G;s/{[^}]*}\([^\n]*\)\n\(.*\)/{\2}\1/' /tmp/a
this
this { THIS needs
changing to
THAT } that
that
# convert multiline test data to a single line
tr '\n' ' ' </tmp/a >/tmp/b
sed '/{/!b;:a;/}/!{$q;N;ba};h;s/[^{]*{//;s/}.*//;s/this\|that/\U&/g;x;G;s/{[^}]*}\([^\n]*\)\n\(.*\)/{\2}\1/' /tmp/b
this this { THIS needs changing to THAT } that that

Explanation:

• Read the data into the pattern space (PS). /{/!b;:a;/}/!{$q;N;ba}
• Copy the data into the hold space (HS). h
• Strip non-data from front and back of string. s/[^{]*{//;s/}.*//
• Convert data e.g. s/this\|that/\U&/g
• Swap to HS and append converted data. x;G
• Replace old data with converted data.s/{[^}]*}\([^\n]*\)\n\(.*\)/{\2}\1/

EDIT:

A more complicated answer which I think caters for more than one block per line.

# slurp file into pattern space (PS)
:a
$! {
N
ba
}
# check for presence of \v if so quit with exit value 1
/\v/q1
# replace original newlines with \v's
y/\n/\v/
# append a newline to PS as a delimiter
G
# copy PS to hold space (HS)
h
# starting from right to left delete everything but blocks
:b
s/\(.*\)\({.*}\).*\n/\1\n\2/
tb
# delete any non-block details form the start of the file
s/.*\n//
# PS contains only block details
# do any block processing here e.g. uppercase this and that
s/th\(is\|at\)/\U&/g
# append ps to hs
H
# swap to HS
x
# replace each original block with its processed one from right to left
:c
s/\(.*\){.*}\(.*\)\n\n\(.*\)\({.*}\)/\1\n\n\4\2\3/
tc
# delete newlines
s/\n//g
# restore original newlines
y/\v/\n/
# done!

N.B. This uses GNU specific options but could be tweaked to work with generic sed’s.