Question

When I run this command in the bash terminal it works:

awk '{ sum += $1 } END { print sum }' /user/dnaiel/test.txt > /user/dnaiel/sum.txt

But when I run this:

bsub -q priority -R'rusage[mem=2000]' -oo bin${count}.out -eo bin${count}.err \
"awk '{ sum += $1 } END { print sum }' /user/dnaiel/test.txt > /user/dnaiel/sum.txt"

It does not work. I also tried changing ‘ to \’ but also does not work.

I get the following errors:
for the first case:

awk: { sum +=  } END { print sum }
awk:           ^ syntax error

for the case I used \’

awk: '{
awk: ^ invalid char ''' in expression

Any ideas where I am messing up with the syntax? I am quite puzzled.

Thanks

Answer 1

notice how the $1 has disappeared in the error message?

awk: { sum +=  } END { print sum }
awk:           ^ syntax error

This is because in shell, when you quote something FIRST with dbl-quotes, as you have done with

bsub -q priority -R'rusage[mem=2000]' -oo bin${count}.out -eo bin${count}.err \
"awk '{ sum += $1 } END { print sum }' /user/dnaiel/test.txt > /user/dnaiel/sum.txt"

any ${var} references are expanded to their value. The single-quotes have lost their magic power to prevent variable expansion when they are inside a dbl-quoted string.

How to fix, escape your $s. not sure what bsub is, but this should do it:

bsub -q priority -R'rusage[mem=2000]' -oo bin${count}.out -eo bin${count}.err \
"awk '{ sum += \$1 } END { print sum }' /user/dnaiel/test.txt > /user/dnaiel/sum.txt"
# -------------^^^

IHTH