Question

I use this piece of code in my bash script to read a file containing several hex strings, do some substitution and then write it to a new file. It takes about 30 minutes for about 300 Mb.
I’m wondering if this can be done faster ?

sed 's,[0-9A-Z]\{2\},\\\\x&,g' ${in_file} | while read line; do
 printf "%b" ${line} >> ${out_file}
 printf '\000\000' >> ${out_file}
done

Update:

I did some testing and got the following results:

The winner is:

---

sed 's,[0-9A-Z]\{2\},\\\\x&,g' ${in_file} | while read line; do
    printf "%b" ${line} >> ${out_file}
    printf '\000\000' >> ${out_file}
done

real 44m27.021s
user 29m17.640s
sys 15m1.070s

---

sed 's,[0-9A-Z]\{2\},\\\\x&,g' ${in_file} | while read line; do
    printf '%b\000\000' ${line} 
done >> ${out_file}

real 18m50.288s
user 8m46.400s
sys 10m10.170s

---

export LANG=C
sed 's/$/0000/' ${in_file} | xxd -r -ps >> ${out_file}

real 0m31.528s
user 0m1.850s
sys 0m29.450s

---

Answer 1

You need xxd command that comes with Vim.

export LANG=C
sed 's/$/0000/' ${in_file} | xxd -r -ps > ${out_file}