Question

Good day,

I am writing a relatively simple BASH script that performs an SVN UP command, captures the console output, then does some post processing on the text.

For example:

#!/bin/bash
# A script to alter SVN logs a bit

# Update and get output
echo "Waiting for update command to complete..."
TEST_TEXT=$(svn up --set-depth infinity)
echo "Done"

# Count number of lines in output and report it
NUM_LINES=$(echo $TEST_TEXT | grep -c '.*')
echo "Number of lines in output log: $NUM_LINES"

# Print out only lines containing Makefile
echo $TEST_TEXT | grep Makefile

This works as expected (ie: as commented in the code above), but I am concerned about what would happen if I ran this on a very large repository. Is there a limit on the maximum buffer size BASH can use to hold the output of a console command?

I have looked for similar questions, but nothing quite like what I’m searching for. I’ve read up on how certain scripts need to use the xargs in cases of large intermediate buffers, and I’m wondering if something similar applies here with respect to capturing console output.

eg:

# Might fail if we have a LOT of results
find -iname *.cpp | rm

# Shouldn't fail, regardless of number of results
find -iname *.cpp | xargs rm

Thank you.

Answer 1

Using

var=$(hexdump /dev/urandom | tee out)

bash didn’t complain; I killed it at a bit over 1G and 23.5M lines. You don’t need to worry as long as your output fits in your system’s memory.