Question

I wish to split a large file (with ~ 17M Lines of Strings) into multiple files with varying number of lines in each chunk. Would it be possible to send in an array to the ‘split -l‘ command like this:

[
 1=>1000000,
 2=>1000537,
 ...
]

so as to send those many number of lines to each chunk

Answer 1

Use a compound command:

{
  head -n 10000 > output1
  head -n   200 > output2
  head -n  1234 > output3
  cat > remainder
} < yourbigfile

This also works with loops:

{
  i=1
  for n in 10000 200 1234
  do
      head -n $n > output$i
      let i++
  done
  cat > remainder
} < yourbigfile

This does not work on OS X, where head reads and discards additional output.