Today I ran a command like this in the emacs shell:
./someBinary | grep foo | cut -c30- | sort | uniq -c
which in bash takes a bit but not that long (about 15 seconds) because the output is over a million lines easily. When I ran this command in the emacs shell, however, I waited over an hour and it’s still running, with the processes visibly doing work if i check with top. I wonder if this is because emacs implements the unix tools I am piping to in lisp – and if this is the reason, if there’s a way to have it default to the system ones.
Emacs is capturing the final output in the shell buffer, and applying font-lock and other analysis (line number counting, for example) to display it. It also scrolls the display to show the latest output. While Emacs has provisions for culling pathologically long command output, it’s not really optimized for truly huge quantities of output counting in millions of lines, so it performs visibly worse than your terminal emulator, slowing down the whole pipeline.
If you’re not interested in the output, redirect it to
/dev/nullor totail -500, which is how much you’d see of it in a typical terminal scrollback anyway.