Question

A little background first – When I do apt-get install downloads from my company internet it provides a high burst of speed (400-500KB/s) for the first 10 seconds or so before dropping down to a tenth of that (40-50KB/s), and then after a few minutes to a truly miserable (4-5KB/s). This makes me think that the sysadmin has implemented some sort of a network throttling scheme.

Now I know that the network is not simply erratic, because if I start an apt-get install foo, Ctrl-C it after 10 seconds and immediately run apt-get install foo again (by doing an up arrow and enter to use bash history), and then keep repeating this process for a few minutes till all packages are downloaded, I can download even large packages very fast. In particular, even after aborting a download with Ctrl-C, apt-get seems to be able to resume the download in the next invocation.

Of course, staring at the screen doing Ctrl-C Up Enter every 10 seconds gets really boring real fast, so I wrote a shell script –

#!/bin/sh
for i in `seq 1 100` ; do
    sudo apt-get install foo -y &
    sleep 10
    sudo kill -2 $!
done

This seems to work. It spawns apt-get, runs it for 10 seconds and then kills (by sending a SIGINT) it and starts it up again. However, it doesn’t really work because now apt-get does not resume downloads on subsequent invocations!

An an experiment I ran sudo apt-get install foo from one terminal and then ran kill -2 <PID of apt-get> from another terminal. And even in that case, when I restart apt-get, it does not resume the download.

So clearly a Ctrl-C is not equivalent to SIGINT. And something else is happening when I do Ctrl-C manually which gives apt-get a chance to save the state of the download. The question is – what is it?

Edit

These are the suggestions I have received so far, but no cigars. The mystery deepens! –

1. On sudo kill -2 $! the signal might be going to sudo instead of apt-get. This is not the reason because as mentioned above I also tried sending SIGINT specifically to apt-get’s PID and even that prevented apt-get from saving its state.

2. Sudo catches the signal and sends some other signal to apt-get. I tried sending apt-get all the signals I can think of! It still does not resume the download for any of them. It only resumes downloads when I do Ctrl-C to kill it.

3. Apt-get handles SIGINT differently if it is from a script instead of an interactive shell. Again, the “experiment” above proves that this is not true.

Answer 1

Okay mystery solved! Thanks to the helpful folks over at the Indian Linux Users Group.

The answer here is two-fold –

Firstly, apt-get invokes another program called http for downloading data.

[~] ➔ file /usr/lib/apt/methods/http

/usr/lib/apt/methods/http: ELF 32-bit LSB executable, Intel 80386, version 1
(SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.15,
stripped

Note that that’s an executable, not even a script, probably to support downloading files during system installation when none of perl/python/ruby etc. are yet available.

Secondly, when you press Ctrl-C after running apt-get, the SIGINT gets sent to http, and not to apt-get. When http receives the SIGINT, it saves the download state before shutting down.

Here’s the updated script that works perfectly –

#!/bin/sh
for i in `seq 1 100` ; do
    sudo apt-get install foo -y &
    sleep 10
    sudo kill -2 `ps -ae | grep " http" | awk '{print $1}'`
done