In my spare time, I have been working on implementing a BitTorrent client in C. Currently it communicates with the tracker, connects to the swarm, requests pieces of the torrent file from peers, and receives pieces of the torrent file. However, when it comes to verifying that the received piece is correct (by taking a SHA1 hash and comparing it to the hash provided in the .torrent metadata), it always fails.
To debug this, I downloaded a torrent with a known-working BitTorrent client, and then modified my own BitTorrent implementation to request and download only the very beginning of the torrent (the first piece). I then compared the two files with Emacs’ hexl-mode.
Known good:
00000000: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000010: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000020: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000030: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000040: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000050: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000060: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000070: 0000 0000 0000 0000 0000 0000 0000 0000 ................
...
00008000: 0143 4430 3031 0100 4c49 4e55 5820 2020 .CD001..LINUX
00008010: 2020 2020 2020 2020 2020 2020 2020 2020
00008020: 2020 2020 2020 2020 5562 756e 7475 2031 Ubuntu 1
00008030: 312e 3034 2069 3338 3620 2020 2020 2020 1.04 i386
My implementation:
00000000: a616 f132 7f00 0080 5066 0000 0000 0080 ...2....Pf......
00000010: 5066 0000 0000 0060 3b62 0000 0000 0098 Pf.....`;b......
00000020: 3b62 0000 0000 00d0 3b62 0000 0000 0008 ;b......;b......
00000030: 3c62 0000 0000 0040 3c62 0000 0000 0078 <b.....@<b.....x
00000040: 3c62 0000 0000 00b0 3c62 0000 0000 00e8 <b......<b......
00000050: 3c62 0000 0000 0020 3d62 0000 0000 0058 <b..... =b.....X
00000060: 3d62 0000 0000 0090 3d62 0000 0000 00c8 =b......=b......
00000070: 3d62 0000 0000 0000 3e62 0000 0000 0038 =b......>b.....8
...
0000d000: 0243 4430 3031 0100 004c 0049 004e 0055 .CD001...L.I.N.U
0000d010: 0058 0020 0020 0020 0020 0020 0020 0020 .X. . . . . . .
0000d020: 0020 0020 0020 0020 0055 0062 0075 006e . . . . .U.b.u.n
0000d030: 0074 0075 0020 0031 0031 002e 0030 0034 .t.u. .1.1...0.4
0000d040: 0020 0069 0033 0038 0000 0000 0000 0000 . .i.3.8........
I figured, then, that I must be writing the received piece to the incorrect offset, resulting in the correct data occuring at the wrong location in the file. To verify this, I fired up gdb and inspected the very beginning of the first piece after receiving it from a peer, expecting it to contain all zeroes, like the beginning of the known-good file.
(gdb) break network.c:40
Breakpoint 1 at 0x402fe7: file network.c, line 40.
(gdb) run
Starting program: /home/robb/slug/slug
[Thread debugging using libthread_db enabled]
[New Thread 0x7fffcb58d700 (LWP 12936)]
[Thread 0x7fffcb58d700 (LWP 12936) exited]
ANNOUNCE: 50 peers.
CONNECTED: 62.245.41.28
CONNECTED: 89.178.142.45
CONNECTED: 66.65.166.17
...
UNCHOKE: 95.26.0.1
Requested piece 0 from peer 95.26.0.1.
UNCHOKE: 202.231.116.163
PIECE: #0 from 95.26.0.1
Breakpoint 1, handle_piece (p=0x42d7e0) at network.c:41
41 memcpy(p->torrent->mmap + length, &p->message[9], REQUEST_LENGTH);
(gdb) p off
$1 = 0
(gdb) p index
$2 = 0
(gdb) p p->message[9]
$3 = 46 '.'
(gdb) p p->message[10]
$4 = 67 'C'
(gdb) p p->message[11]
$5 = 0 '\000'
(gdb) p p->message[12]
$6 = 0 '\000'
(gdb) p p->message[13]
$7 = 0 '\000'
(gdb) p p->message[14]
$8 = 0 '\000'
(gdb) p p->message[15]
$9 = 0 '\000'
(gdb) p p->message[16]
$10 = 128 '\200'
(gdb) p p->message[17]
$11 = 46 '.'
(gdb) p p->message[18]
$12 = 67 'C'
As you can see, the data I received from the peer doesn’t contain all zeroes like the beginning of the known-good file. Why?
The full source of my program is availabe at https://github.com/robertseaton/slug.
This fails to take into account that bufferevent_read may fail and return a negative amount:
Replace with: