Question

I am doing push using mercurial hg to bitbucket.org using https.

There is a bunch of changes to text files and also binary files added. So I would like to capture the real traffic of this command in http format to analyze. How can I make it? Or at least inspecting an example of captured human-readable push would be great.

There is a link for mercurial wire protocol, but no example how it might really look.

Answer 1

There are a couple of proxies (http://mitmproxy.org/ is popular, I really like http://www.charlesproxy.com/) which can MITM the HTTPS connection… However, it might be simpler to start a local Mercurial server, then sniff that connection:

$ cd some-hg-repo/
$ hg serve
… listening at http://127.0.0.1:8000/ …

Then fire up your packet sniffer watching on the loopback interface, and from another shell:

% hg clone http://127.0.0.1:8000/ repo-clone
% cd repo-clone
% fortune > foo.c
% hg commit -m "change to foo"
% hg push

And here’s a bit of what it looks like:

$ sudo tcpdump -i lo0 -A 'tcp port 8000 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)'
03:15:05.515867 IP localhost.52031 > localhost.irdmi: Flags [P.], seq 2116430132:2116430284, ack 835526317, win 40830, options [nop,nop,TS val 269453377 ecr 269453377], length 152
E....2@.@............?.@~&)41......~.......
...A...AGET /?cmd=capabilities HTTP/1.1
Accept-Encoding: identity
host: localhost:8000
accept: application/mercurial-0.1
user-agent: mercurial/proto-1.0
… snip …
03:15:05.516780 IP localhost.irdmi > localhost.52031: Flags [P.], seq 173:303, ack 152, win 40830, options [nop,nop,TS val 269453378 ecr 269453378], length 130
E...8b@.@............@.?1..Y~&)....~.......
...B...Blookup changegroupsubset branchmap pushkey known getbundle unbundlehash batch stream unbundle=HG10GZ,HG10BZ,HG10UN httpheader=1024

GET /?cmd=batch HTTP/1.1
Accept-Encoding: identity
x-hgarg-1: cmds=heads+%3Bknown+nodes%3D
host: localhost:8000
vary: X-HgArg-1
accept: application/mercurial-0.1
user-agent: mercurial/proto-1.0

…snip…

03:15:05.528852 IP localhost.irdmi > localhost.52033: Flags [P.], seq 474:516, ack 355, win 40830, options [nop,nop,TS val 269453389 ecr 269453389], length 42
E..^.h@.@............@.A.W...,.....~.R.....
...M...M92550c48fd2dc2c112ac88215eff29a5012abff1
;
03:15:05.529756 IP localhost.52033 > localhost.irdmi: Flags [P.], seq 355:628, ack 516, win 40830, options [nop,nop,TS val 269453390 ecr 269453389], length 273
E..E.N@.@............A.@.,...W.....~.9.....
...N...MGET /?cmd=getbundle HTTP/1.1
Accept-Encoding: identity
x-hgarg-1: common=0000000000000000000000000000000000000000&heads=92550c48fd2dc2c112ac88215eff29a5012abff1
host: localhost:8000
vary: X-HgArg-1
accept: application/mercurial-0.1
user-agent: mercurial/proto-1.0

…snip…

03:15:05.535163 IP localhost.irdmi > localhost.52033: Flags [P.], seq 688:6194, ack 628, win 40830, options [nop,nop,TS val 269453395 ecr 269453394], length 5506
E...AZ@.@............@.A.W...,.....~.......
........]..>O..3.x....L .-.....I.mh....M}.i!..Bh8.PL.   .O
1iB ...C.....4.4....:...H..w....7.\..#.{.p.......-g.....^u....5...H...MWu.@....c.C4Y^19QP....l.....1.d.ukh.5..M.....k.A..<'.2..,.2.......{.q.(?.....rc"._.........m.xx.';...]V_0..e..j..{....OWf.n........J.bZ&kVXAR4...!....*..J.b..x.....@.Y..P........e.i;#....c.F..._.m.a|. .........=.

… snip …