Our site just rolled a new version, and now pages have Unicode in the url. I see that Rails have properly url escaped these UTF-8 characters when rendering the anchor tags.
/regions/%E4%B8%AD%E5%BD%B0%E6%8A%95/
However I still see a lot of traffic with incorrectly encoded urls:
/regions/%A4%A4%B9%FC%A7%EB/
Apparently this is the same address, but encoded in something other than UTF-8, and then url escaped.
Question
I am wondering if there is any old browser, which will take correctly escaped url, unescape it to get UTF-8, encode it in some other encoding, and then url-escape it when requesting the server?
Otherwise I don’t know how to explain these traffic.
I have tested in Internet Explorer 6 and 7. I also tested the “Always send URLs as UTF-8” option. None of the combination caused incorrect encoded request.
I am guessing this was by some web crawler which handles the decoding but not encoding.