I’m trying to extract some data from a craigslist HTML page, but I seem to be running into a strange bug- every once in a while, the page I try to load with an NSURLRequest comes back as some strange form of data, which when converted to a parseable string, returns null. However, I can’t consistently reproduce it- it’ll suddenly stop working, and then I’ll try it again an hour later and it’ll be working perfectly, and then some time later it’ll stop working again. Anyone know what could be causing it? I’m using an NSURLRequest, asynchronous, with the ‘didReceiveData’ and ‘didReceiveResponse’ delegate methods. If I cast the NSURLResponse to an NSHTTPURLResponse and check the response code, I get 200, meaning there were no issues. But when I go to initialize a string with the response data, it returns null, and I obviously then can’t parse it.
The URL that seems to do it most often is: http://sarasota.craigslist.org/app/
I’ve tried messing with the User-Agent header for the request, the cache policy, everything I can think of… but nothing seems to fix it.
If there is data but when you ask for a string it’s null then I might suspect the string encoding you’re using to decode the data? Is there an odd character that is only sometimes in craigs list advert titles?
Just out of interest, why don’t you use the rss feed instead – it’s probably more consistently formatted / strictly encoded as it’s xml not html.