I am writing a web-spider program with C. Now I am given a url’s list, and first I need to get the server IP address using function: getaddrinfo, and then ridiculous thing happended:
In the url’s list there are about 4,000,000 url’s, the first about 6,000 url’s are processed very well, and then suddenly all of the url’s behind failed! getaddrinfo returns “temporary failure in name resolution” for every url. In addition, if I restart program from the first ‘bad’ url, it work again.
I am really confused and stuck for 2 days, I felt that the DNS is working well, but some limited resources have been used up, can any one give me some suggestions?
Are you calling
freeaddrinfo()on the returned address information? Very basic, but since you’re not showing your code it’s the first theory that comes to mind.