Question

I have been working on operations with strings in a recent 100 level CompSci course.
I got the very “original” idea that I might write up a simple domain name generator/checker.

So I did a little homework and discovered that the various whois servers understandably limit the number of queries allowed.

So, I decided to first check for a DNS boolean. If no records are found I then check a MySQL database to make sure the same query hasn’t been sent recently. If it hasn’t I fire off a whois query with PHP using fsockopen. So, I was just getting ready to finish up my little script and upload it from my development server to my production server and I found some sites suggesting that various whois servers limit the queries to only 1,000.

My question:

Am I approaching this appropriately?
The simple math suggests that only 10 users each checking out 10 searches each search providing only 10 results (10**3) might result in exceeding the limit and a temporary ban.

Are there any methods of doing bulk queries to the whois server?

Are other sites using some form of client-side javascript query or server-side proxy?
I found another similar question here at stackoverflow suggesting that *NIX systems have access to a terminal command that has no limits. Other questions I have found deal with parsing the data – which is not a concern of mine.

I understand that this is a vague question. I do not want to inappropriately burden the whois servers. I do not expect, nor want, a ready-made code solution. A basic discussion of alternative programmatic strategies to go about this would make me a very satisfied friend 🙂 Anyone have a keyword or two with which I can continue my research?

Answer 1

The whois unix command appears to be less limited (https://superuser.com/questions/452751/what-are-the-limits-of-whois-command-on-unix). It might be easiest to do what I assume whois is doing under the covers and open a tcp connection to whois.internic.net on port 43.

<?php

$fp = fsockopen("whois.internic.net", 43);
fwrite($fp, "hello.com\n");

$response = "";
while (!feof($fp)) {
    $response .= fread($fp, 8192);
}

fclose($fp);
echo $response;

?>

If that’s what you’re already doing, then that’s probably your best bet. I’m guessing a 1,000 query limit likely refers to the use of somebody’s web service that does this for you (e.g. whois.com). I think you can make a lot more queries than that if you’re doing what I showed above.

(I’ve made a lot of guesses and assumptions here.)

P.S. A lot of good info here: http://semmyfun.blogspot.com/2010/08/how-does-whois-work-dirty-guide.html