Question

I’m currently working on a project in which access to an API is restricted to registered users. The API itself is already finished and works as expected. Limiting access to the API has turned out fairly straightforward as well. However, my problem (or question, rather) is how to go about ensuring the efficiency of the database interactions for the registration, verification, and/or lost and found process.

Here’s an example of what currently happens:

1. User requests an API key by entering their email address
2. User is sent a verification email
3. User clicks link in email and php checks hash against database
4. Once hash is verified, API key is generated, stored, and emailed
5. If user forgets/loses API key, it can be emailed again
6. If verification email wasn’t received, it can be emailed again

Here’s an example of the database structure:
http://s13.postimage.org/h8ao5oo2v/dbstructure.png

As you can probably imagine, there is a LOT of database interaction going on behind the scenes for each of these particular steps in the process. One step that I’m wondering about the efficiency of is that of checking uniqueness of certain items. Obviously, we don’t want any duplicate API keys floating around, nor do we want any duplicate email verification hashes.

So, I wrote a dead simple function that checks the database for these things before inserting them into the database. However, this project is on the order of hundreds of times larger than any I’ve undertaken before. I’ve built and maintained projects that serviced 500 – 1,000 users before… but this project is estimated to be servicing a minimum of around 50,000 users daily. I’m extremely happy that I’ve finally landed a large project, but becoming increasingly daunted at the scale of it.

At any rate, here’s the function I wrote to interact with the database to check uniqueness of items before storing them.

function isUnique($table, $col, $data) {
  mysql_connect("localhost", "root", "") or die(mysql_error());  
  mysql_select_db("api") or die(mysql_error());
  $check = mysql_query("SELECT ".$col." FROM ".$table." WHERE ".$col."='".$data."'");
  $match = mysql_num_rows($check);
  if($match < 1) {
    return true;
  }
  return false;
  mysql_close('localhost');
}

This function is used in conjunction with another function which just generates a random 40 digit string of 0-9, a-z, and A-Z for the email verification hash as well as the API key itself. (function below)

function makeRandom($length = 40) {
  $characters = '0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ';
  $randomString = '';
  for($i = 0; $i < $length; $i++) {
    $randomString .= $characters[mt_rand(0, strlen($characters) - 1)];
  }
  return $randomString;
}

And then the combination of those 2 functions is used in 3 different pages related to the API key issuance: Page one for registration/request, Page two for verification of email, Page 3 for lost keys or unreceived email. Now here it is in practice:

$hash   = makeRandom();
$unique = isUnique('users', 'hash', $hash);
if($unique == false) {
  while($unique == false) {
    $hash   = makeRandom();
    $unique = isUnique('users', 'hash', $hash);
  }
}
else {
  $searchactive   = mysql_query("SELECT email, active FROM users WHERE email='".$email."' AND active='1'") or die(mysql_error());
  $matchactive    = mysql_num_rows($searchactive);
  $searchinactive = mysql_query("SELECT email, active FROM users WHERE email='".$email."' AND active='0'") or die(mysql_error());
  $matchinactive  = mysql_num_rows($searchinactive);

  if($matchactive > 0) {
    $hash = mysql_query("SELECT hash FROM users WHERE email='".$email."' AND active='1'") or die(mysql_error());
    $hash = mysql_fetch_assoc($hash);
    $hash = $hash['hash'];
    $msg = 'The email address you entered is already associated with an active API key. <a href="lost.php?email='.$email.'&amp;hash='.$hash.'&active=1">[Recover Lost API Key]</a>';
  }
  elseif($matchinactive > 0) {
    $hash = mysql_query("SELECT hash FROM users WHERE email='".$email."' AND active='0'") or die(mysql_error());
    $hash = mysql_fetch_assoc($hash);
    $hash = $hash['hash'];
    $msg = 'The email address you entered is already pending verification. <a href="lost.php?email='.$email.'&amp;hash='.$hash.'&active=0">[Resend Verification Email]</a>';
  }
}

My primary question is this: With this much query’g going on just for such a (seemingly) simple function, is this going to create more problems than it solves? I really need to make sure that there aren’t any duplicate verification hashes or API keys for obvious reasons. However, with an estimated 50k people using this feature, is this going to bog down the server due to the amount of SQL queries? Primary concern is due to the while() loop used to check the uniqueness of the generated content before inserting it.

I know this isn’t a complete picture of what’s going on behind the scenes, but it does give a clue as to how the rest of the pages work. If more information about the process as a whole is needed, I’ll be happy to post it.

Thanks for any insight you can offer!

Answer 1

One way you can address this is to not check for duplicates, but just ensure that they never happen in the first place. So, just version your user table (add a field for version). This will just be an int that advances any time the user’s row is changed.

Then, when you generate your random key, append user_id and user_version to it before you store the key.

Example:

11ap0w9jfoaiwej203989wesef

Where the first 1 is the user_id and the second 1 is the user version.

Then, even on the statistically small chance that a large key is generated twice, it will always be unique because your user id’s will be unique.