Question

I am trying to get do this:

<?php
  $good_customer = 0;
  $q = mysql_query("SELECT user FROM users WHERE activated = '1'"); // this gives me about 40k users

  while($r = mysql_fetch_assoc($q)){
    $money_spent = 0;

    $user = $r['user'];
    // Do queries on another 20 tables
    for($i = 1; $i<=20 ; $i++){
      $tbl_name = 'data' . $i;

      $q2 = mysql_query("SELECT money_spent FROM $tbl_name WHERE user = '{$user}'");
      while($r2 = mysql_fetch_assoc($q2)){
        $money_spend += $r2['money_spent'];
      }

      if($money_spend > 1000000){
        $good_customer += 1;
      }
    }
  }

This is just an example. I am testing on localhost, for single user, it returns very fast. But when I try 1000, it takes forever, not even mentioned 40k users.

Anyway to optimise/improve this code?

EDIT:
By the way, each of the others 20 tables has ~20 – 40k records

EDIT2:

Okay, drop the “money spend” idea. This is my current structures:

user table => user is PK

logs_week_1 table => user is FK.

logs_week_2 table => user is FK

logs_week_3 table => user is FK

… will have more logs tables in future.

I want to find the “average time” they spend on my site which the time stored in each of the logs tables.

So you guys were saying, storing the logs weekly is a bad idea? I should merge into one table?

Answer 1

Sounds like you have a problem with your model. Why do you have 20 data-tables instead of one with a week-column?

Then you could do a

Select user, Sum( money_spent ) As total_money_spent
From data
Group By user

or even

Select Count(*) As good_customer_count
From data
Group By user
Having Sum( money_spent ) > 1000000

---

With your current structure you can only do something like this:

Select u.user, d1.money_spent + d2.money_spent + ...
From users u
Join data1 d1 On ( d1.user = u.user )
Join data2 d2 On ( d2.user = u.user )
...

or

Select Count(*) As good_customer_count
From
  ( Select d1.money_spent + d2.money_spent + ... As total_money_spent
    From data1 d1
    Join data1 d1 On ( d1.user = u.user )
    Join data2 d2 On ( d2.user = u.user )
    ...
  )
Where total_money_spent > 1000000

This will certainly be faster than your current solution.

---

And the time spent on a page should be stored in a numeric field.