Question

Nijas

I’m having a problem with a query for receiving data to generate a classic forum index with all it’s information, you something like phpBB.

My tables looks like this:

categories:
  gategory varchar(50)  -> primary key

forums:
  id int -> primary key
  name varchar(255)
  description text
  category varchar(50)  -> foreign key to category

topics:
  id int -> primary key
  forum_id int -> foreign key to forums
  subject varchar(255)

posts:
  id int -> primary key
  topic_id int -> foreign key to topics
  user_id int -> foreign key to users
  post text
  create_date datetime
  modify_date timestramp, on_update(current_time)

users:
  id int -> primary key
  username varchar(32)
  password varchar(32)

And that is just great easy peasy.
Then I began building the query, and it got really complex (in my world) pretty fast.
I would like to get:

catories:
  forums:
    name,
    description,
    count(topics)
    count(posts)
      last_post user_id
      last_post username
      last_post create_date

I ended up with a working query looking like this:

SELECT
  f.id              as fid,
  f.name            as name,
  f.description     as description,
  f.category        as category,
  ( SELECT COUNT(*)
    FROM forum_topics
    WHERE forum_id = f.id
  )                 as topics,
  ( SELECT COUNT(*)
    FROM forum_posts fp
    WHERE fp.topic_id IN (
      SELECT id
      FROM forum_topics
      WHERE forum_id = f.id
    )
  )                 as posts,
  lp.user_id        as lp_userid,
  u.username        as lp_username,
  lp.create_date    as lp_date
FROM forums f
LEFT OUTER JOIN (
  SELECT p.create_date, p.user_id, t.forum_id
  FROM forum_topics t
  INNER JOIN forum_posts p ON ( t.id = p.topic_id )
  ORDER BY p.create_date DESC
) lp ON (lp.forum_id = f.id)
LEFT OUTER JOIN users u ON ( u.id = lp.user_id )
GROUP BY category, f.order

It’s fine; it works, but it performs very badly.
So I was wondering some of you clever folks at this place,
could give me some advice on how to optimize the query,
maybe put in some indices some smart places, or reconstruct the schema in a smarter way.

// Thank you very much in advance

Answer 1

The basic query is to join all the tables together along their nature dimensions. This gets you everything except for the last post.

The following query uses standard SQL and should work in both mysql and SQL Server (except for typos).

SELECT
  f.category,
  f.id,
  f.name,
  f.description,
    count(distinct t.id) AS topics,
    count(distinct p.id) AS posts,
    min(lastuser.id),
    min(lastuser.username),
    min(p.create_date)
FROM posts p
JOIN users u ON p.user_id = u.id
JOIN topics t ON p.topic_id = t.id
JOIN forums f ON t.forum_id = f.id
JOIN (SELECT
        t.forum_id,
        u.id,
        u.username,
        p.create_date
        FROM posts p
        JOIN topics t ON p.topic_id = t.id
        JOIN users u ON p.user_id = u.id
        JOIN (SELECT
               t.forum_id, max(p.id) AS max_postid
              FROM posts p
              JOIN topics t ON p.topic_id = t.id
              GROUP BY t.forum_id
              ) lastpost ON p.id = lastpost.max_postid
             AND t.forum_id = lastpost.forum_id
       ) lastuser on lastuser.forum_id = f.id
GROUP BY f.category, f.id, f.name, f.description

It gets the last user by another complicated set of joins. The query assumes that the posts are assigned monotonically, so the most recent post has the highest post id.

There are other approaches. In particular, SQL Server supports window functions, which would simplify the query.