Question

I am re-writing the SQL which lets a user search for any other user on our site and also shows their roles.

An an example, roles can be “Writer”, “Editor”, “Publisher”.

Each role links a User to a Publication.

Users can take multiple roles within multiple publications.

Example table setup:

"users" : user_id, firstname, lastname
"publications" : publication_id, name  
"link_writers" : user_id, publication_id  
"link_editors" : user_id, publication_id  

Current psuedo SQL:

SELECT * FROM (
  (SELECT user_id FROM users WHERE firstname LIKE '%Jenkz%') 
  UNION 
  (SELECT user_id FROM users WHERE lastname LIKE '%Jenkz%')
) AS dt
JOIN (ROLES STATEMENT) AS roles ON roles.user_id = dt.user_id

At the moment my roles statement is:

SELECT  dt2.user_id, dt2.publication_id, dt.role FROM (
  (SELECT 'writer' AS role, link_writers.user_id, link_writers.publication_id
  FROM link_writers)
  UNION
  (SELECT 'editor' AS role, link_editors.user_id, link_editors.publication_id
  FROM link_editors)
) AS dt2

The reason for wrapping the roles statement in UNION clauses is that some roles are more complex and require a table join to find the publication_id and user_id.

As an example “publishers” might be linked accross two tables

"link_publishers": user_id, publisher_group_id
"link_publisher_groups": publisher_group_id, publication_id

So in that instance, the query forming part of my UNION would be:

SELECT 'publisher' AS role, link_publishers.user_id, link_publisher_groups.publication_id
FROM link_publishers
JOIN link_publisher_groups ON lpg.group_id = lp.group_id

I’m pretty confident that my table setup is good (I was warned off the one-table-for-all system when researching the layout). My problem is that there are now 100,000 rows in the users table and upto 70,000 rows in each of the link tables.

Initial lookup in the users table is fast, but the joining really slows things down.

How can I only join on the relevant roles?

————————– EDIT ———————————-

Explain above (open in a new window to see full resolution).

The bottom bit in red, is the “WHERE firstname LIKE ‘%Jenkz%'” the third row searches WHERE CONCAT(firstname, ‘ ‘, lastname) LIKE ‘%Jenkz%’. Hence the large row count, but I think this is unavoidable, unless there is a way to put an index accross concatenated fields?

The green bit at the top just shows the total rows scanned from the ROLES STATEMENT.

You can then see each individual UNION clause (#6 – #12) which all show a large number of rows. Some of the indexes are normal, some are unique.

It seems that MySQL isn’t optimizing to use the dt.user_id as a comparison for the internal of the UNION statements. Is there any way to force this behaviour?

Please note that my real setup is not publications and writers but “webmasters”, “players”, “teams” etc.

Answer 1

Having checked out OMG Ponies’ answer for SO – Use Of Correlated Subquery, I came up with this:

SELECT * FROM (
  (SELECT user_id FROM users WHERE firstname LIKE '%Jenkz%') 
  UNION 
  (SELECT user_id FROM users WHERE lastname LIKE '%Jenkz%')
) AS dt
JOIN ( SELECT 'writer' AS role, link_writers.user_id, link_writers.publication_id
       FROM link_writers
       UNION
       SELECT 'editor' AS role, link_editors.user_id, link_editors.publication_id
       FROM link_editors
       UNION
       SELECT 'publisher' AS role, lp.user_id, lpg.publication_id
       FROM link_publishers lp
       JOIN link_publisher_groups lpg ON lpg.publisher_group_id = lp.publisher_group_id
     ) roles on roles.user_id = dt.user_id

The explain looks reasonable on my tiny data set. What does it look like on the real thing ?