Question

The tables

CREATE TABLE `pending` (
  `auto_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
  `username` varchar(255) NOT NULL,
  `password` varchar(255) NOT NULL,
  PRIMARY KEY (`auto_id`)
) ENGINE=MyISAM  DEFAULT CHARSET=latin1 AUTO_INCREMENT=68176 ;

CREATE TABLE `errors` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `username` varchar(255) NOT NULL,
  `password` varchar(255) NOT NULL,
  `error` varchar(200) NOT NULL,
  `datechecked` date NOT NULL,
  PRIMARY KEY (`id`)
) ENGINE=MyISAM  DEFAULT CHARSET=latin1 AUTO_INCREMENT=11553 ;

I am using the following code if I want to view records where pending’s username and password combination dont have a match in errors:

SELECT `pending`.username, `pending`.password FROM `pending` 
LEFT OUTER JOIN `errors` ON (
    `errors`.username = `pending`.username 
            AND 
    `errors`.password = `pending`.password
) WHERE (`errors`.username IS NULL)

To elaborate on what I mean by username and password combination is that given these tables, the result should be:

||||||pending table|||||||||
----------------------------
username    | password 
----------------------------
brian       | password1
brian       | password2
brian       | password3
brian       | password4



||||||errors table|||||||||
----------------------------
username    | password 
----------------------------
brian       | password2
brian       | password4


Result:

----------------------------
username    | password 
----------------------------
brian       | password1
brian       | password3

This works, but it takes a long time to complete. I am running this 20 times a day or so and each request is getting longer and longer as the errors table grows. I would say I am up to 5 minutes per SQL statement given their entry size by their AUTO_INCREMENT value.

I have a feeling I can make some sort of index using username and password and increase performance. Although I am not 100% sure, which is why I am asking SO.

Answer 1

Try NOT EXISTS instead of the LEFT JOIN. MySQL is relatively slow with joins.

SELECT p.username, p.password
FROM   pending p
WHERE  NOT EXISTS (SELECT * FROM errors WHERE (username, password) = (p.username, p.password))

Also, make sure you have indexes on pending (username, password) and errors (username, password).

CREATE INDEX username_password_idx ON pending (username, password);
CREATE INDEX username_password_idx ON errors (username, password);