Question

I am storing timestamps and some associated user data sent up by android devices in a MySQL table, the structure of which is as follows:

CREATE TABLE IF NOT EXISTS `breadcrumbs` (   
  `breadcrumbs_id` bigint(20) NOT NULL AUTO_INCREMENT,   
  `users_id` int(10) NOT NULL,   
  `timestamp` bigint(20) NOT NULL,   
  `data` text,   
  PRIMARY KEY (`breadcrumbs_id`,`users_id`),   
  KEY `fk_breadcrumbs_users1` (`users_id`)   
) ENGINE=InnoDB  DEFAULT CHARSET=utf8 AUTO_INCREMENT=468792 ;   

The android devices are sending up the timestamps in milliseconds and is following the UTC time standard to build the timestamps.
I have been trying to remove breadcrumbs older than one month. As MySQL does not have milliseconds format, I have been using the following query:

DELETE FROM breadcrumbs WHERE ((`timestamp`)/1000) < UNIX_TIMESTAMP() - 2592000;

It takes quite some time to delete the records. Below is the result of running the delete query:

/*382,580 rows affected, 0 rows found. Duration for 1 query: 36.894 sec. */

My questions are:

1. Am I picking up the breadcrumbs correctly?
2. Can I optimize the DELETE query so it runs faster than currently it is.

Answer 1

Your query logic is correct, but I would suggest not storing the milliseconds format in the first place, and instead, divide by 1000 on INSERT (or store both the milliseconds and the plain UNIX timestamp format). Or, store it as a DATETIME using FROM_UNIXTIME(timestamp / 1000) when you INSERT it. Then, create an index on the timestamp column and deletions should be much faster.

CREATE TABLE IF NOT EXISTS `breadcrumbs` (   
  `breadcrumbs_id` bigint(20) NOT NULL AUTO_INCREMENT,   
  `users_id` int(10) NOT NULL, 
  `timestamp` bigint(20) NOT NULL,
  /* Also consider storing `timestamp` as a DATETIME */
  /* `timestamp` DATETIME NOT NULL, */
  `data` text,   
  PRIMARY KEY (`breadcrumbs_id`,`users_id`),
  KEY `fk_breadcrumbs_users1` (`users_id`),
  /* index on timestamp */
  INDEX `idx_timestamp` (`timestamp`) 
) ENGINE=InnoDB  DEFAULT CHARSET=utf8 AUTO_INCREMENT=468792 ;   

/* Store the timestamp without the milliseconds (or store it both ways in 2 columns) */
INSERT INTO breadcrumbs (`breadcrumbs_id`, `users_id`, `timestamp`, `data`) VALUES (123,123, input_timestamp / 1000, data)
/* Or even better, store it as a MySQL DATETIME */
INSERT INTO breadcrumbs (`breadcrumbs_id`, `users_id`, `timestamp`, `data`) VALUES (123,123, FROM_UNIXTIME(input_timestamp / 1000), data)

As an indexed DATETIME column, your DELETE query would look like:

DELETE FROM breadcrumbs WHERE `timestamp` < NOW() - INTERVAL 1 MONTH;