Question

I’m working on a stats project, in which we are interested in recording different events about different listings on a website.

An example of an event would be “click on logo”, “click on more info” and etc, each event is going to be represented by a number, that I’m calling eventid.

There are going to be around 20,000 events that need to be stored in this table per day, so for 5 years, somewhere around 18,000,000 rows will be in this table.

Here is what I have in mind:

CREATE TABLE IF NOT EXISTS `stattableindexed` (
  `id` int(10) unsigned NOT NULL auto_increment,
  `lid` int(10) unsigned NOT NULL COMMENT 'Listing ID',
  `serviceareaid` int(10) unsigned NOT NULL COMMENT 'Service Area ID',
  `domainid` int(10) unsigned NOT NULL COMMENT 'Domain ID',
  `bizgroup` int(10) unsigned NOT NULL COMMENT 'Business Group',
  `eventid` int(10) unsigned NOT NULL COMMENT 'Event ID',
  `eventtype` int(10) unsigned NOT NULL COMMENT 'Event Type',
  `count` int(10) unsigned NOT NULL COMMENT 'Count of Events',
  `timeunitid` bigint(10) unsigned NOT NULL COMMENT 'Time units',
  PRIMARY KEY  (`id`),
  KEY `timeunitid` (`timeunitid`),
  KEY `lid` (`lid`),
  KEY `serviceareaid` (`serviceareaid`),
  KEY `domainid` (`domainid`),
  KEY `bizgroup` (`bizgroup`),
  KEY `eventid` (`eventid`),
  KEY `eventtype` (`eventtype`),
  KEY `count` (`count`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=1 ;

Timeunit is going to be a unix epoch time, for a day, so I’m only keeping the count of events of the same kind for every day.

This way I can query the table for a range of days.

Do you think this is a good idea?
If not what is a better way of doing it?

Answer 1

Have you considered using a NoSQL solution for performance reasons? i.e MongoDB It’s a great tool for these kind of large scale logging dbs.