Question

I am not MySQL expert and am stuck in a problem. I have a table which currently hold 16GB of data and it will grow further. The structure of the table is given below,

CREATE TABLE `t_xyz_tracking` (
`id` BIGINT(20) NOT NULL AUTO_INCREMENT,
`word` VARCHAR(200) NOT NULL,
`xyzId` BIGINT(100) NOT NULL,
`xyzText` VARCHAR(800) NULL DEFAULT NULL,
`language` VARCHAR(2000) NULL DEFAULT NULL,
`links` VARCHAR(2000) NULL DEFAULT NULL,
`xyzType` VARCHAR(20) NULL DEFAULT NULL,
`source` VARCHAR(1500) NULL DEFAULT NULL,
`sourceStripped` TEXT NULL,
`isTruncated` VARCHAR(40) NULL DEFAULT NULL,
`inReplyToStatusId` BIGINT(30) NULL DEFAULT NULL,
`inReplyToUserId` INT(11) NULL DEFAULT NULL,
`rtUsrProfilePicUrl` TEXT NULL,
`isFavorited` VARCHAR(40) NULL DEFAULT NULL,
`inReplyToScreenName` VARCHAR(40) NULL DEFAULT NULL,
`latitude` BIGINT(100) NOT NULL,
`longitude` BIGINT(100) NOT NULL,
`rexyzStatus` VARCHAR(40) NULL DEFAULT NULL,
`statusInReplyToStatusId` BIGINT(100) NOT NULL,
`statusInReplyToUserId` BIGINT(100) NOT NULL,
`statusFavorited` VARCHAR(40) NULL DEFAULT NULL,
`statusInReplyToScreenName` TEXT NULL,
`screenName` TEXT NULL,
`profilePicUrl` TEXT NULL,
`xyzId` BIGINT(100) NOT NULL,
`name` TEXT NULL,
`location` VARCHAR(200) NULL DEFAULT NULL,
`bio` TEXT NULL,
`url` TEXT NULL COLLATE 'latin1_swedish_ci',
`utcOffset` INT(11) NULL DEFAULT NULL,
`timeZone` VARCHAR(100) NULL DEFAULT NULL,
`frenCnt` BIGINT(20) NULL DEFAULT '0',
`createdAt` DATETIME NULL DEFAULT NULL,
`createdOnGMT` VARCHAR(40) NULL DEFAULT NULL,
`createdOnServerTime` DATETIME NULL DEFAULT NULL,
`follCnt` BIGINT(20) NULL DEFAULT '0',
`favCnt` BIGINT(20) NULL DEFAULT '0',
`totStatusCnt` BIGINT(20) NULL DEFAULT NULL,
`usrCrtDate` VARCHAR(200) NULL DEFAULT NULL,
`humanSentiment` VARCHAR(30) NULL DEFAULT NULL,
`replied` BIT(1) NULL DEFAULT NULL,
`replyMsg` TEXT NULL,
`classified` INT(32) NULL DEFAULT NULL,
`createdOnGMTDate` DATETIME NULL DEFAULT NULL,
PRIMARY KEY (`id`),
INDEX `id` (`id`, `word`),
INDEX `word_index` (`word`) USING BTREE,
INDEX `classified_index` (`classified`) USING BTREE,
INDEX `createdOnGMT_index` (`createdOnGMT`) USING BTREE,
INDEX `location_index` (`location`) USING BTREE,
INDEX `word_createdOnGMT` (`word`, `createdOnGMT`),
INDEX `timeZone` (`timeZone`) USING BTREE,
INDEX `language` (`language`(255)) USING BTREE,
INDEX `source` (`source`(255)) USING BTREE,
INDEX `xyzId` (`xyzId`) USING BTREE,
INDEX `getunclassified_index` (`classified`, `xyzType`) USING BTREE,
INDEX `createdOnGMTDate_index` (`createdOnGMTDate`, `word`) USING BTREE,
INDEX `links` (`links`(255)) USING BTREE,
INDEX `xyzType_classified` (`classified`, `xyzType`) USING BTREE,
INDEX `word_createdOnGMTDate` (`word`, `createdOnGMTDate`) USING BTREE
    )COLLATE='utf8_general_ci'
    ENGINE=InnoDB
    ROW_FORMAT=DEFAULT
    AUTO_INCREMENT=17540328

The queries on this table are running slow now and I am expecting them to slow down further, my server configuration is given below,

Intel Xeon E5220 @2.27GHz (2 processors)
12GB Ram
Windows 2008 Server R2

my.ini file details are given below,

default-storage-engine=INNODB
sql-mode="STRICT_TRANS_TABLES,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION"
max_connections=300
query_cache_size=0
table_cache=256
tmp_table_size=205M
thread_cache_size=8
myisam_max_sort_file_size=3G
myisam_sort_buffer_size=410M
key_buffer_size=354M
read_buffer_size=64K
read_rnd_buffer_size=256K
sort_buffer_size = 64M
join_buffer_size = 64M
thread_cache_size = 8
thread_concurrency = 8
query_cache_size = 128M
innodb_additional_mem_pool_size=15M
innodb_flush_log_at_trx_commit=1
innodb_log_buffer_size=30M
innodb_buffer_pool_size=6G
innodb_log_file_size=343M
innodb_thread_concurrency=44
max_allowed_packet = 16M
slow_query_log
long_query_time = 6

What can be done to improve performance,

1. Would converting to MyISAM table help, I have INNODB since this table has frequent write and even more frequent reads.
2. I have noticed high disk I/O, at time as high as 20-40MB/sec

Thanks,
Rohit

Answer 1

One suggestion is to run

SELECT * FROM t_xyz_tracking PROCEDURE ANALYSE()

PROCEDURE ANALYSE will tell you, based on the data in the table, the suggested types for the columns in the table. This should help increase your efficiency.