Question

I am working with a geospatial database from geonames.org. I currently have an autocompletion input field on my website which forwards search terms to the database and returns appropriate results. One important thing is, that the results must be ordered by country.

The table from which i select is about 900.000 rows large and is created with:

CREATE TABLE IF NOT EXISTS `geonames` (
`id` integer NOT NULL AUTO_INCREMENT PRIMARY KEY,
`country_code` char(2) NOT NULL,
`postal_code` varchar(20) NOT NULL,
`place_name` varchar(180) NOT NULL,
...
FULLTEXT(country_code),
FULLTEXT(postal_code),
FULLTEXT(place_name)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_general_ci;

A typical statement looks like this:

SELECT postal_code, place_name FROM geonames WHERE LOWER(`place_name`)
LIKE 'washin%' ORDER BY FIELD (country_code, 'JE', 'GB', 'FR', 'LI', 'CH',
'DK', 'LU', 'BE', 'NL', ... many more countries in that list ... ) DESC;

I use the FULLTEXT indices for speeding up the WHERE place_name LIKE 'washin%' part. But still the query is somewhat slow. The task of the SQL query is to search for every place_name in the table which matches 'washin%' and then sort the results according to the countries specified. Is the query slow because of the large amount of data being requested at one time? If yes, how could i decrease the runtime at this bottleneck?

I am no expert in MySQL by any means, so i would be glad if someone more experienced could help me out to speed up the shown SQL query or at least point me in a direction where to go for optimization.

Thank you very much!

Answer 1

You should avoid LOWER in your where clause because then an index can’t be used efficiently:

SELECT postal_code, place_name FROM geonames
WHERE `place_name` LIKE 'washin%'
ORDER BY FIELD(country_code, 'JE', 'GB',  ...) DESC;

Instead you should use a case insensitive collation. The collations ending in _ci are case insensitive. Case sensitive collations end in _cs.

Also your full text index will not help you for queries using LIKE. You should use a B-TREE index on place_name.

B-Tree Index Characteristics

A B-tree index can be used for column comparisons in expressions that use the =, >, >=, <, <=, or BETWEEN operators. The index also can be used for LIKE comparisons if the argument to LIKE is a constant string that does not start with a wildcard character.

You could also choose to include country_code and postal_code in the index (but not as the first column). This will then give you a covering index for your query.

The ORDER BY will also be unable to use an index efficiently because of the FIELD call, but if the number of results returned is relatively small it shouldn’t be a problem.