Question

I have a database that contains data in different languages. Some languages use accents (like áéíóú) and I need to search in this data as the accents doesn’t exist (search for ‘campeon’ should return ‘campeón’ as a valir result).

The problem is that the tables in my database (utf8_unicode_ci) are not storing utf8 characteres. If you see the data through phpmyadmin the words with accents looks like this: campeón

After some researching, I’ve found (in a StackOverflow question) that the problem is related to the inexistence of a SET NAMES [charset]. In fact, I’ve made some testings and if I set names to utf8, everything works as expected.

Well, I have the solution, what’s the problem? The problem is that the database is in production, so there are thousands of strings in the database. If I change the character set the client will use, all already existing string will become invalid. The question is: is there any way to:

1. perform accent-insensitive searches in a database that uses a wrong charset like mine?
2. transform safely the data in the tables to the appropriate charset?
3. continue working with mixed charsets (latin1 and utf8) in the database, assuming that latin1 data will not be accent-insensitive?

If anybody has experience in any of the solutions I propose or has a new one, I’ll be very thankful if share.

Answer 1

The problem being that the data was inserted using the wrong connection encoding, you can fix it by

1. Exporting the data using the wrong connection encoding, just like you have used it thus far, followed by
2. Importing the data using the correct utf8 connection encoding.

That will fix the encoding problem, after which search will work as expected.