Question

We have an application hosted on both a local development server and a live site. We are experiencing UTF-8 corruption issues and are looking to figure out how to resolve them.

The system is run using symfony 1.0 with Propel.

On our development server, we are running PHP 5.2.0 and MySQL 5.0.32. We do not experience corrupted UTF-8 characters there.

On our live site, PHP 5.2.10 and MySQL 5.0.81 is running. On that server, certain characters such as ố and Σ are corrupted once they are stored in the database. The corrupted characters are showing up as either question marks or approximations of the original character with adjacent question marks.

Examples of corruption:

Uncorrupted: ố
Corrupted: ô?

Uncorrupted: Σ
Corrupted: ?

We are currently using the following techniques on both development and live servers:

1. Executing the following queries prior to execution of any other queries:

   SET NAMES 'utf8' COLLATE 'utf8_unicode_ci'
   SET CHARSET 'utf8'
   

2. Setting the <meta> Content-Type value to:

   <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
   

3. Adding the following to our .htaccess file:

   AddDefaultCharset utf-8
   

4. Using mb_* (multibyte) PHP functions where necessary.

5. Being sure to set database columns to use utf8_unicode_ci collation.

   These techniques are sufficient for our development site, but do not work on the live site.

On the live site I’ve also tried adding mysql_set_encoding('ut8', $mysql_connection) but this does not help either. I have found some evidence that newer versions of PHP and MySQL are mishandling UTF-8 character encodings.

Answer 1

Our experienced system administrator discovered a fix:

alter database DB_NAME character set utf8;

This solved our problems completely.