Question

For some strange reason I can’t seem to add UTF-8 data to my MySQL database. When I enter a non-latin character, it’s stored as ?????. Everything else is stored fine. So for example, “this is an example®™” is stored fine, but “和英辞典” is stored as “????”.

The connection url is fine:

private DataSource getDB() throws PropertyVetoException {
    ComboPooledDataSource db = new ComboPooledDataSource();
    db.setDriverClass("com.mysql.jdbc.Driver");
    db.setJdbcUrl("jdbc:mysql://domain.com:3306/db?useUnicode=true&characterEncoding=UTF-8");
    db.setUser("...");
    db.setPassword("...");
    return db;
}

I’m using PreparedStatement as you would expect, I even tried entering “set names utf8” as someone suggested.

    Connection conn = null;
    PreparedStatement stmt = null;
    ResultSet rs = null;
    try {
        conn = db.getConnection();

        stmt = conn.prepareStatement("set names utf8");
        stmt.execute();
        stmt = conn.prepareStatement("set character set utf8");
        stmt.execute();

                    ... set title...
        stmt = conn.prepareStatement("INSERT INTO Table (title) VALUES (?)");
        stmt.setString(1,title);

        stmt.execute();
    } catch (final SQLException e) {
    ...

The table itself seems to be fine.

Default Character Set: utf8
Default Collation: utf8_general_ci
...
Field title:
Type text
Character Set: utf8
Collation: utf8_unicode_ci

I tested it by entering in Unicode (“和英辞典” specifically) through a GUI editor and then selecting from the table — and it was returned just fine. So this seems to be an issue with JDBC.

What am I missing?

Answer 1

There is 2 points in the mysql server to check in order to correctly set the UTF-8 charset.

Database Level

This is obtained by creating it :

CREATE DATABASE 'db' CHARACTER SET 'utf8';

Table Level

All of the tables need to be in UTF-8 also (which seems to be the case for you)

CREATE TABLE  `Table1` (
    [...]
) DEFAULT CHARSET=utf8 COLLATE=utf8_general_ci;

The important part being DEFAULT CHARSET=utf8 COLLATE=utf8_general_ci

Finally, if your code weren’t handling utf8 correctly, you could have forced your JVM to use utf8 encoding by changing the settings by on startup :

java -Dfile.encoding=UTF-8 [...]

or changing the environment variable

"**JAVA_TOOLS_OPTIONS**" to -Dfile.encoding="UTF-8"

or programmatically by using :

System.setProperty("file.encoding" , "UTF-8");

(this last one may not have the desire effect since the JVM caches value of default character encoding on startup)

Hope that helped.