Question

I have looked through several question about AJAX and character encoding, but I can only seem to find questions about the response’s encoding, but not where the data sent to the server’s encoding.

I am sending a form serialized using jQuery to the server. PHP then saves this form’s fields to a database. The database uses latin1-swedish-ci. The page with the form uses iso-8859-1.

When I do this by using AJAX to send the data to the server, characters like “ get converted to “ on the way to the server, and get saved in the database as such. When I skip the AJAX and use a normal html form submit, the characters turn out fine in the database.

It is obvious that the jQuery serialization of the form data and the AJAX submission are what’s causing the characters to get mangled, but how can I fix this? I have tried changing the contentType of the AJAX call. I have tried using iconv() to convert the characters server side, which causes errors or incorrect translations. I have tried to use utf8_decode() server side, but this converts it into a question mark. I cannot change the output pages to UTF-8 encoding, because this causes other problems with other content on these pages. Ideally the solution would prevent the characters from being saved in the database wrong in the first place, as would happen with a normal html form submit. Any ideas on how to solve this?

javascript that sends the form to the server:

BlogEditForm.prototype.save = function() {
    var self = this;
    $.ajax( {
        type: 'POST',
        url: 'blog_edit_ajax.php',
        dataType: 'json',
        data: self.dialog.find( 'form' ).serialize(),
        success: function( response ) {
            self.processSaveResponse.call( self, response );
        }
    } );
};

Answer 1

Seems that Javascript’s encodeURIComponent() encode characters differently than an actual form submition. Ended up re-writing javascript encodeURIComponent function to solve this… Not the most performant solution, but no other solutions presented themselves:

var encodeURIComponent = ( function() {
    var encodeURIComponentOriginal = encodeURIComponent;
    return function ( input ) {
        var toReplace = [
            '%E2%80%9A',
            '%E2%80%A6',
            '%E2%80%98',
            '%E2%80%99',
            '%E2%80%9C',
            '%E2%80%9D',
            '%E2%80%93',
            '%E2%80%94',
            '%E2%84%A2',
            '%C2%A9',
            '%C2%AE',
        ];
        var replaceWith = [
            '%82', // ‚
            '%85', // …
            '%91', // '
            '%92', // '
            '%93', // "
            '%94', // "
            '%96', // –
            '%97', // —
            '%99', // ™
            '%A9', // ©
            '%AE', // ®
        ];               

        var output = encodeURIComponentOriginal( input );

        var length = toReplace.length;

        for ( var i = 0; i < length; i++ ) {
            var regexp = new RegExp( toReplace[i], 'g' );
            output = output.replace( regexp , replaceWith[i] );
        } 

        return output;
    };
} )();