Home/ Questions/Q 7435401
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-29T10:04:04+00:00

I’m trying to move over some fish species information profiles from a bespoke CMS

  • 0

I’m trying to move over some fish species information profiles from a bespoke CMS using latin1 charset to a WordPress customised (custom post type, with numerous meta fields) database which uses UTF-8.

On top of that, the old CMS uses some odd bbCode bits.

Basically, I’m looking for a function which will do this:

  • Take information from my old database with latin1_swedish_ci collation (and latin1 charset)
  • Convert all of the non-standard characters (we have characters from languages including but not exclusive of Croatian, Czech, Spanish, French and German) to HTML entities such as á (numbers like &134; fine too).
  • Convert all of the bbCode (see below) to HTML
  • Convert ' and " to HTML entities
  • Return the information with utf-8 charset to my new database

The bbCode to and from are:

$search = array( '[i]', '[/i]', '[b]', '[/b]', '[pl]', '[/pl]' );
$replace = array( '<i>', '</i>', '<strong>', '</strong>', '', '' );

The function that I’ve tried so far is:

$search = array( '[i]', '[/i]', '[b]', '[/b]', '[pl]', '[/pl]' );
$replace = array( '<i>', '</i>', '<strong>', '</strong>', '', '' );

function _convert($content) { 
    if(!mb_check_encoding($content, 'UTF-8') 
        OR !($content === mb_convert_encoding(mb_convert_encoding($content, 'UTF-32', 'UTF-8' ), 'UTF-8', 'UTF-32'))) { 

        $content = mb_convert_encoding($content, 'UTF-8'); 

        if (mb_check_encoding($content, 'UTF-8')) { 
            return $content;
        } else { 
            echo "<p>Couldn't convert to UTF-8.</p>";
        } 
    } 
} 

function _clean($content) {
    $content = _convert( $content );
    /* edited out because otherwise all HTML appears as &lt;html&gt; rather than <html>
    //$content = htmlentities( $content, ENT_QUOTES, "UTF-8" );
    $content = str_replace( $search, $replace, $content );

    return $content;
}

However this is stopping some fields from being imported to the new database and isn’t replacing the bbCode.

If I use the following code, it mostly works:

$var = str_replace( $search, $replace, htmlentities( $row["var"], ENT_QUOTES, "UTF-8" ) );

However, certain fields containing what I think are Czech/Croatian characters don’t appear at all.

Does anyone have any suggestions for how I can, in the order listed above, successfully convert the information from the “old format” to the new?

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    I would say if you want to convert all your non-ASCII characters you won’t need to do any latin1 to UTF-8 conversion what so ever. Let’s say you run a function such as htmlspecialchars or htmlentities on your data, then all non-ASCII characters will be replaced with their corresponding entity code.

    Basically, after this step, there shouldn’t be any characters left that needs conversion to UTF-8. Also, if you wanted to convert your latin1 encoding string into UTF-8 i strongly suspect utf8_encode will du just fine.

    PS. When it comes to converting bbCode into HTML I would recommend using regular expressions instead. For example you could do it all in a line like this:

    $html_data = preg_replace('/\[(/?[a-z]+)\]/i', '<$1>', $bb_code_data);
    • 0