Question

I trying to generate an XML document which is around 23 to 30 MB, when i open it with Firefox i receive

XML Parsing Error: not well-formed
Location: file:///Users/User/Downloads/export(2).xml
Line Number 137725, Column 1343:

After that I try to validate the document with XML Nanny and I receive the following error:

Invalid Character (Unicode: 0xB)

On several (13) lines: 137725, 137738, 137751, 137764, 137777, 137790, 137803, 137816, 146834, 189949, 193444, 193457, 193470

I’ve tried several “solutions” which include:

1. Regular Expression:

   preg_replace(
     '/[^\x9\xA\xD\x20-\x{D7FF}\x{E000}-\x{FFFD}\x{10000}-\x{10FFFF}]+/'
     , ' ', $data->Description);
   

   The problem here is that I’am not quite sure that this is valid RegEx, because I receive Internal Server Error because of enabled mod security in our apache.

2. I’ve tried to save my file in UTF-8 with BOM, but that was desperate trying

3. I’ve tried to use iconv with ‘UTF-8//IGNORE’ but and this didn’t help

4. I’ve tried to use character by character replacement, but this didn’t work well with my file because i have 230k lines.. even if i replace the specific tag in which i have that problem i’ve trigger max_execution_time directive in php and my script is killed.

For now my solution is to clear database records of this invalid characters manually, but this is now proper and correct solution to my problem because in future this script will be used to automate this export and manual editing isn’t option or subject of speech.

Answer 1

I’d first of all stick to the information given by XML Nanny:

Invalid Character (Unicode: 0xB) (several lines)

0xB is a character from the control character range, but only very limited control characters are allowed in a XML document. I suggest you start replacing those with numerical entities and try again:

$xml = strtr($xml, array("\x0B" => ""));

Firefox might accept those.