Question

Maybe you can help with a Yaml encoding I have.

We have an application that stores some settings serialized in a database as a Yaml string, for example:

---
quantity_units: Stunden,Tage, Monate, Pauschal, Jahre, GB, MB, Stück, Seite, SMS
categories: Shirts
number_schema: P-[Y4]-[CY3]

We are in a process of moving from Ruby 1.8.7 to Ruby 1.9.3, and the Yaml parsing library has changed between the versions, leaving us with decoded strings like this from Stück to Stück.

I only want to know how could properly convert these strings into unicode, and I’ll take care of the rest.

I don’t know which encoding was using in 1.8.7 Yaml parser.

Answer 1

This looks like utf8 read as iso-8895-1, and interpreted as utf-8 by
the ruby adapter. You might want to check your current locale and the
locale of the database server. Also see what happens if you access the
data directly via console, and check the encoding there as well. It
looks like utf-8 on the database, but gets interpreted as iso-8859-1
somewhere in between.

If nothing helps, there’s a snippet to pass your data through (and
write it back).

 "Stück".encode('iso-8859-1').force_encoding('utf-8') # I've no idea what I'm doing.
 # => "Stück"