Question

I’m having problems with Ruby 1.9 CSV and invalid UTF-8 characters in my data.

My code looks something like this:

CSV.foreach("small-test2.csv", options) do |row |

    name, workgroup, address, actual, output = row
    next if nbname == "NBName"
    @ssl_info[name] = workgroup, address, actual, output

   ic = Iconv.new('UTF-8//IGNORE', 'UTF-8')
   clean = ic.iconv(output + ' ')[0..-2]

   puts clean

end

However I’m still getting the following:

ArgumentError: invalid byte sequence in UTF-8
           =~ at org/jruby/RubyRegexp.java:1487
           =~ at org/jruby/RubyString.java:1686

Is there anything I’m missing here?

Answer 1

try this,

output.encode("UTF-8", :invalid => :replace, :undef => :replace, :replace => "")