Question

I am working with Rubyoverflow gem which allow to work with Stackoverflow API from Ruby code. And I noticed that sometimes I get error message:

Invalid JSON string

I have tried to analyze code and find out that the error occured when the Stackoverflow’s answer has non ascii characters.

Rubyoverflow uses HTTParty gem, and HTTParty gem uses Crack gem to parse JSON files.

module Crack
  class JSON
    def self.parse(json)
      json =  json.delete!("^\u{0000}-\u{007F}")  # here is my fix
      YAML.load(unescape(convert_json_to_yaml(json)))
    rescue ArgumentError => e
      raise ParseError, "Invalid JSON string"
    end

I have added one line of code to remove all non ascii characers from JSON and the error was fixed.

Now I want to understand who is the person I need to report about this error.

1. Is it possible to include non ascii characters to JSON document?
2. Is it possible to include non ascii characters to YAML document?
3. If standarts allow to include non-ascii characters to JSON and YAML -> error in YAML class. Is it right? Who is the author of this class and which is the best way to fix this bug?

Answer 1

JSON and YAML documents should be decoded according to the charset given in the HTTP headers, and then the resulting document should be parsed appropriately. Both JSON and YAML do have a way of expressing non-ASCII characters in ASCII, but neither mandates their use as opposed to using a full encoding.