Question

Along the lines of my previous question, How do I convert unicode characters to floats in Python? , I would like to find a more elegant solution to calculating the value of a string that contains unicode numeric values.

For example, take the strings “1⅕” and “1 ⅕”. I would like these to resolve to 1.2

I know that I can iterate through the string by character, check for unicodedata.category(x) == “No” on each character, and convert the unicode characters by unicodedata.numeric(x). I would then have to split the string and sum the values. However, this seems rather hacky and unstable. Is there a more elegant solution for this in Python?

Answer 1

I think this is what you want…

import unicodedata
def eval_unicode(s):
    #sum all the unicode fractions
    u = sum(map(unicodedata.numeric, filter(lambda x: unicodedata.category(x)=="No",s)))
    #eval the regular digits (with optional dot) as a float, or default to 0
    n = float("".join(filter(lambda x:x.isdigit() or x==".", s)) or 0)
    return n+u

or the “comprehensive” solution, for those who prefer that style:

import unicodedata
def eval_unicode(s):
    #sum all the unicode fractions
    u = sum(unicodedata.numeric(i) for i in s if unicodedata.category(i)=="No")
    #eval the regular digits (with optional dot) as a float, or default to 0
    n = float("".join(i for i in s if i.isdigit() or i==".") or 0)
    return n+u

But beware, there are many unicode values that seem to not have a numeric value assigned in python (for example ⅜⅝ don’t work… or maybe is just a matter with my keyboard xD).

Another note on the implementation: it’s “too robust”, it will work even will malformed numbers like “123½3 ½” and will eval it to 1234.0… but it won’t work if there are more than one dots.