Question

I’m dealing with a balance sheet which I’ve parsed into pandas using:

    table = xls_file.parse('Consolidated_Balance_Sheet')
    table.ix[:, 1]

    0         None
    1         None
    2      $ 3,029
    3         1989
    5         None
    6     $ 34,479

I’m trying to identify the rows with unicode and strip the $ sign and comma, converting to float.

    for row in table.ix[:, 1]:
        if isinstance(row, unicode):
            print type(row), row
            num = float(row.lstrip('$').replace(',',''))
            print num
            row = num
            print type(row), row

This produces the following output:

    <type 'unicode'> $ 3,029
    3029.0
    <type 'float'> 3029.0
    <type 'unicode'> $ 34,479
    34479.0
    <type 'float'> 34479.0

However, the value is unchanged when I check the table

    table.ix[2, 1]
    u'$ 3,029'

How can I correctly change the value to a float?

EDIT: Thanks for the two responses, I can reproduce those with no problem. However when I use the apply function to my case I get an ‘unhashable type’ error.

In [167]: thead = table.head()
In [168]: thead

Out[168]:
         Consolidated Balance Sheet (USD $)  Sep. 30, 2012  Dec. 31, 2011
    0    In Millions, unless otherwise specified     None    None
    1    Current assets                              None    None
    2    Cash and cash equivalents                   $ 3,029 $ 2,219
    3    Marketable securities - current             1989    1461
    4    Accounts receivable - net                   4409    3867

In [170]: def no_comma_or_dollar(num):
              if isinstance(num, unicode):
                  return float(num.lstrip('$').replace(',',''))
              else:
                  return num

          thead[:, 1] = thead[:, 1].apply(no_comma_or_dollar)

Produces the following:

 TypeError: unhashable type

I can’t get my head around why as I’m not changing the keys, just the values. Is there another way to change the values in the dataframe?

EDIT2:

In [171]: thead.to_dict()
Out[171]: {u'Consolidated Balance Sheet (USD $)': {0: u'In Millions, unless otherwise specified',
  1: u'Current assets',
  2: u'Cash and cash equivalents',
  3: u'Marketable securities - current',
  4: u'Accounts receivable - net'},
 u'Dec. 31, 2011': {0: None, 1: None, 2: u'$ 2,219', 3: 1461.0, 4: 3867.0},
 u'Sep. 30, 2012': {0: None, 1: None, 2: u'$ 3,029', 3: 1989.0, 4: 4409.0}}

Answer 1

You are just printing these and not apply-ing them to the DataFrame, here’s one way to do it:

Create a function to do the striping (if unicode) or leave it if already a number:

def no_comma_or_dollar(num):
    if isinstance(num, unicode):
        return float(num.lstrip('$').replace(',',''))
    else:
        return num

table[col_name] = table[col_name].apply(no_comma_or_dollar)

For example:

df = pd.DataFrame([[u'$1,000'], [200.]])

In [3]: df[0].apply(no_comma_or_dollar)
Out[3]: 
0    1000
1     200
Name: 0

Update:

With the thread which you give, I would be tempted to give a slightly lazier version of no_comma_or_dollar and applymap:

def no_comma_or_dollar2(num):
    try:
        return float(num.lstrip('$').replace(',',''))
    except: # if you can't strip/replace/convert just leave it
        return num

In [5]: thread.applymap(no_comma_or_dollar2)
Out[5]: 
        Consolidated Balance Sheet (USD $)  Dec. 31, 2011  Sep. 30, 2012
0  In Millions, unless otherwise specified            NaN            NaN
1                           Current assets            NaN            NaN
2                Cash and cash equivalents           2219           3029
3          Marketable securities - current           1461           1989
4                Accounts receivable - net           3867           4409