Question

I am using the loadtxt function of numpy in python in order to load data from a text file to an array. Standard stuff that shouldn’t cause any problem.

I had written a script to do this under ubuntu 10.04 and it worked flawlessly. However I recently upgraded to ubuntu 12.04 (complete reinstall, not an upgrade) and now the script doesn’t work.

Here’s one line from the file, all lines have the same structure: 11 columns separated by tabs:

Av = 0° Bv = 78°    273013  247744  1327      ****  Av' = 156°  Bv = 78°    259197  247803  1551

I want the data of the 5th and 11th column, so I use this command to load the data:
loadtxt(file, usecols=(4,10),delimiter=’\t’)

But it doesn’t work, here what ipython spits out:

In [46]: test=loadtxt(file,usecols=(4,10),delimiter='\t')
--------------------------------------------------------------------------- ValueError   
Traceback (most recent call last)  <ipython-input-46-20a0923eaf9b> in <module>()
----> 1 test=loadtxt(file,usecols=(4,10),delimiter='\t')

/usr/lib/python2.7/dist-packages/numpy/lib/npyio.pyc in loadtxt(fname, dtype, comments, delimiter, converters, skiprows, usecols, unpack, ndmin)
    794                 vals = [vals[i] for i in usecols]
    795             # Convert each value according to its column and store

--> 796             items = [conv(val) for (conv, val) in zip(converters, vals)]
    797             # Then pack it according to the dtype's nesting

    798             items = pack_items(items, packing)

ValueError: could not convert string to float:

It used to work with my old system (ubuntu 10.04) but I can’t manage to load data from this file under my new install (ubuntu 12.04).

If anyone could point me what’s wrong it’d be greatly appreciated!

Answer 1

I am not sure why loadtxt used to work, it currently doesn’t handle strings very well. But you can use genfromtxt to process this record:

import numpy as np

out = np.genfromtxt(file, usecols=(4,10), delimiter='\t')

If you omit the usecols and load the entire record, then it replaces the string entries with nans.