Question

I am a bit familiar with Python. I have a file with information that I need to read in a very specific way. Below is an example…

1
6
0.714285714286
0    0    1.00000000000
0    1    0.61356352337
...
-1  -1    0.00000000000
0    0    5.13787636499
0    1    0.97147643932
...
-1  -1    0.00000000000
0    0    5.13787636499
0    1    0.97147643932
...
-1  -1    0.00000000000
0 0 0 0   5.13787636499
0 0 0 1   0.97147643932
....

So every file will have this structure (tab delimited).

• The first line must be read in as a variable as well as the second and third lines.
• Next we have four blocks of code separated by a -1 -1 0.0000000000. Each block of code is ‘n’ lines long. The first two numbers represent the position/location that the 3rd number in the line is to be inserted in an array. Only the unique positions are listed (so, position 0 1 would be the same as 1 0 but that information would not be shown).
• Note: The 4th block of code has a 4-index number.

What I need

• The first 3 lines read in as unique variables
• Each block of data read into an array using the first 2 (or 4 ) column of numbers as the array index and the 3rd column as the value being inserted into an array.
• Only unique array elements shown. I need the mirrored position to be filled with the proper value as well (a 0 1 value should also appear in 1 0).
• The last block would need to be inserted into a 4-dimensional array.

Answer 1

I rewrote the code. Now it’s almost what you need. You only need fine tuning.

I decided to leave the old answer – perhaps it would be helpful too.
Because the new is feature-rich enough, and sometimes may not be clear to understand.

def the_function(filename):
    """
    returns tuple of list of independent values and list of sparsed arrays as dicts
    e.g. ( [1,2,0.5], [{(0.0):1,(0,1):2},...] )
    on fail prints the reason and returns None:
    e.g. 'failed on text.txt: invalid literal for int() with base 10: '0.0', line: 5'
    """

    # open file and read content
    try:
        with open(filename, "r") as f:
            data_txt = [line.split() for line in f]
    # no such file
    except IOError, e:
        print 'fail on open ' + str(e)

    # try to get the first 3 variables
    try:
        vars =[int(data_txt[0][0]), int(data_txt[1][0]), float(data_txt[2][0])]
    except ValueError,e:
        print 'failed on '+filename+': '+str(e)+', somewhere on lines 1-3'
        return

    # now get arrays
    arrays =[dict()]
    for lineidx, item in enumerate(data_txt[3:]):
        try:
            # for 2d array data
            if len(item) == 3:
                i, j = map(int, item[:2])
                val = float(item[-1])
                # check for 'block separator'
                if (i,j,val) == (-1,-1,0.0):
                    # make new array
                    arrays.append(dict())
                else:
                    # update last, existing
                    arrays[-1][(i,j)] = val
            # almost the same for 4d array data
            if len(item) == 5:
                i, j, k, m = map(int, item[:4])
                val = float(item[-1])
                arrays[-1][(i,j,k,m)] = val
        # if value is unparsable like '0.00' for int or 'text'
        except ValueError,e:
            print 'failed on '+filename+': '+str(e)+', line: '+str(lineidx+3)
            return
    return vars, arrays