Question

My input files could be arbitrary, and so I will use

f = open("in-file", 'rb')

The chunk size is about 4K Bytes, and so I will use

f.read(4096)

What I want to do is to read chunks by chunks from the file.
Moreover, as chunk is actually a $2^15$-bit (4KB) sequence, when reading a chunk, I need to transform it into a decimal value for further computation.

For example, if the first chunk is of form 0000…10, what I want is having another variable keeping the corresponding decimal value, eg., x=2.

From Convert string to list of bits and viceversa I know that its code can help me read chunks by chunks.

def tobits(s):
    result = []
    for c in s:
        bits = bin(ord(c))[2:]
        bits = '00000000'[len(bits):] + bits
        result.extend([int(b) for b in bits])
    return result

However, I don’t know how to transform the output list into decimal value. Could someone give me some sample code? Thank you.

Answer 1

If I understand the question right, you want something like the following:

def bytes_to_long(bytes):
    result = 0l
    for c in bytes:
        result *= 256
        result += ord(c)
    return result

That said, it’s likely this is going to be somewhat slow, 4kB is a fairly big long and a lot of garbage ones are going to be created. You could probably improve this by using struct.unpack() and processing more than one byte per iteration, but then you have to deal with the right endianness and everything. On Python 3 you also probably don’t need the ord() since it should return the bytes type from IO methods.