Question

This is my first cython (pyx) module. How can I make this faster? Specifically I’m looking for help on the last line. It compiles and runs, but I am concerned it is transformed into Py* object stuff, which could probably be much faster.

Also, if you see any obvious errors, please let me know!

ctypedef unsigned short UInt8

DEF BASE64 = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_"

def encode_16_bytes(char *thebytes):
    cdef UInt8 *b      # the bytes cast to an array of UInt8
    cdef int i         # used to assign a value to the buffer
    cdef int j         # used to count up through the bytes
    cdef char buf[23]  # the resulting buffer of characters

    # initialize variables
    for i in range(23): # set buf array to zeros
        buf[i] = '\0'
    b = <UInt8 *>thebytes
    i = 0
    j = 1

    i += 1
    buf[i] = BASE64[(b[0] >> 6) & 0x3F]
    i += 1
    buf[i] = BASE64[b[0] & 0x3F]

    # iterate through the bytes 4 words at a time, setting each byte to its
    # mapped BASE64 counterpart
    while j < 16:
        i += 1
        buf[i] = BASE64[(b[j] >> 2) & 0x3F]
        i += 1
        buf[i] = BASE64[((b[j] << 4) | (b[j + 1] >> 4)) & 0x3F]
        i += 1
        buf[i] = BASE64[((b[j + 1] << 2) | (b[j + 2] >> 6)) & 0x3F]
        i += 1
        buf[i] = BASE64[b[j + 2] & 0x3F]
        j += 3

    # join up the characters into a string
    return "".join([chr(buf[i]) for i in range(23)])

Thanks!

Answer 1

You can create a Python bytes string out of the C char buffer by replacing the “”.join(…) with the following:

bytes_string = chr[:23]

Or more generally:

bytes_string = chr[:buffer_length]

See here for reference and more examples.