Question

Given log(a) and log(b), I want to compute log(a+b) (in a numerically stable way).

I wrote a little function for this:

def log_add(logA,logB):
    if logA == log(0):
        return logB
    if logA<logB:
        return log_add(logB,logA)
    return log( 1 + math.exp(logB-logA) ) + logA

I wrote a program where this is by far the most time-consuming piece of code. Obviously I could try to optimize it (eliminate the recursive call, for instance).

Do you know of a standard math or numpy function for computing log(a+b) from log(a) and log(b)?

If not, do you know of a simple way to make a single C++ hook for this function? It’s not a complicated function (it uses floats), and as I said, it’s taking up the majority of my runtime.

Thanks in advance, numerical methods ninja!

Answer 1

Note: Best answer until now is to simply use numpy.logaddexp(logA,logB).

Why exactly do you compare with log(0)? This is equal to -numpy.inf, in this case you come to log(1 + math.exp(-inf-logB) ) + logB Which reduces itself to logB. This call always will give an warning message which is extremely slow.

I could come up with this one-liner. However you’ll need to really measure to see if this is actually faster. It does only use one ‘complex’ calculation function instead of the two that you use, and no recursion is happening, the if is still there but hidden (and maybe optimized) in fabs/maximum.

def log_add(logA,logB):
    return numpy.logaddexp(0,-numpy.fabs(logB-logA)) + numpy.maximum(logA,logB)

edit:

I did a quick timeit() with following results :

1. Your original version took about 120s
2. My version took about 30s
3. I removed the compare with log(0) from your version and it came down to 20s
4. I edited my code to keep the logaddexp but also worked with your recursive if and it went down to 18s.

Updated code, you could also switch the recursive call with an inline updated formula but this made little difference in my timing tests:

def log_add2(logA, logB):
    if logA < logB:
        return log_add2(logB, logA)
    return numpy.logaddexp(0,logB-logA)+logA

Edit 2:

As pv noted in comments, you could actually just do numpy.logaddexp(logA, logB) which comes down to calculating log(exp(logA)+exp(logB)) which is of course equal to log(A+B). I timed it (on the same machine as above) and it went further down to about 10s. So we’ve come down to about 1/12, not bad ;).