Question

I came up with this code:

def DSigmoid(value):  
    return (math.exp(float(value))/((1+math.exp(float(value)))**2))  

a.) Will this return the correct derivative?
b.) Is this an efficient method?

Friendly regards,
Daquicker

Answer 1

Looks correct to me. In general, two good ways of checking such a derivative computation are:

1. Wolfram Alpha. Inputting the sigmoid function 1/(1+e^(-t)), we are given an explicit formula for the derivative, which matches yours. To be a little more direct, you can input D[1/(1+e^(-t)), t] to get the derivative without all the additional information.

2. Compare it to a numerical approximation. In your case, I will assume you already have a function Sigmoid(value). Taking

   Dapprox = (Sigmoid(value+epsilon) - Sigmoid(value)) / epsilon

   for some small epsilon and comparing it to the output of your function DSigmoid(value) should catch all but the tiniest errors. In general, estimating the derivative numerically is the best way to double check that you’ve actually coded the derivative correctly, even if you’re already sure about the formula, and it takes almost no effort.