Question

I am currently writing a calculator application. I am trying to write a derivative estimator into it. The formula below is a simple way to do it. Normally on paper you would use the smallest h possible to get the most accurate estimate. The problem is doubles can’t handle adding really small numbers to comparatively huge numbers. Such as 4+1E-200 will just result in 4.0. Even if h was just 1E-16, 4+1E16 will in fact give you the right value but doing math it it is inaccurate because anything after the 16th place is lost and rounding can’t happen correctly. I have heard the general rule of thumb for doubles is 1E-8 or 1E-7. The issue with this is large numbers wont work as 2E231+1E-8 will just be 2E23, the 1E-8 will be lost because of size issues.

f'(x)=(f(x+h)-f(x))/h as x approaches 0

When I test f(x)=x^2 at the point 4 so f'(4), it should be exactly 8
now I understand that I will probably never get exactly 8. but I the most accurate seems to be around 1E-7 or 1E8
the funny thing is 1E-9 all the to 1E-11 give the same answer.
Here is a list of h’s and results for f(x)=x^2 at x=4

1E-7 8.000000129015916
1E-8 7.999999951380232
1E-9 8.000000661922968
1E-10 8.000000661922968
1E-11 8.000000661922968
1E-12 8.000711204658728

Here are my questions:

1. What is the best way to pick h, obviously 1E-8 or 1E-7 make sense but how can I pick an h based off of x, so that it will work with any sized number even if x is 3.14E203 or 2E-231.
2. How many decimals of precision should I account for.
3. Do you have any idea how Texas instruments does it, the TI 83, 84, and Inspire can numerically figure out derivatives to 12 decimals or precision and almost always be right, but the maximum precision of their numbers is 12 digits anyway and those calculators are non CAS so they aren’t actually deriving anything
4. Logically there is a number somewhere between 1E-7 and 1E-8 that will give me a more precise result, is there any way to find that number, or at least get close to it.

ANSWERED

Thank you very much BobG. The application is currently planned to be in 2 forms, a command line PC application . And an Android application. You will be mentioned in the special thanks to portions of the About page. If you would like it will be open source but I am not posting links to the project site until I work out some very very large bugs. At the moment I have been calling it Mathulator, but the name will likely change because there is already a copyright on it and it sounds stupid.I have no clue when the release candidate will be running, at the moment I have no clue when it will be stable. But it will be very powerful if I can implement everything I want too.Thanks again. Happy Programming.

Answer 1

There’s a book that answers this question (and others like it):

Numerical Recipes in C, 2nd Edition, by Press, Vetterling, Teukolsky, and Flannery. This book also comes in C++, Fortran, and BASIC versions, as well. Sadly, no Java version exists. Additionally, I believe this book is out of print, but it is possible to buy used versions of some of the flavors online (at least through bn.com.)

Section 5.7, “Numerical Derivatives,” p. 186 explains exactly the problem you’re seeing with numerical derivatives and the math behind why it happens, along with a function for how to compute a numerical derivative properly (in C, but it should be easy to translate to Java). A summary of their simple approximation is presented here:

1) Numerically, you’re better off computing the symmetric version:

f'(x) = (f(x + h) – f(x – h)) / 2h

2) h should be approximately (sigma_f)^(1/3) * x_c

where

sigma_f =~ fractional accuracy of the computation of f(x) for simple functions

x_c =~ x, unless x is equal to zero.

However, this does not result in optimal derivatives, as the error is ~ (sigma_f)^(2/3). A better solution is Ridders’ algorithm, which is presented as the C program in the book (ref. Ridders, C.J.F. 1982, Advances in Engineering Software, vol. 4, no. 2, pp 75-76.)