Question

after optimization, I calculate the residuals with optimal parameters both in Python and C++. The gap in results is huge. Here is how I proceed more precisely:

I generate data according to a parametric model in Python. I store X and Y in Excel file. I load this file in my C++ program and run optimization. I come up with optimal parameters, that are pretty close to parameters used to generate the series. I then compute residuals (sum of squared difference between Y and model output with optimal parameters), in Python and with C++. Results are huge, with up to 10^3 difference for models that are very sensible to changes in parameters. Can these differences be imputable to different way to deal with precision in Python and C++, or can something else be wrong? Once optimization is finished, residuals computation is a simple calculation, and I wonder where the problem could lie if not in precision matter.

Thanks a lot for any advice or reference.

EDIT — I can easily show Python code for generating data and calculating sum of squared residuals, but not C++ code since calculation is performed via an interpreter. Thanks for any comments.

P1 =  5.21
P2 = 0.22

X_= list(range(0,100,1))
X=[float(x)/float(10) for x in X_]
Y = [P1*numpy.exp(-1*P2*x) for x in X]

##plt.plot(X,Y)
##plt.show()

##for j in range(len(Y)):
##    Y[j]+=rg.normal(0,0.01)

#build some input files
X1f = open('F:\WORK\SOLVEUR\ALGOCODE\PYTHON_\DataSets\exponential1X.txt', 'w')
for i in range(len(X)):
     X1f.write(str(X[i])+'\n')
X1f.close()

Yf = open('F:\WORK\SOLVEUR\ALGOCODE\PYTHON_\DataSets\exponential1Y.txt', 'w')
for i in range(len(Y)):
    Yf.write(str(Y[i])+'\n')
Yf.close()


def func_exp_1(param, x1, y):
   p1, p2 = param
   res = sum((y_i - p1*numpy.exp(-1*p2*x))**2 for x1_i, y_i in zip(x1, y))
   return res
print func_exp_1([5.2132,0.2202],x1,y)

Answer 1

Both Python and C++ use the machine native format; Python’s float is
the equivalent of C++’s double. Any differences
would be due to differences in the way the algorithm is implemented,
or, if the hardward has an extended format which is used for
intermediate values (the case for Intel), when and where the language
stores the values back to memory—values will probably be stored to
memory more often in Python than in C++. Without seeing exact code,
it’s impossible to say more (but the sum of a large number of elements
can be off significantly, depending on the order and relative magnitudes
of the elements).