I’m trying to compute this equation:
int n = 89;
int SUMx = 347762, SUMy = 4898, SUMxy = 19671265;
long long SUMxx = 1370329156;
printf("(float)(( SUMx*SUMy - n*SUMxy ) / ( SUMx*SUMx - n*SUMxx ) = %f \n", (float)(( SUMx*SUMy - n*SUMxy ) / ( SUMx*SUMx - n*SUMxx )));
SUMxx used to be int, but doing n*SUMxx overflowed, so I changed it to long long. Now it is just giving me an answer of 0.000000 where I want it to be 0.0464.
How do I accomplish this? Is there some discrepancy where I need to change the other datatypes to long long as well?
Thank you 🙂
You’re doing integer division so anything between 0 and 1 will be truncated before you cast it to float. Cast one of the values to double (or float) before doing the arithmetic so you’re using floating point arithmetic instead