Click here to Skip to main content
Rate this: bad
good
Please Sign up or sign in to vote.
See more: C++ Beginner maths
I was working with Visual Studio 2005 on a relatively simply problem, but it was fairly heavy in mathematical calculations. However, my program simply was not giving me the correct output and I could not figure out why. Long story short, I found out it was because I was basically doing this:

double x = 2/5;

However, I eventually found out that the compiler was turning that into 0 instead of .4! If I entered...

double x = .4;

Then it worked correctly. Is this an issue that was fixed in more recent Visual Studios, or what? I was extremely disappointed (yet also relieved since the program was so simple yet I was unable to do it!) when I figured out that it was as simple as a "fraction" being set to 0 instead of being divided out.
Posted 3-May-10 17:40pm
Edited 3-May-10 21:19pm
Moak11.4K
v2

1 solution

Rate this: bad
good
Please Sign up or sign in to vote.

Solution 1

This is an issue with the C++ standard. The compiler is doing EXACTLY what you ask of it.

2/5

is the same as

(int)2/(int)5

So, the result that is returned, is an int, which is THEN turned into a double.

2/(double)5 or
2/5.0

will work fine.
  Permalink  
Comments
slapshot6819 at 4-May-10 1:12am
   
Thanks for the response, very informative. Interesting. Interesting, but annoying. I rewrote my code and redid the math multiple times before I finally found out the problem heh...
emilio_grv at 7-May-10 7:50am
   
Welcome to the C++ enigmistics!

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

  Print Answers RSS
0 Sascha Lefévre 485
1 Sergey Alexandrovich Kryukov 325
2 Maciej Los 240
3 Abhinav S 174
4 Peter Leow 120
0 Sergey Alexandrovich Kryukov 6,953
1 OriginalGriff 6,311
2 Maciej Los 2,732
3 Peter Leow 2,674
4 Abhinav S 2,562


Advertise | Privacy | Mobile
Web02 | 2.8.150414.1 | Last Updated 4 May 2010
Copyright © CodeProject, 1999-2015
All Rights Reserved. Terms of Service
Layout: fixed | fluid

CodeProject, 503-250 Ferrand Drive Toronto Ontario, M3C 3G8 Canada +1 416-849-8900 x 100