Click here to Skip to main content
Rate this: bad
good
Please Sign up or sign in to vote.
See more: C++ Beginner maths
I was working with Visual Studio 2005 on a relatively simply problem, but it was fairly heavy in mathematical calculations. However, my program simply was not giving me the correct output and I could not figure out why. Long story short, I found out it was because I was basically doing this:
 
double x = 2/5;
 
However, I eventually found out that the compiler was turning that into 0 instead of .4! If I entered...
 
double x = .4;
 
Then it worked correctly. Is this an issue that was fixed in more recent Visual Studios, or what? I was extremely disappointed (yet also relieved since the program was so simple yet I was unable to do it!) when I figured out that it was as simple as a "fraction" being set to 0 instead of being divided out.
Posted 3-May-10 17:40pm
Edited 3-May-10 21:19pm
Moak11.4K
v2

1 solution

Rate this: bad
good
Please Sign up or sign in to vote.

Solution 1

This is an issue with the C++ standard. The compiler is doing EXACTLY what you ask of it.
 
2/5
 
is the same as
 
(int)2/(int)5
 
So, the result that is returned, is an int, which is THEN turned into a double.
 
2/(double)5 or
2/5.0
 
will work fine.
  Permalink  
Comments
slapshot6819 at 4-May-10 1:12am
   
Thanks for the response, very informative. Interesting. Interesting, but annoying. I rewrote my code and redid the math multiple times before I finally found out the problem heh...
emilio_grv at 7-May-10 7:50am
   
Welcome to the C++ enigmistics!

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

  Print Answers RSS
0 OriginalGriff 245
1 Jochen Arndt 155
2 PIEBALDconsult 150
3 Afzaal Ahmad Zeeshan 120
4 DamithSL 115
0 OriginalGriff 5,695
1 DamithSL 4,591
2 Maciej Los 4,012
3 Kornfeld Eliyahu Peter 3,480
4 Sergey Alexandrovich Kryukov 3,190


Advertise | Privacy | Mobile
Web02 | 2.8.141220.1 | Last Updated 4 May 2010
Copyright © CodeProject, 1999-2014
All Rights Reserved. Terms of Service
Layout: fixed | fluid

CodeProject, 503-250 Ferrand Drive Toronto Ontario, M3C 3G8 Canada +1 416-849-8900 x 100