Click here to Skip to main content
11,647,748 members (70,333 online)
Rate this: bad
good
Please Sign up or sign in to vote.
See more: C++ Beginner maths
I was working with Visual Studio 2005 on a relatively simply problem, but it was fairly heavy in mathematical calculations. However, my program simply was not giving me the correct output and I could not figure out why. Long story short, I found out it was because I was basically doing this: double x = 2/5; However, I eventually found out that the compiler was turning that into 0 instead of .4! If I entered... double x = .4; Then it worked correctly. Is this an issue that was fixed in more recent Visual Studios, or what? I was extremely disappointed (yet also relieved since the program was so simple yet I was unable to do it!) when I figured out that it was as simple as a "fraction" being set to 0 instead of being divided out.
Posted 3-May-10 16:40pm
Edited 3-May-10 20:19pm
Moak11.5K
v2

1 solution

Rate this: bad
good
Please Sign up or sign in to vote.

Solution 1

This is an issue with the C++ standard. The compiler is doing EXACTLY what you ask of it. 2/5 is the same as (int)2/(int)5 So, the result that is returned, is an int, which is THEN turned into a double. 2/(double)5 or 2/5.0 will work fine.
  Permalink  
Comments
slapshot6819 at 4-May-10 1:12am
   
Thanks for the response, very informative. Interesting. Interesting, but annoying. I rewrote my code and redid the math multiple times before I finally found out the problem heh...
emilio_grv at 7-May-10 7:50am
   
Welcome to the C++ enigmistics!

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

  Print Answers RSS
0 jyo.net 500
1 F-ES Sitecore 410
2 OriginalGriff 322
3 DamithSL 315
4 CPallini 250
0 OriginalGriff 1,342
1 jyo.net 994
2 DamithSL 971
3 Sergey Alexandrovich Kryukov 873
4 CPallini 795


Advertise | Privacy | Mobile
Web03 | 2.8.150804.3 | Last Updated 4 May 2010
Copyright © CodeProject, 1999-2015
All Rights Reserved. Terms of Service
Layout: fixed | fluid

CodeProject, 503-250 Ferrand Drive Toronto Ontario, M3C 3G8 Canada +1 416-849-8900 x 100