Click here to Skip to main content
15,891,372 members
Please Sign up or sign in to vote.
1.00/5 (1 vote)
See more:
when I divide two numbers 9.6 and 3.2. Number two on the double data type. However, when dividing the result (outcome variable is type double) is 2.9999999999999996 instead of 3 new right. can someone help me?
Posted
Comments
Sergey Alexandrovich Kryukov 30-Nov-13 23:23pm    
Help with what? This is a perfectly accurate result.
—SA

1 solution

Please see my comment to the question. Perhaps all your problem is to get familiar with floating-point numbers, as an approximate computer representation of such concept and mathematical abstraction as real numbers:
http://en.wikipedia.org/wiki/Floating-point_number[^],
http://en.wikipedia.org/wiki/Real_number[^].

Just to note: the theory of real numbers, being a base of calculus, is an extremely deep and complex field of mathematics.

You can have 100% precise calculations with limited functionality, if you use fixed-point arithmetic, which, in .NET, can be modeled based on integer types:
http://en.wikipedia.org/wiki/Fixed-point_arithmetic[^].

The idea is this: say, you want to calculate the number in your example with two digits after point. Multiply your numbers by 100, perform integer calculations: 9.60/3.20 = 960/320 = 3.

However, fixed-point operations are really limited. You cannot calculate such thing as 1/3, for example. For such calculations, you still need to use floating-point arithmetic.

—SA
 
Share this answer
 

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900