Click here to Skip to main content
15,896,453 members
Please Sign up or sign in to vote.
2.50/5 (2 votes)
See more:
Dear friend,

I have the following code in two different projects:

Dim MySingle1 As Single = 1.15
Dim MySingle2 As Single = 1.13
Dim MyResult As Single = MySingle1 - MySingle2


But the most interesting thing is, in one project MyResult=0.02 and in another project MyResult=0.01999998.

I faild to solve this. Please please please suggest me.

Thanking you in advance.

Regards
SKPaul
Posted

This is due to the way that floating point numbers are held in computers, and how they are rounded when being displayed. See this white paper[^] for more details.
 
Share this answer
 
Comments
Saumitra Kumar Paul 30-Jul-11 5:01am    
Thank you Richard,
You are right. But on the same PC it generates different value. My head is puzzled.
Philippe Mori 30-Jul-11 8:45am    
Well if does depend on the exact sequence of generated code. If the value is a constant evaluated at compile-time or a variable evaluate at run time. If some common subexpressions are reused. Or the actual data type.
Saumitra Kumar Paul 30-Jul-11 5:04am    
how should i handle it? MyResult.Tosting("0.00") ? or any other way??
Richard MacCutchan 30-Jul-11 8:08am    
See the answer by John Simmons for suggestions.
Richard MacCutchan 30-Jul-11 9:23am    
If I display the answer with precision of 8 I get 0.01999998, anything less and it rounds up to 0.02.
Floating point math is inherently inaccurate. If you want accuracy, you MUST use decimal types. Yes, it's a hassle to convert before doing the math, but it's necessary if you want guaranteed precision.

BTW, a float (or double) will maintain it's value until you perform a math operation on it. At that point, it's value is NOT guaranteed to be completely accurate, and comparisons are not guaranteed to have the results you expect.

Using decimal is the one true path.

Try it this way:

VB
MyResult = Convert.ToSingle(Convert.ToDecimal(MySingle1) - Convert.ToDecimal(MySingle2)) 

And see if the value isn't precisely what you expect.

To avoid the casting, simply start using decimal for all of your Single or Double variables.

ALTERNATIVE - use the Math.Round method to round to a certain precision, but that's fake math. I prefer the precision of the decimal type (even though it takes a little longer to do math ops on them).
 
Share this answer
 
v4
Comments
Saumitra Kumar Paul 30-Jul-11 10:07am    
Hello John,

I am really grateful to you for such a nice solution. You deserve 5, and i did so. Thank you.

Saumitra Kumar Paul
I suspect that there is little or no difference - it may well be the way you are outputting the numbers, rather than the values themselves.

However, if they are in different projects, are all the options the same?
Is one 32 bit, and the other 64?
 
Share this answer
 
Comments
Saumitra Kumar Paul 30-Jul-11 4:43am    
Dear OriginalGriff
Thank you for showing your interest.
same code in the same PC. But in different vb project (WindowsApplication)

Why such difference?

SKPaul
OriginalGriff 30-Jul-11 4:50am    
Are the projects building for the same target? How do you display the data?
Saumitra Kumar Paul 30-Jul-11 4:55am    
yes it is for same target.

Msgbox MySingle.tostring
Saumitra Kumar Paul 30-Jul-11 5:04am    
how should i handle it? MyResult.Tosting("0.00") ? or any other way??
You can also write this

VB
Dim MySingle1 As Decimal = 1.15
Dim MySingle2 As Decimal = 1.13
Dim MyResult As Decimal = MySingle1 - MySingle2
 
Share this answer
 
v2
Comments
#realJSOP 31-Jul-11 16:19pm    
That is essentially what I suggested, with the exception that I assumed he had to start and end the calculation with doubles.
Ilias Tsakiridis 31-Jul-11 17:35pm    
Yeah but with your sugestion converts the numbers to decimals and sidesteps many of them(decimals)
Ilias Tsakiridis 31-Jul-11 17:40pm    
the point isn't the right answer the point is right code because probably in other circumstances this " MyResult = Convert.ToSingle(Convert.ToDecimal(MySingle1) - Convert.ToDecimal(MySingle2)) "may cause wrong answer
Ilias Tsakiridis 31-Jul-11 17:45pm    
But that doesn't change the fact that both things work! whatever the problem is.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900