Truncate Issue - Double vs Decimal Data Type

Question

0.00/5 (No votes)

See more:

I need to truncate a double to the hundredths place for a program. I first looked at Math.Truncate, but it truncates to a whole number. In order to be able to truncate to whatever decimal place I want, I decided to write my own function. Here it is:

VB

Private Function TruncateFld(ByVal dblValue As Double, Optional ByVal intDecimalPlaces As Integer = 0) As Double
    Dim sb As New System.Text.StringBuilder("1")
    For intI As Integer = 1 To intDecimalPlaces
        sb.Append("0")
    Next
    Return CDbl(Math.Truncate(dblValue * CInt(sb.ToString)) / CInt(sb.ToString))
End Function

It seems to work, but then when I put it in my program I found a problem. When I passed in 16.9 as the value and 2 as the number of decimal places to round to it returned 16.89. On the Return line, dblValue * CInt(sb.ToString) is 16.9 * 100 and instead of calculating to 1690 it comes back with 1689.9999999999998.

What is going on here and how can I fix it?

---------------------------
Regarding Answer 1:

I changed the Return line to this:
Return CDbl(Math.Truncate(String.Format("0 : 0.0", dblValue) * CInt(sb.ToString)) / CInt(sb.ToString))
and got this error:
Conversion from string "0 : 0.0" to type 'Double' is not valid.

Okay...You meant just use that to do the truncate. The problem is that it doesn't seem to truncate, it seems to round. So maybe I have the wrong format thing...I tried this:
Return dblValue.ToString("0.00") but if the number I pass in is 16.9091 it return 16.91 when I really want 16.90.

---------------------------
Regarding Answer 4:
I didn't know about the Math.Pow function. That is really great for getting rid of my stringbuilder work around. However, your code still doesn't solve the issue. In my testing I used the number 1.21 rounding it to 2 decimal places. In your code Value - intPortion becomes 1.21 - 1 which for some crazy reason becomes .2099999999996 instead of .21. Could you please let me know where you found the info about the problem being the double? Because I was taught to only use the Decimal type when dealing with currency and to use the Double type everywhere else. So now I'm thinking that there are probably crazy calculation errors all over my programs.

--------------
Regarding Comment:
You're right. I was using the code in your 3rd answer and had just changed a few of the variable names to my own naming convention. The one line statement does work. Thanks for your help, William Winner! Guess I have to stop using the Double data type.

For anyone else having similar problems, I found this article[^] which seems to describe the Double vs Decimal data type issue clearly.

Posted 19-Apr-10 6:11am

Kschuler

Updated 19-Apr-10 8:16am

v9

Add a Solution

4 solutions

Solution 3

Honestly, it seems that you're approaching this in a very strange way...using strings and all.

Look at it step by step.

First, get the integer portion. Ok, that's easy

VB

Dim intPortion As Long = Math.Truncate(Value)

Now, get the decimal portion with the number of digits that you want.

VB

Dim decimalPortion As Long = Math.Truncate((Value - intPortion) * Math.Pow(10, NumberOfDecimals))

Now return the integer portion plus the decimal portion converting the decimal portion back into a decimal.

VB

Return intPortion + (decimalPortion / Math.Pow(10, NumberOfDecimals))

If you are doing math, you shouldn't use strings at all...there's no need to.

Also, if your intent is to return "16.90" if you want 2 digits, then you need to return a string, not a double.

Anyway, my way put together looks like:

VB

Private Function Truncate(ByVal Value As Double, Optional ByVal NumberOfDecimals As Integer = 0) As Double
    Dim intPortion As Long = Math.Truncate(Value)
    Dim decimalPortion As Long = Math.Truncate((Value - intPortion) * Math.Pow(10, NumberOfDecimals))

    Return intPortion + (decimalPortion / Math.Pow(10, NumberOfDecimals))
End Function

Just as an FYI...the better, more concise way would be one line of code:

VB

Return (Math.Truncate(Value * Math.Pow(10, NumberOfDecimals)) _
                / Math.Pow(10, NumberOfDecimals))

...at least mathematically. For some reason, the Value * Math.Pow(10, NumberOfDecimals) also returns 16.89 when 16.9 is the value and NumberOfDecimals is 2.

Posted 19-Apr-10 7:03am

William Winner

Solution 2

I think there is some bug in calculating the round about in math.truncate method.

When I tried following values it will return me the correct value.

Math.Truncate(32.9 * 100)
32.9 return 3290
15.9 returns 1590

but when I try
16.9 it returns me 1689

So I think there is some problem in calculation within Math.Truncate method.

[From WW]
Actually, the problem is in the multiplication part of the code. If you step through the code and go into the QuickWatch (Shift + F9) and just type 16.9 * 100, it return 16.899999998 for whatever reason. 16.9 * 10 works and 16.9 * 1000 works, but not 16.9 * 100.

Posted 19-Apr-10 6:56am

Peace ON

Updated 19-Apr-10 7:09am

v2

Solution 1

Just use string.format("0 : 0.0", dblValue)

Posted 19-Apr-10 6:19am

Not Active

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

William Winner · Accepted Answer · 2010-04-19T07:13:00

I did a little research. It seems to be a problem with the Double.

You can try this though: Change your Doubles to Decimals. This function works the way it should:

VB

Private Function Truncate(ByVal Value as Decimal, _
                          Optional ByVal NumberOfDecimals as Integer = 0) as Decimal
     Return (Math.Truncate(Value * CDec(Math.Pow(10, NumberOfDecimals))) _
             / Math.Pow(10, NumberOfDecimals))
End Function

[Minor change] Changed ByValue to ByVal