Click here to Skip to main content
15,904,415 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
See more:
Over the last 2 days I have been doing a LOT of reading about decimal vs. double about rounding.

What I would like to know is does anyone know how the Math.Round actually works under the covers? Like does it do a conversion to bits then do rounding on those bits or does it round first etc? Does it import or execute any native code that does the rounding ? etc

If anyone has any technical whitepapers or links, that would be awesome!

Thank u
Posted
Comments
Sergey Alexandrovich Kryukov 1-Nov-13 2:20am    
The question might be important for certain class of applications, only your assumption on "conversion to bits" does not seem to make sense. Essentially, everything is permanently kept "converted to bits". How would you imagine rounding by a bit pattern?

I also would like to note that I saw many cases when the mere fact of using rounding was a big mistake. In fact, rounding is rarely needed. You don't need to round data for the purposes like presenting on screen, because formatting does it better. At the same time, rounding of intermediate calculations could be just fatal. Still, rounding is used sometimes. One example is: calculation of position of some graphical element, when it is required to present the position with the accuracy of one pixel (for API not dealing with sub-pixels). Note, this is also a final presentation...

Sorry that I'm not answering your question. In case I come across relevant source of information, I'll try to remember it and reply...

—SA
Sokka_sta 1-Nov-13 2:52am    
Thanks for the reply Sergey. I read up some more and found an answer on stackoverflow: http://stackoverflow.com/questions/8870442/how-is-math-pow-implemented-in-net-framework

If u do come across anymore info, please send :)

Thanks for the pointers

1 solution

 
Share this answer
 

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900