Click here to Skip to main content
15,887,485 members

Welcome to the Lounge

   

For discussing anything related to a software developer's life but is not for programming questions. Got a programming question?

The Lounge is rated Safe For Work. If you're about to post something inappropriate for a shared office environment, then don't post it. No ads, no abuse, and no programming questions. Trolling, (political, climate, religious or whatever) will result in your account being removed.

 
GeneralRe: What's y'all's favorite way of dealing with floating point precision? Pin
Jeremy Falcon23-Mar-24 4:57
professionalJeremy Falcon23-Mar-24 4:57 
GeneralRe: What's y'all's favorite way of dealing with floating point precision? Pin
Jeremy Falcon23-Mar-24 5:00
professionalJeremy Falcon23-Mar-24 5:00 
GeneralRe: What's y'all's favorite way of dealing with floating point precision? Pin
Jeremy Falcon21-Mar-24 5:24
professionalJeremy Falcon21-Mar-24 5:24 
GeneralRe: What's y'all's favorite way of dealing with floating point precision? Pin
englebart23-Mar-24 5:47
professionalenglebart23-Mar-24 5:47 
JokeRe: What's y'all's favorite way of dealing with floating point precision? Pin
Richard Deeming20-Mar-24 23:06
mveRichard Deeming20-Mar-24 23:06 
GeneralRe: What's y'all's favorite way of dealing with floating point precision? Pin
Jeremy Falcon21-Mar-24 5:00
professionalJeremy Falcon21-Mar-24 5:00 
AnswerRe: What's y'all's favorite way of dealing with floating point precision? Pin
Gary Wheeler21-Mar-24 1:01
Gary Wheeler21-Mar-24 1:01 
GeneralRe: What's y'all's favorite way of dealing with floating point precision? Pin
Mark Whybird from Brisbane21-Mar-24 20:10
Mark Whybird from Brisbane21-Mar-24 20:10 
This is the only correct answer: it has to be appropriate to the calculation. In some cases, perfect precision is not even mathematically possible (e.g. pi = ratio of a circle's diameter to circumference, or anything involving it like sin() or cos()).

Rational libraries are great when appropriate, but (a) they can be a lot of computing for no real world benefit, and (b) if you have any intrinsically irrational number like pi in there anywhere, their usefulness instantly becomes limited (or you have to factor out the irrationals and express the answers as ratios of them...)

So, yeah, in real life, calculate at higher than needed* precision and round to needed precision. For accounting, use appropriate accounting rounding rules for exact 0.5 values.

*what precision? Max total of your errors all added and multiplied together appropriately according to the calculations involved must be less than half of the actual final precision.
AnswerRe: What's y'all's favorite way of dealing with floating point precision? Pin
Marc Clifton21-Mar-24 2:28
mvaMarc Clifton21-Mar-24 2:28 
GeneralRe: What's y'all's favorite way of dealing with floating point precision? Pin
Jeremy Falcon21-Mar-24 4:58
professionalJeremy Falcon21-Mar-24 4:58 
GeneralRe: What's y'all's favorite way of dealing with floating point precision? Pin
Marc Clifton21-Mar-24 7:10
mvaMarc Clifton21-Mar-24 7:10 
AnswerRe: What's y'all's favorite way of dealing with floating point precision? Pin
den2k8821-Mar-24 23:01
professionalden2k8821-Mar-24 23:01 
AnswerRe: What's y'all's favorite way of dealing with floating point precision? Pin
Lloyd Folden 202422-Mar-24 5:33
Lloyd Folden 202422-Mar-24 5:33 
GeneralRe: What's y'all's favorite way of dealing with floating point precision? Pin
Jeremy Falcon23-Mar-24 5:06
professionalJeremy Falcon23-Mar-24 5:06 
AnswerRe: What's y'all's favorite way of dealing with floating point precision? Pin
Matt Bond22-Mar-24 4:45
Matt Bond22-Mar-24 4:45 
GeneralRe: What's y'all's favorite way of dealing with floating point precision? Pin
Jeremy Falcon23-Mar-24 5:06
professionalJeremy Falcon23-Mar-24 5:06 
GeneralRe: What's y'all's favorite way of dealing with floating point precision? Pin
Matt Bond25-Mar-24 2:52
Matt Bond25-Mar-24 2:52 
GeneralRe: What's y'all's favorite way of dealing with floating point precision? Pin
Jeremy Falcon26-Mar-24 10:38
professionalJeremy Falcon26-Mar-24 10:38 
AnswerRe: What's y'all's favorite way of dealing with floating point precision? Pin
Bruce Patin22-Mar-24 6:01
Bruce Patin22-Mar-24 6:01 
GeneralRe: What's y'all's favorite way of dealing with floating point precision? Pin
Jeremy Falcon23-Mar-24 5:07
professionalJeremy Falcon23-Mar-24 5:07 
AnswerRe: What's y'all's favorite way of dealing with floating point precision? Pin
charlieg22-Mar-24 8:46
charlieg22-Mar-24 8:46 
GeneralRe: What's y'all's favorite way of dealing with floating point precision? Pin
Jeremy Falcon23-Mar-24 5:09
professionalJeremy Falcon23-Mar-24 5:09 
AnswerRe: What's y'all's favorite way of dealing with floating point precision? Pin
#realJSOP22-Mar-24 10:16
mve#realJSOP22-Mar-24 10:16 
GeneralRe: What's y'all's favorite way of dealing with floating point precision? Pin
Jeremy Falcon23-Mar-24 5:11
professionalJeremy Falcon23-Mar-24 5:11 
GeneralRe: What's y'all's favorite way of dealing with floating point precision? Pin
#realJSOP24-Mar-24 11:54
mve#realJSOP24-Mar-24 11:54 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.