Click here to Skip to main content
15,887,027 members

Welcome to the Lounge

   

For discussing anything related to a software developer's life but is not for programming questions. Got a programming question?

The Lounge is rated Safe For Work. If you're about to post something inappropriate for a shared office environment, then don't post it. No ads, no abuse, and no programming questions. Trolling, (political, climate, religious or whatever) will result in your account being removed.

 
GeneralRe: What's y'all's favorite way of dealing with floating point precision? Pin
Jeremy Falcon20-Mar-24 18:56
professionalJeremy Falcon20-Mar-24 18:56 
GeneralRe: What's y'all's favorite way of dealing with floating point precision? Pin
Jörgen Andersson20-Mar-24 21:40
professionalJörgen Andersson20-Mar-24 21:40 
GeneralRe: What's y'all's favorite way of dealing with floating point precision? Pin
Jeremy Falcon21-Mar-24 5:00
professionalJeremy Falcon21-Mar-24 5:00 
GeneralRe: What's y'all's favorite way of dealing with floating point precision? Pin
Jörgen Andersson22-Mar-24 1:48
professionalJörgen Andersson22-Mar-24 1:48 
GeneralRe: What's y'all's favorite way of dealing with floating point precision? Pin
jmaida21-Mar-24 11:15
jmaida21-Mar-24 11:15 
GeneralRe: What's y'all's favorite way of dealing with floating point precision? Pin
jschell21-Mar-24 16:30
jschell21-Mar-24 16:30 
AnswerRe: What's y'all's favorite way of dealing with floating point precision? Pin
Sander Rossel20-Mar-24 22:35
professionalSander Rossel20-Mar-24 22:35 
GeneralRe: What's y'all's favorite way of dealing with floating point precision? Pin
Jeremy Falcon21-Mar-24 5:06
professionalJeremy Falcon21-Mar-24 5:06 
Sander Rossel wrote:
.toFixed(x) should do the trick
There's actually some rounding errors you'll find with that. I have a means to do the rounding, just curious to know what's peep's favorite way.
Sander Rossel wrote:
But ultimately, I do all my calculations in C# that has a decent decimal type.
Win for C#. This project is in 100% Node, so me no get that.

Btw, here's the routine I use to do more accurate rounding, if you want it...
TypeScript
// this part at the top of the module

// according to the ECMAScript specification, the Number type uses double-precision floating points
// which have a 64-bit format (binary64), and consists of a sign bit with 11 exponent bits and 52
// fraction bits (each digit represents 4-bits, hence 64-bit values have a max of 16 decimals)
const MAX_DECIMALS = 16;

// ..........

/**
 * This will round off a number, but with special considerations for decimal places in a fractional
 * floating point number.
 * @param {number} number The number to round off.
 * @param {number} [decimals=0] The amount of decimal places, if any, to retain when rounding the number.
 * @returns {number} Returns the rounded number.
 */

export function roundOff(number: number, decimals = 0): number {
  const exponent = Math.min(Math.max(decimals, 0), MAX_DECIMALS);
  const factor = 10 ** exponent; // we're treating this as base 10

  // the value of Math.round(x) is the same as the value of Math.floor(x+0.5), except when x is −0 or is less
  // than 0 but greater than or equal to -0.5; for these cases Math.round(x) returns −0, but
  // Math.floor(x+0.5) returns +0. so, the last OR zero check looks for -0 and converts it to +0
  return (Math.round(((number || 0) + Number.EPSILON) * factor) || 0) / factor;
}

Sander Rossel wrote:
I once had a customer who wanted to calculate VAT for each sales order row and then got mad the total didn't add up due to rounding errors Sigh | :sigh:
Good times. Good times. Laugh | :laugh:
Jeremy Falcon

GeneralRe: What's y'all's favorite way of dealing with floating point precision? Pin
Sander Rossel21-Mar-24 5:54
professionalSander Rossel21-Mar-24 5:54 
GeneralRe: What's y'all's favorite way of dealing with floating point precision? Pin
Jeremy Falcon21-Mar-24 6:22
professionalJeremy Falcon21-Mar-24 6:22 
GeneralRe: What's y'all's favorite way of dealing with floating point precision? Pin
Sander Rossel23-Mar-24 1:43
professionalSander Rossel23-Mar-24 1:43 
GeneralRe: What's y'all's favorite way of dealing with floating point precision? Pin
Jeremy Falcon23-Mar-24 4:57
professionalJeremy Falcon23-Mar-24 4:57 
GeneralRe: What's y'all's favorite way of dealing with floating point precision? Pin
Jeremy Falcon23-Mar-24 5:00
professionalJeremy Falcon23-Mar-24 5:00 
GeneralRe: What's y'all's favorite way of dealing with floating point precision? Pin
Jeremy Falcon21-Mar-24 5:24
professionalJeremy Falcon21-Mar-24 5:24 
GeneralRe: What's y'all's favorite way of dealing with floating point precision? Pin
englebart23-Mar-24 5:47
professionalenglebart23-Mar-24 5:47 
JokeRe: What's y'all's favorite way of dealing with floating point precision? Pin
Richard Deeming20-Mar-24 23:06
mveRichard Deeming20-Mar-24 23:06 
GeneralRe: What's y'all's favorite way of dealing with floating point precision? Pin
Jeremy Falcon21-Mar-24 5:00
professionalJeremy Falcon21-Mar-24 5:00 
AnswerRe: What's y'all's favorite way of dealing with floating point precision? Pin
Gary Wheeler21-Mar-24 1:01
Gary Wheeler21-Mar-24 1:01 
GeneralRe: What's y'all's favorite way of dealing with floating point precision? Pin
Mark Whybird from Brisbane21-Mar-24 20:10
Mark Whybird from Brisbane21-Mar-24 20:10 
AnswerRe: What's y'all's favorite way of dealing with floating point precision? Pin
Marc Clifton21-Mar-24 2:28
mvaMarc Clifton21-Mar-24 2:28 
GeneralRe: What's y'all's favorite way of dealing with floating point precision? Pin
Jeremy Falcon21-Mar-24 4:58
professionalJeremy Falcon21-Mar-24 4:58 
GeneralRe: What's y'all's favorite way of dealing with floating point precision? Pin
Marc Clifton21-Mar-24 7:10
mvaMarc Clifton21-Mar-24 7:10 
AnswerRe: What's y'all's favorite way of dealing with floating point precision? Pin
den2k8821-Mar-24 23:01
professionalden2k8821-Mar-24 23:01 
AnswerRe: What's y'all's favorite way of dealing with floating point precision? Pin
Lloyd Folden 202422-Mar-24 5:33
Lloyd Folden 202422-Mar-24 5:33 
GeneralRe: What's y'all's favorite way of dealing with floating point precision? Pin
Jeremy Falcon23-Mar-24 5:06
professionalJeremy Falcon23-Mar-24 5:06 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.