15,181,010 members
Articles / General Programming / Algorithms
Article
Posted 12 May 2012

49K views
25 bookmarked

# Demystify C# floating-point equality and relation operations

Rate me:
Make floating point equality and relation operators reliable based on a comparison epsilon value.

## Introduction

My introduction is as often: why to bother at all? The C# `double` value type is known to everybody.

Do we know that type really? What is the result of this?

C#
```double a = 1.0 / 6.0;
Console.WriteLine(a + a + a + a + a + a == 1.0);```
Answer: `False` (adding six sixth does not result in 1.0)!

Or what do we get for this?

C#
```double a 1E300 / 1E-100;
Console.WriteLine(a - a == 0.0);```
Answer: `False` (`a` is +∞, and `∞ - ∞` is `NaN`, which is not `0.0`)!

Or finally, what do we get with the following:

C#
```Random r = new Random();
foreach (var n in Enumerable.Range(0, 10))
{
double sum = 0.0;
foreach (int count in Enumerable.Range(0, 100))
{
sum += Math.Round(r.NextDouble() / 50.0, 2);
}
Console.WriteLine(sum);
}```
Answer: you get random sums, of course, but not always to two decimal digits only (rounding is not "exact", since the results is still a double value...)!

This article details on how to deal with issues in comparing `double` values. The following items are discussed:

• double type at a glance
• double Equality
• Quirks of double Equality
• Make double Equality tolerant
• double Relation operations
• Quirks of double Relations
• Make double Relations tolerant
• Summary

## double type at a glance

The C# `double` type is defined according to IEEE-754 specification. That means that `double`:

• is a floating point type
• has a range from about -10308 to 10308
• has a precision of about 15 decimal digits
• has a smallest number (closest to 0.0) of about +/- 10-308
• has two zero values: +/- 0.0
• has two infinty values: +/- ∞
• has a NaN "value" (Not a Number)
• has the commonly known arithmetic operations
• has the commonly known equality and relation operations
• has Math library support for further functions like absolute value, rounding, etc.

## double Equality

As we've seen in the intro and in the overview, `double` has some special values:

• - 0.0
• + 0.0
• - ∞
• + ∞
• NaN

Let's look at the equality table for all these values (plus `1.0` representing a "normal" `double` value):

Equality (`x==y`)y=1.0y=-0.0y=+0.0y=-∞y=+∞y=NaN
x=1.0 True FalseFalseFalseFalseFalse
x=-0.0 FalseTrue True FalseFalseFalse
x=+0.0 FalseTrue True FalseFalseFalse
x=-∞FalseFalseFalseTrue FalseFalse
x=+∞FalseFalseFalseFalseTrue False
x=NaN FalseFalseFalseFalseFalseFalse

Inequality (`x!=y`)y=1.0y=-0.0y=+0.0y=-∞y=+∞y=NaN
x=1.0 FalseTrue True True True True
x=-0.0 True FalseFalseTrue True True
x=+0.0 True FalseFalseTrue True True
x=-∞True True True FalseTrue True
x=+∞True True True True FalseTrue
x=NaN True True True True True True

Notes:

• The surprising equality is given for `NaN`: two `NaN` values are not identical!
• `+0.0` and `-0.0` are identical - luckily

### Bonus

What to do if you want to know if a value is `NaN`?
The equality operator returns always `False` for `a == double.NaN`...

Answer: use the specific function `double.IsNaN(a)`. Likewise, it's prudent to check for infinity by the specific fucntions `double.IsPositiveInfinity(a)` and `double.IsNegativeInfinity(a)` respectively.

## Quirks of double Equality

The nature of floating point arithmetic is that they do implicit rounding. Not all values can be stored with unlimited precision, leaving an approximation of these values only. So, this approximation may result in a tiny "rounding error".

Adding up values with rounding errors may result in even larger errors.

C#
```double a = 1.0/6.0; // has a small rounding error
Console.WriteLine("equal = {0}", a+a+a+a+a+a == 1.0); // False --> *not* equal
Console.WriteLine("delta = {0}", a+a+a+a+a+a - 1.0);  // delta = -1.11022302462516E-16 --> *not* 0.0```

This means, you cannot trust the equality operators `==` and `!=` on `double` values. As a conclusion, we need an equality comparison that allows for "tiny" delta.

Note: As shown in the introduction, rounding does not help in this. A reliable comparison is needed.

## Make double Equality tolerant

To gain trust into the equality operator again, we must provide an equality function that allows for tolerant comparison. For that, we must make sure that the equality/inequality table is met.

A possible solution:

C#
```public static class RealExtensions
{
public struct Epsilon
{
public Epsilon(double value) { _value = value; }
private double _value;
internal bool IsEqual   (double a, double b) { return (a == b) ||  (Math.Abs(a - b) < _value); }
internal bool IsNotEqual(double a, double b) { return (a != b) && !(Math.Abs(a - b) < _value); }
}
public static bool EQ(this double a, double b, Epsilon e) { return e.IsEqual   (a, b); }
public static bool NE(this double a, double b, Epsilon e) { return e.IsNotEqual(a, b); }
}```
Usage:
C#
```var epsilon = new RealExtension.Epsilon(1E-3);
...
double x = 1.0 / 6.0;
double y = x+x+x+x+x+x;
if (y.EQ(1.0, epsilon)) ...```

Notes:

• These equality/inequality functions work also for the special values.
• Choose the Epsilon value carefully: e.g. angle epsilon are usually very different to length epsilons.
• Epsilon values can be expected to be in the range 1E-1 to 1E-15 (a meaningful limit is 1E-15 since the double prercision is about 15 decimal digits).
• The `double.Epsilon` is not useful for epsilon - it is the smallest possible only.

That's all! Well, not yet...

What about the relation operators: `<`, `<=`, `>=`, `>`. They also involve equality and inequality operations.

## double Relation operations

First, let's list again the tables of the operations:

Less-than (`x<y`)y=1.0y=-0.0y=+0.0y=-∞y=+∞y=NaN
x=1.0 FalseFalseFalseFalseTrue False
x=-0.0 True FalseFalseFalseTrue False
x=+0.0 True FalseFalseFalseTrue False
x=-∞True True True FalseTrue False
x=+∞FalseFalseFalseFalseFalseFalse
x=NaN FalseFalseFalseFalseFalseFalse

Less-equal (`x<=y`)y=1.0y=-0.0y=+0.0y=-∞y=+∞y=NaN
x=1.0 True FalseFalseFalseTrue False
x=-0.0 True True True FalseTrue False
x=+0.0 True True True FalseTrue False
x=-∞True True True True True False
x=+∞FalseFalseFalseFalseTrue False
x=NaN FalseFalseFalseFalseFalseFalse

Greater-equal (`x>=y`)y=1.0y=-0.0y=+0.0y=-∞y=+∞y=NaN
x=1.0 True True True True FalseFalse
x=-0.0 FalseTrue True True FalseFalse
x=+0.0 FalseTrue True True FalseFalse
x=-∞FalseFalseFalseTrue FalseFalse
x=+∞True True True True True False
x=NaN FalseFalseFalseFalseFalseFalse

Greater-than (`x>y`)y=1.0y=-0.0y=+0.0y=-∞y=+∞y=NaN
x=1.0 FalseTrue True True FalseFalse
x=-0.0 FalseFalseFalseTrue FalseFalse
x=+0.0 FalseFalseFalseTrue FalseFalse
x=-∞FalseFalseFalseFalseFalseFalse
x=+∞True True True True FalseFalse
x=NaN FalseFalseFalseFalseFalseFalse

Notes:

• `double.NaN` results always in False (i.e. `a < b` is not the opposite of `a >= b`).
• `+0.0` and `-0.0` are identical again.

What is the issue with the equality/inequality functions is similar with the relations. Let's look into that in the following section.

## Quirks of double Relations

Since one cannot trust the native equality/inequality operators, the relation operators cannot be trusted neither.

C#
```double a = 1.0 / 6.0;
double b = a + a + a + a + a + a;
...
if (1.0 < b) ... // delta = +1.11022302462516E-16 --> False (delta is close to zero but not exactly)
...
if (b < 1.0) ... // delta = -1.11022302462516E-16 --> True (delta is close to zero but not exactly)```
Likewise for all relation operators.

This calls for specific epsilon based relation functions as above for equality/inequality functions.

## Make double Relations tolerant

Let's just start with a possible solution, based on the EQ/NE from above.

C#
```public static class RealExtensions
{
public struct Epsilon
{
public Epsilon(double value) { _value = value; }
private double _value;
internal bool IsEqual   (double a, double b) { return (a == b) ||  (Math.Abs(a - b) < _value); }
internal bool IsNotEqual(double a, double b) { return (a != b) && !(Math.Abs(a - b) < _value); }
}
public static bool EQ(this double a, double b, Epsilon e) { return e.IsEqual   (a, b); }
public static bool LE(this double a, double b, Epsilon e) { return e.IsEqual   (a, b) || (a < b); }
public static bool GE(this double a, double b, Epsilon e) { return e.IsEqual   (a, b) || (a > b); }

public static bool NE(this double a, double b, Epsilon e) { return e.IsNotEqual(a, b); }
public static bool LT(this double a, double b, Epsilon e) { return e.IsNotEqual(a, b) && (a < b); }
public static bool GT(this double a, double b, Epsilon e) { return e.IsNotEqual(a, b) && (a > b); }
}```
Usage:
C#
```var epsilon = new RealExtension.Epsilon(1E-3);
...
double x = 1.0 / 6.0;
double y = x+x+x+x+x+x;
if (y.LT(1.0, epsilon)) ...```

Notes:

• These relation functions work also for the special values.

*************** Now, that's all! Finally. *******************

## Summary

Native equality and relation operations on floating point values cannot be trusted. A possible solution is to invent the six operations based on some epsilon value. Special care must be taken for the special values like `Infinity` and `NaN`.

## What else?

The `float` value type suffers the same problem. I.e. that type needs that same solution as for `double`, adapted to `float`.

The `decimal` has a bit a different situation: it is not considered as a C# floating-point type but has similar issues with implicit rounding errors. I.e. the same solution (adapted to `decimal`) will solve the problem here too.

## Revisions

VersionDateDescription
V1.02012-05-13Initial version
V1.12012-05-14Fix formatting
V1.22012-05-14Added `double.IsNaN(x)`,
`double.IsPositiveInfinity(x)`,
and `double.IsNegativeInfinity(x)`

## Share

 Founder eXternSoft GmbH Switzerland
I feel comfortable on a variety of systems (UNIX, Windows, cross-compiled embedded systems, etc.) in a variety of languages, environments, and tools.
I have a particular affinity to computer language analysis, testing, as well as quality management.

More information about what I do for a living can be found at my LinkedIn Profile and on my company's web page (German only).

 First Prev Next
 My vote of 5 Pratik Bhuva14-Oct-14 5:16 Pratik Bhuva 14-Oct-14 5:16
 Re: My vote of 5 Andreas Gieriet14-Oct-14 6:15 Andreas Gieriet 14-Oct-14 6:15
 My vote of 5 Amir Mohammad Nasrollahi11-Aug-13 21:13 Amir Mohammad Nasrollahi 11-Aug-13 21:13
 Re: My vote of 5 Andreas Gieriet13-Aug-13 10:56 Andreas Gieriet 13-Aug-13 10:56
 Re: My vote of 5 Amir Mohammad Nasrollahi13-Aug-13 11:21 Amir Mohammad Nasrollahi 13-Aug-13 11:21
 Interfaces, relative tolerance Henning Dieterichs20-Jun-12 20:51 Henning Dieterichs 20-Jun-12 20:51
 Re: Interfaces, relative tolerance Andreas Gieriet20-Jun-12 21:52 Andreas Gieriet 20-Jun-12 21:52
 Re: Interfaces, relative tolerance Henning Dieterichs21-Jun-12 1:01 Henning Dieterichs 21-Jun-12 1:01
 Re: Interfaces, relative tolerance Andreas Gieriet21-Jun-12 8:55 Andreas Gieriet 21-Jun-12 8:55
 Re: Interfaces, relative tolerance Andreas Gieriet20-Jun-12 21:56 Andreas Gieriet 20-Jun-12 21:56
 Re: Interfaces, relative tolerance Henning Dieterichs20-Jun-12 23:59 Henning Dieterichs 20-Jun-12 23:59
 Thanks again for your response! The HDUnitsOfMeasure project is only the (reusable) base for a (currently not finished) compile time validator task. Therefore, it has to be a general purpose solution for each physical size. And this is the cause for writing this library: The most other existing unit library follows your approach (this allows the pretty use of generics). But nearly none of them is able to comprehend the association of multiple units, like kg*m/s^2 (1 newton). Still, you could define the newton as new struct, but what happens, if you try to divide 1 N through 1 second (which is a specific speed)? Hence, using estimated epsilons for each unit is not possible. So how would you solve this? To the relative tolerance: If you just want to compare two absolut values (e.g. 6 to the sum of 6 * 1/6), it wont be a problem (and more I don't need in my library). It will be a problem if you compare a value to 0 (e.g. the difference of 6 and the sum of 6 * 1/6 compared to 0). So comparisons to exact 0 always does not work. But specific to the guaranteed precision, you could compare to 0.0001 or 0.1. Since units with a factor of exactly 0 are not allowed, this equality check will always work. The offsets of two units can be compared by using the precision of their factors: E.g. 273*K + 0.001 is equal to 272*K + 0, since 0.001 is very small in contrast to 273 and the difference between 273 and 272 is less than 1 % (the precision of a double is of course a lot smaller than 1 %). But 0.01*X + 0.0001 is not equal to 0.01*X + 0.000000000001, since the difference of the offsets is nearly 10 % of the factor (this is only brainstorming, far away from mathematical proofs). It would be great, if the given precision is stored in the double value itself... This gives me an idea If you find spelling- or grammer-mistakes, please let me know, so that I can correct them (at least for me) - english is not my first language...
 Nice review :) Thanks! PiligrimJob12-Jun-12 23:35 PiligrimJob 12-Jun-12 23:35
 Re: Nice review :) Thanks! Andreas Gieriet13-Jun-12 11:56 Andreas Gieriet 13-Jun-12 11:56
 Last Visit: 31-Dec-99 19:00     Last Update: 29-Jan-22 0:08 Refresh 1