The problem with floating numbers

This is a inherent property of floating-point computer arithmetic, i.e. arithmetic with limited precision.

As example, take the number 1/7 = 0.142857142857142857142857...
If you now calculate 7x(1/7) with limited precision (e.g. with paper and pencil) you see the same effect:

precision	1/7	7x(1/7)	rounding to 2 digits
0	0	0	0.00
1	0.1	0.7	0.70
2	0.14	0.98	0.98
3	0.142	0.994	0.99
4	0.1428	0.9996	1.00
5	0.14285	0.99995	1.00
6	0.142857	0.999999	1.00
...	...		...
12	0.142857142857	0.999999999999	1.00
...	...	...	...

Analogous with computer arithmetic, you have to take care of exactly this issue.
You have to consider three issues if working with floating point numbers:

1. Decide on a precision for displaying numbers (e.g. a physical length given in meters [m] and precise to 1 millimeter [mm] --> display precision = 3 --> rounding to 3 fraction digits while displaying). E.g. d.ToString("F3").
2. Decide on a comparison delta used for equality and relation operations (like ==, !=, <, <=, >=, >). E.g. display precision + 1 --> see Demystify C# floating-point equality and relation operations[^]. E.g. var epsilon = new RealExtension.Epsilon(1E-4);...if (x.EQ(y, epsilon)) ...
3. Do not use GetHashCode() on floating point numbers, neither directly nor indirectly e.g. as key in dictionaries and hash collections (due to the not precise nature of the floating point numbers).

Cheers
Andi

PS: Do not cast forth and back between number domains. This is a bad habit that cost you more in the long run than it helps. Decide on a domain and stay in it. Single has the same problem as stated above, it is simply shifted to other places.

MSDN:

Single values have less precision than Double values. A Single value that is converted to a seemingly equivalent Double often does not equal the Double value because of differences in precision. In the following example, the result of identical division operations is assigned to a Double and a Single value. After the Single value is cast to a Double, a comparison of the two values shows that they are unequal.

http://msdn.microsoft.com/en-us/library/system.double.aspx?cs-save-lang=1&cs-lang=csharp#code-snippet-7[^]

When you write the line

double f = 10.123 - 10.0;

it is the same as to write

double f = 10.123d - 10.0d;

as C# default is to double precision floating point...
There is no way to get the exact value but you can get closer (or further) to it...All you have to do is to decide what the precision you need...
for example by changing your line to

double f = 10.123f - 10.0f

you will get the result of 0.12300014495849609...
As you can see at the bottom line you will have to play with rounding up/down while using floating point numbers...

However you are fortunate - this problem of floating points was known to Microsoft and there is an other type that can be used for floating point computations - decimal[^]...
It has it drawbacks (read the MSDN page) but for your range (according to your case) it will give the correct answer. Try...

decimal f = 10.123m - 10.0m;

Yes this is the common bug for converting single to double
to avoid it do one of the this:
convert number to double
get result in single
convert single to string then convert them to double

single f = 10.123 - 10.0;

Check these threads :-) you will find the answer

C#: Weird outcome when subtracting doubles
C# wrong subtraction?
Double subtraction...extra value unexpected