Print Fixeddocument/Xps to PDF Without Showing File Save Dialog

C# - Inconsistent math operation result on 32-bit and 64-bit

The x86 instruction set has tricky floating point consistency issues due the way the FPU works. Internal calculations are performed with more significant bits than can be stored in a double, causing truncation when the number is flushed from the FPU stack to memory.

That got fixed in the x64 JIT compiler, it uses SSE instructions, the SSE registers have the same size as a double.

This is going to byte you when your calculations test the boundaries of floating point accuracy and range. You never want to get close to needing more than 15 significant digits, you never want to get close to 10E308 or 10E-308. You certainly never want to square the largest representable value. This is never a real problem, numbers that represent physical quantities don't get close.

Use this opportunity to find out what is wrong with your calculations. It is very important that you run the same operating system and hardware that your customer is using, high time that you get the machines needed to do so. Shipping code that is only tested on x86 machine is not tested.

The Q&D fix is Project + Properties, Compile tab, Platform Target = x86.

---

Fwiw, the bad result on x86 is caused by a bug in the JIT compiler. It generates this code:

      double r = Math.Sqrt(v1 * v1);
00000006  fld         dword ptr ds:[009D1578h] 
0000000c  fsqrt            
0000000e  fstp        qword ptr [ebp-8] 

The fmul instruction is missing, removed by the code optimizer in release mode. No doubt triggered by it seeing the value at double.MaxValue. That's a bug, you can report it at connect.microsoft.com. Pretty sure they're not going to fix it though.

Why does Math.Exp give different results between 32-bit and 64-bit, with same input, same hardware

Yes rounding errors, and it is effectively NOT the same hardware. The 32 bit version is targeting a different set of instructions and register sizes.

different behaviour or sqrt when compiled with 64 or 32 bits

You haven't said which compiler or architecure you're using, but assuming gcc on x86 / x86-64 then the difference is likely down to the fact that by default gcc uses 387 floating point instructions on 32 bit x86, whereas it uses SSE instructions on x86-64.

The 387 floating point registers are 80 bits wide, whereas double is 64 bits wide. This means that intermediate results can have higher precision using the 387 instructions, which can result in a slightly different answer after rounding. (The SSE2 instructions operate on packed 64 bit doubles).

There's a few ways to change the way the compiler operates, depending on what you want:

• If you use the -ffloat-store option on x86 builds, the compiler will discard extra precision whenever you store a value in a double variable;
• If you use the -mfpmath=sse options on x86 builds, along with -msse2 or an -march= switch that specifies an SSE2-supporting architecture, the compiler will use SSE instructions for floating point just as on x86-64. The code will only run on CPUs that support SSE2, though (Pentium-M / Pentium 4 and later).
• If you use the -mfpmath=387 option on x86-64 builds, the compiler will use 387 instructions for floating point just as on x86. This isn't recommended, though - the x86-64 ABI specifies that floating point values are passed in SSE registers, so the compiler has to do a lot of shuffling between 387 and SSE registers with this option.

Why does this floating-point calculation give different results on different machines?

Here's an interesting bit of the C# specifiction, from section 4.1.6:

Floating-point operations may be
performed with higher precision than
the result type of the operation. For
example, some hardware architectures
support an “extended” or “long double”
floating-point type with greater range
and precision than the double type,
and implicitly perform all
floating-point operations using this
higher precision type. Only at
excessive cost in performance can such
hardware architectures be made to
perform floating-point operations with
less precision, and rather than
require an implementation to forfeit
both performance and precision, C#
allows a higher precision type to be
used for all floating-point
operations. Other than delivering more
precise results, this rarely has any
measurable effects.

It is possible that this is one of the "measurable effects" thanks to that call to Ceiling. Taking the ceiling of a floating point number, as others have noted, magnifies a difference of 0.000000002 by nine orders of magnitude because it turns 15.99999999 into 16 and 16.00000001 into 17. Two numbers that differ slightly before the operation differ massively afterwards; the tiny difference might be accounted for by the fact that different machines can have more or less "extra precision" in their floating point operations.

Some related issues:

• C# XNA Visual Studio: Difference between "release" and "debug" modes?

• CLR JIT optimizations violates causality?

To address your specific problem of how to compute an aspect ratio from a float: I'd possibly solve this a completely different way. I'd make a table like this:

struct Ratio
{
    public int X { get; private set; }
    public int Y { get; private set; }
    public Ratio (int x, int y) : this()
    {
        this.X = x;
        this.Y = y;
    }
    public double AsDouble() { return (double)X / (double)Y; }
}

Ratio[] commonRatios = { 
   new Ratio(16, 9),
   new Ratio(4, 3), 
   // ... and so on, maybe the few hundred most common ratios here. 
   // since you are pinning results to be less than 20, there cannot possibly
   // be more than a few hundred.
};

and now your implementation is

public string AspectRatioAsString(double ratio)      
{ 
    var results = from commonRatio in commonRatios
                  select new {
                      Ratio = commonRatio, 
                      Diff = Math.Abs(ratio - commonRatio.AsDouble())};

    var smallestResult = results.Min(x=>x.Diff);

    return String.Format("{0}:{1}", smallestResult.Ratio.X, smallestResult.Ratio.Y);
}

Notice how the code now reads very much like the operation you are trying to perform: from this list of common ratios, choose the one where the difference between the given ratio and the common ratio is minimized.

Is floating point arithmetic stable?

But what if the equation used to calculate the number is the same? can I assume the outcome would be the same too?

No, not necessarily.

In particular, in some situations the JIT is permitted to use a more accurate intermediate representation - e.g. 80 bits when your original data is 64 bits - whereas in other situations it won't. That can result in seeing different results when any of the following is true:

• You have slightly different code, e.g. using a local variable instead of a field, which can change whether the value is stored in a register or not. (That's one relatively obvious example; there are other much more subtle ones which can affect things, such as the existence of a try block in the method...)
• You are executing on a different processor (I used to observe differences between AMD and Intel; there can be differences between different CPUs from the same manufacturer too)
• You are executing with different optimization levels (e.g. under a debugger or not)

From the C# 5 specification section 4.1.6:

Floating-point operations may be performed with higher precision than the result type of the operation. For example, some hardware architectures support an "extended" or "long double" floating-point type with greater range and precision than the double type, and implicitly perform all floating-point operations using this higher precision type. Only at excessive cost in performance can such hardware architectures be made to perform floating-point operations with less precision, and rather than require an implementation to forfeit both performance and precision, C# allows a higher precision type to be used for all floating-point operations. Other than delivering more precise results, this rarely has any measurable effects. However, in expressions of the form x * y / z, where the multiplication produces a result that is outside the double range, but the subsequent division brings the temporary result back into the double range, the fact that the expression is evaluated in a higher range format may cause a finite result to be produced instead of an infinity.

Runtime float calculation vs evaluated calculation

I think it's a matter of floating point calculations

Program with 32Bit ( Any CPU -> prefer 32Bit checked )

Program with 64Bit

Visual Studio is wrapped in 32-bit, so I think that's because QuickWatch runs in 32-bit.

This is like calculating (int)(1f/12) in QuickWatch

So I think it would be better to check what was calculated

Inconsistent multiplication performance with floats

As others have mentioned, various processors do not support normal-speed calculations when subnormal floating-point values are involved. This is either a design defect (if the behavior impairs your application or is otherwise troublesome) or a feature (if you prefer the cheaper processor or alternative use of silicon that was enabled by not using gates for this work).

It is illuminating to understand why there is a transition at .5:

Suppose you are multiplying by p. Eventually, the value becomes so small that the result is some subnormal value (below 2^-126 in 32-bit IEEE binary floating point). Then multiplication becomes slow. As you continue multiplying, the value continues decreasing, and it reaches 2^-149, which is the smallest positive number that can be represented. Now, when you multiply by p, the exact result is of course 2^-149p, which is between 0 and 2^-149, which are the two nearest representable values. The machine must round the result and return one of these two values.

Which one? If p is less than ½, then 2^-149p is closer to 0 than to 2^-149, so the machine returns 0. Then you are not working with subnormal values anymore, and multiplication is fast again. If p is greater than ½, then 2^-149p is closer to 2^-149 than to 0, so the machine returns 2^-149, and you continue working with subnormal values, and multiplication remains slow. If p is exactly ½, the rounding rules say to use the value that has zero in the low bit of its significand (fraction portion), which is zero (2^-149 has 1 in its low bit).

You report that .99f appears fast. This should end with the slow behavior. Perhaps the code you posted is not exactly the code for which you measured fast performance with .99f? Perhaps the starting value or the number of iterations were changed?

There are ways to work around this problem. One is that the hardware has mode settings that specify to change any subnormal values used or obtained to zero, called “denormals as zero” or “flush to zero” modes. I do not use .NET and cannot advise you about how to set these modes in .NET.

Another approach is to add a tiny value each time, such as

n = (n+e) * param;

where e is at least 2^-126/param. Note that 2^-126/param should be calculated rounded upward, unless you can guarantee that n is large enough that (n+e) * param does not produce a subnormal value. This also presumes n is not negative. The effect of this is to make sure the calculated value is always large enough to be in the normal range, never subnormal.

Adding e in this way of course changes the results. However, if you are, for example, processing audio with some echo effect (or other filter), then the value of e is too small to cause any effects observable by humans listening to the audio. It is likely too small to cause any change in the hardware behavior when producing the audio.

Why differs floating-point precision in C# when separated by parantheses and when separated by statements?

I couldn't find a reference to back this up but I think it is due to the following:

• float operations are calculated in the precision available in the hardware, that means they can be done with a greater precision than that of float.
• the assignment to the intermediate result2 variable forces rounding back to float precision, but the single expression for rsult1 is computed entirely in native precision before being rounded down.

On a side note, testing float or double with == is always dangerous. The Microsoft Unit testing provides for am Assert.AreEqual(float expected, float actual,float delta) where you would solve this problem with a suitable delta.

---

Related Topics