[PATCH] d3dx9: Reduce rounding error in D3DXQuaternionToAxisAngle

[Chris Robinson]

On 11/09/2017 05:41 AM, Alex Henrie wrote:
> That actually makes the rounding error worse on my machine:
> 
> printf("acos:  %.9f\n", 2*acos(10.0f/22.0f));
> printf("acosf: %.9f\n", 2*acosf(10.0f/22.0f));
> printf("atan:  %.9f\n", 2*atan2f(sqrtf((1.0f - 10.0f/22.0f) * (1.0f +
> 10.0f/22.0f)), 10.0f/22.0f));
> 
> acos:  2.197868979
> acosf: 2.197869062
> atan:  2.197868824

Should be noted that the difference between 2.197869062 and 2.197868824 
is a single bit on floats (0x1.1953c6p+1 and 0x1.1953c4p+1 respectively, 
in hex float format). 2.197868979 is a double-precision result 
(0x1.1953c54cea1bp+1), which will get truncated to one of the other two 
values when assigned to a float depending on how it rounds.

Interestingly, when I do:

volatile float f = 22.0f;
printf("%.9f", 2*acosf(10.0f/f));

I get 2.197868824, the same as your atan2f result. I only get 
2.197869062 when I do:

printf("%.9f", (float)(2*acos(10.0/f)));

Note that 10 is a double. If I give a single-precision input:

volatile float f = 22.0f;
printf("%.9f", 2*acos(10.0f/f));

I again get 2.197868824 (0x1.1953c4p+1).