[PATCH] d3dx9/tests: Provide 8 significant digits in matrix reference values

[Alex Henrie]

2017-11-09 1:52 GMT-07:00 Henri Verbeet <hverbeet at gmail.com>:
> Thanks for fixing these.

You're welcome!

> 2017-11-09 6:10 GMT+01:00 Alex Henrie <alexhenrie24 at gmail.com>:
>> Signed-off-by: Alex Henrie <alexhenrie24 at gmail.com>
>> ---
>> Fixes test failures on 64-bit Linux.
>>
>> A 32-bit float can hold 7.2 decimal digits, so it's best to provide 8 to
>> make sure that the correct values are accurately represented.
>>
> If it were up to me, I'd prefer these to be in the same %.8e format
> the failure messages use.

OK. Let's wait and see what Matteo wants.

>> The correct outputs of functions that take simple floats as inputs were
>> computed by feeding equations into Wolfram Alpha.
> I think that's probably fine, but note that we don't care about the
> "correct" values so much as "whatever Windows produces". Hopefully
> those would be reasonably close, but they're by no means guaranteed to
> be the same.

That's a good point, but unless there's an application that depends on
the specific rounding error from Windows, the exact values will be
fine. Using exact values will also keep the tests passing if the
rounding error changes between Windows versions or CPU architectures.

-Alex