SOC: D3D Test Suite

[Joseph Garvin]

On Thu, 2006-04-20 at 11:42 -0400, H. Verbeet wrote:
> "The OpenGL specification is not pixel exact. It therefore does not
> guarantee an exact match between images produced by different GL
> implementations."
> 
> That's from the OpenGL specification, Appendix A.
> Also note that we wouldn't just be comparing different OpenGL
> implementations with eachother, but different OpenGL implementations
> with different DirectX implementations.

What about a test that compared image similarity? Output maybe different
but only up to a certain point -- obvious cases of breakage like the
screen being entirely black are easily detectable, and I'm sure there
are some clever algorithms out there for comparing how close images are
to each other. How far apart each pixel's color is maybe simple but
maybe effective enough. Then mark the test as failed if above a certain
high threshold (say 80% dissimilar).

The tricky details of animation/timing could simply be avoided by having
the tests just draw one frame, which should be enough to detect when
something is really out of whack.