Direct3D Game test framework

Stefan Dösinger stefan at codeweavers.com
Mon Jun 25 13:35:48 CDT 2007


> cxtest has already impressive infratructure (installation, creation of
> clean environment) and nice scripting. However, it requires Metacity WM
> and VNC.
The Metacity WM doesn't really concern me, the VNC thing however does. This 
makes game testing harder since VNC isn't a part of the average game setup. 
We don't have to transfer all the rendering screenshots over VNC though. I am 
CCing Martin Pilka on this, maybe he can give us some more insight.

> I'm personally using AutoHotkey(http://www.autohotkey.com) as an
> automatic testing framework backend. It's a Windows application, so it
> can be run through Wine itself and it's sandboxed from X11.
It looks interesting, this would also allow us to run automated tests on 
Windows, perhaps to track the effect of driver updates, etc.

> I can imagine Winebot packages to be easily converted to an automatic
> testing engine. Winebot actually covers automated installation and
> creation of clean independent environment.
I don't know if we have to automate the installations as well. Reinstalling 
the apps excludes a few factors and makes the tests more repeatable, on the 
other hand it is not something that occurs in the average gaming environment. 
Not reinstalling can bring up problems related to the game's state on the 
disk, like fragmenting game caches. When we see a problem we can reset the 
environment manually to exclude the disk content as the cause. Automated game 
installation is a nice to have, but not a strict requirement IMHO.

Not reinstalling would ease the use of some controller application that runs 
inside wine, since constant installing and removing of the app can leave 
garbage in the C drive and the registry over time. On the other hand, an 
external application could remove the wineprefix and start with a completely 
clean Windows environment.

> Running applications with different log levels, tracing output,
> comparing screenshots with different levels of fuzzy comparaion
> - it's not faraway.
How would fuzzy comparison work?



More information about the wine-devel mailing list