Google Summer of Code Test Suite project

[Francois Gouget]

On Wed, 19 Mar 2008, Austin English wrote:
[...]
> Using a program like AutoHotkey to install a program and test its
> features would be a better route IMHO.

I agree but I'm not sure AutoHotKey is quite ready yet. But I think that 
getting a robust graphical testing environment would be hugely 
beneficial for Wine.

There are a couple of options to get there:
 * cxtest
   http://www.cxtest.org/
   It has tools to record and replay clicks.
   One big drawback is that in its current form it is very Unixy, that 
   is you need bash and VNC in order to record an application 
   installation. This prevents running it on Windows which is really 
   needed imho. On the other hand it can handle both Unix and Windows 
   scripting. It is also missing support for some things like menus and 
   combo boxes.
   However it has a great backend for automating and reporting the 
   tests, identifying known issues and automatically grouping them 
   together, etc.

 * AutoHotKey
   http://www.autohotkey.com/
   It runs on Windows and has a very nice 'record' feature: you can 
   start it in record mode, install your application, stop the recording 
   and presto, you have a script ready to replay. However it generates 
   scripts that are coordinates based (i.e. left-click at 123x456), 
   rather than 'controls' based (i.e. left-click on the 'Ok' button in 
   the 'Step 3 windows'). That worries me as I'm not convinced these 
   coordinates will match well when the application is run in Wine 
   (potential font issues). I suspect that even on Windows there could 
   be issues if the user changes the font size, DPI or stuff like that. 
   However I don't know how commercial graphical regression testing 
   tools handle that so maybe it's not such a big issue.
   Also AutoHotKey has not yet been integrated in a framework with a 
   good backend.

So one thing that would be very good for Wine would be to improve 
AutoHotKey so it really does all that we need. It should be good enough 
that open-source projects would want to use it for their own regression 
testing on Windows, so that we would just reuse their scripts. However I 
don't know if it's suitable for a GSoC project. Also it may be more of 
an AutoHotKey GSoC than a Wine one.

-- 
Francois Gouget <fgouget at free.fr>              http://fgouget.free.fr/
    Indifference will certainly be the downfall of mankind, but who cares?