Proof of concept - The crashbot

[Austin English]

On Fri, Sep 16, 2011 at 10:38, Dan Kegel <dank at kegel.com> wrote:
> Hi Bruno,
> it would of course be great.  Check out
> http://code.google.com/p/winetricks/source/browse/trunk/tests/winetricks-test
> for starters, we could just run that (or a subset of it) daily.
> It's fairly reliable, as I ran it manually every week for a while
> and weeded out a lot of flaky bits.
>
> When I get a chance, I'll try integrating that into buildbot.
> - Dan
>
> p.s. As Henri mentioned, it's been on people's minds for years.
> The graveyard is littered with previous attempts, e.g.
> http://web.archive.org/web/20070209153523/http://www.cxtest.org/
> I think it's important to do the easy part first and not do anything fancy.

You can see appinstall here:
http://code.google.com/p/winezeug/source/browse/#svn%2Ftrunk%2Fappinstall

which used a shell script to automate testing with AutoHotKey
(checking for expected crashes was supported as well). Automated app
testing is very useful, but at the others pointed out, it's quite a
pain to get right and have it be reliable.

That said, simply running an application and checking for a crash
isn't that difficult, it's the more tedious testing of menus/input
that get really nasty. And installation of apps that don't have a
silent option is also annoying..

Good luck! Feel free to contact me privately if needed.

-- 
-Austin