ANNOUNCE: Wine/Mono Regression Test Suite

[Matt Jones]

Hi Everyone -

I'm working on getting mono to run happily under wine, thus opening
wine (& linux) up to a much larger range of .NET applications.
Unfortunately, mono-under-wine fails in rather cryptic ways at the
moment, so I've found that running the mono regression test suite has
given me a better idea of how to approach debugging mono/wine.

Double unfortunately, running the mono regression test suite on linux,
windows & wine presents a large series of interesting problems. So
I've developed a python driver for the tests that spits out some html
highlighting interesting test results and unifies the testing method
on all three platforms.

The code is available at [1]
There is a git repository at [2]

The scripts can generate static html pages for tests run, which are
available at:
[3] (you can download the entire set of gzipped pages from [4])

Many of the tests fail as a result of bug 8757, which causes mono to
lock up while running a GC cycle. For now the scripts kill test cases
after 5 minutes, but I've also included patches against wine to
prevent the lockup (horrible hacks - disable modal message boxes & the
debugbreak function). If you want to save time when running the tests,
you can apply these patches in a git branch and test against that.

The tests are designed to require little-to-no user interaction -
after an initial linux & windows test run are performed (sample test
data at [5]), an automated update script (nightly-update.sh) can be
called to automatically update wine, run the tests and publish the
results.

Feedback is greatly appreciated - the tests are very temperamental, so
it wouldn't surprise me at all if things break on other machines (I've
tested pretty extensively and documented any non-automated actions
necessary, but I want to make sure I didn't miss anything.)

Thanks,
Matt Jones

[1] http://mhjones.org/wine/wine-tests/wine-test-reader.tar.gz
[2] http://mhjones.org/wine/wine-tests/wine-test-reader.git
[3] http://mhjones.org/wine/wine-tests/htmltest/index.py.html
[4] http://mhjones.org/wine/wine-tests/htmltest.tar.gz
[5] http://mhjones.org/wine/wine-tests/test-runs-example.tar.bz2