War on conformance test suite failures!

Dan Kegel dank at kegel.com
Mon Feb 4 17:50:55 CST 2008

Great progress was made at Wineconf last year on
getting our test suite closer to always passing on wine, but
http://test.winehq.org/data/200801301937/#Wine shows
that, according to the results collected from people who run
programs/winetest on Wine, twelve DLLs' tests are currently failing.
(Let alone the test failures on Windows!)
http://bugs.winehq.org/show_bug.cgi?id=9916 also
talks about the fact that "make test" shows failures.

This situation greatly confuses new developers,
who assume that any test failure is somehow their
fault.  And that's a reasonable assumption, let's make it so!
It would make validating a new Wine development
environment easy -- it'd be great to be able to say
  If it doesn't pass "make test" with no errors, something's
  wrong with the build environment.

What's the best way to encourage people to fix these problems?
Should we designate some upcoming day, week, or month
as a conformance test bug squishing party?
Should we offer prizes (or at least notoriety) for
the person who does the best job fixing
conformance test suite bugs (as judged by Alexandre
or his designate)?
Or is a simple call to arms enough?

More information about the wine-devel mailing list