Cascading test failures

[Greg Geldorp]

Ideally, tests should be independent on each other. That's not the case right now. A clear example is msi:package, which often fails (on XP) with 1574 errors, see http://test.winehq.org/data/tests/msi:package.html. For all the cases I've checked, when these 1574 errors occur the preceding msi:install test timed out (the reverse is not true, I see msi:install timeouts followed by a successful msi:package). My current theory is that the Windows Installer service gets confused when the msi:install test times out and is bluntly terminated by the winetest wrapper.

A possible solution might be to introduce a cleanup mechanism invoked by winetest.exe when it detects a timeout or crash in one of the test executables. After terminating the test executable, winetest.exe would then call "testname_test.exe --cleanup failed_test_name". The --cleanup argument could be handled by the tests main(), just like --list is handled by main(). A test executable would be able to register a cleanup function by setting a global function pointer to its cleanup function. In the case of msi_test.exe, that cleanup function could try to stop and restart the Windows Installer service.

Does this sound worthwhile to pursue? Any other/better ideas?

Ge.