[Bug 9916] "make test" usually fails
wine-bugs at winehq.org
wine-bugs at winehq.org
Thu Sep 8 11:27:33 CDT 2011
http://bugs.winehq.org/show_bug.cgi?id=9916
--- Comment #95 from Dan Kegel <dank at kegel.com> 2011-09-08 11:27:31 CDT ---
Wine's tests are in a lot better shape than when this bug was filed,
but still, odds are that "make test" will fail on the average machine.
I've been collecting a blacklist of bad and/or flaky tests.
Here's one way to use it to measure how unreliable the flaky tests are
on a particular system:
wget -c http://winezeug.googlecode.com/svn/trunk/buildbot/dotests.sh
wget -c http://winezeug.googlecode.com/svn/trunk/buildbot/dotests_blacklist.txt
rm -rf ~/.wine
winetricks nocrashdialog
while true
do
make testclean
sh dotests.sh flakytests
done
Out of 40 or so runs on one system,
five dll/foo.ok files exhibited crashes or failures:
7 dlls/wininet/tests/urlcache.ok (bug 28038, workaround: clear cache between
runs)
4 dlls/msctf/tests/inputprocessor.ok (bug 28288)
4 dlls/kernel32/tests/pipe.ok (bug 28257)
3 dlls/user32/tests/static.ok (bug 20149)
2 dlls/urlmon/tests/url.ok (bug 28108)
So about half of 40 runs of "make test" would probably fail because of
flaky tests on this machine without manual workarounds.
Your mileage may vary. Other systems tickle other bugs.
That blacklist and script can also look for tests that have failed on any
of my systems repeatably.
Running 'sh dotests.sh badtests' twice on that same (64 bit) system, I got
2 dlls/ieframe/tests/ie.ok FAILED; blacklist says BAD64 , see bug 26768 .
2 dlls/kernel32/tests/process.ok FAILED; blacklist says NOTTY , see bug
28220 .
2 dlls/oleaut32/tests/tmarshal.ok FAILED; blacklist says BAD64 , see bug
26768 .
1 dlls/user32/tests/msg.ok FAILED; blacklist says SYS , see bug 12053 .
2 programs/wscript/tests/run.ok FAILED; blacklist says BAD64 , see bug
28285 .
so "make test" will always fail on this system because of msg tests
(probable workaround: run a blessed window manager) and because
win64 isn't finished. And a kernel test will always fail because
I tend to redirect the output of "make test".
--
Configure bugmail: http://bugs.winehq.org/userprefs.cgi?tab=email
Do not reply to this email, post in Bugzilla using the
above URL to reply.
------- You are receiving this mail because: -------
You are watching all bug changes.
More information about the wine-bugs
mailing list