Flaky test update
Dan Kegel
dank at kegel.com
Wed Sep 24 09:04:24 CDT 2008
Here's some data over 1042 test runs over the last
three weeks, all on a dual core Ubuntu 8.04 linux box with
a cheap nVidia GeForce 8500 GT graphics card.
Three tests crashed from 1% to 10% of the time (as opposed to failed):
ddraw 136
d3d9 48
d3d8 12
Three tests always failed:
urlmon:protocol.c 1042
dsound:ds3d8.c 1042
dsound:ds3d.c 1042
Two tests usually failed:
user32:msg.c 940
user32:input.c 756
Nine tests failed from 0.1% to 20% of the time.
(Even tests that only fail once in a thousand runs
are bad, since they cause spurious patch failure reports.
I've blacklisted all the tests listed here, which means
patchwatcher *ignores* these tests, even if they have
valid failures.)
comctl32:tooltips.c 206
user32:win.c 205
riched20:editor.c 174
kernel32:thread.c 124
wininet:http.c 59
ddraw:visual.c 7
advapi32:service.c 7
kernel32:debugger.c 2
urlmon:url.c 1
Of those nine test, the following six had a fairly small number of
characteristic error messages:
comctl32:tooltips.c
riched20:editor.c
kernel32:thread.c
advapi32:service.c
kernel32:debugger.c
urlmon:url.c
And here are those error messages:
206 comctl32:tooltips.c:223: Test failed: CustomDraw run 0 stages
0, expected 1
174 riched20:editor.c:5322: Test failed: Cursor is at 0 instead of 4
174 riched20:editor.c:5328: Test failed: Cursor is at 4 instead of 9
174 riched20:editor.c:5334: Test failed: Cursor is at 0 instead of 4
20 kernel32:thread.c:1181: Test failed: WaitForMultipleObjects failed
4 kernel32:thread.c:1181: Test failedthread.c:130: Test failed:
WaitForSingleObject failed
4 kernel32:thread.c:1181: Test failedthread.c:144: Test failed:
WaitForSingleObject failed
17 kernel32:thread.c:1184: Test failed: WaitForMultipleObjects failed
8 kernel32:thread.c:1184: Test failed: thread.c:144: Test
failed: WaitForSingleObject failed
22 kernel32:thread.c:1186: Test failed: WaitForMultipleObjects failed
11 kernel32:thread.c:1186: Test failed: thread.c:144: Test
failed: WaitForSingleObject failed
120 kernel32:thread.c:130: Test failed: WaitForSingleObject failed
1 kernel32:thread.c:130: Test failedthread.c:1181: Test failed:
: WaitForSingleObject failed
134 kernel32:thread.c:144: Test failed: WaitForSingleObject failed
6 advapi32:service.c:2032: Test failed: Expected success
7 advapi32:service.c:2043: Test failed: Expected success
7 advapi32:service.c:2047: Test failed: Expected success
2 kernel32:debugger.c:300: Test failed: exit code = 00000000
1 urlmon:url.c:2125: Test succeeded inside todo block:
unexpected OnProgress_CONNECTING
1 urlmon:url.c:2129: Test failed: expected OnProgress_SENDINGREQUEST
1 urlmon:url.c:2131: Test failed: expected OnResponse
All the data is online at http://kegel.com/wine/patchwatcher/results
if you want to look for details.
- Dan
More information about the wine-devel
mailing list