RFC How to get rid of "always new" TestBot false positives?
fgouget at codeweavers.com
Fri Mar 27 12:05:42 CDT 2020
On Fri, 27 Mar 2020, Zebediah Figura wrote:
> (2) Remi's comment inspired another option, though it's definitely more work
> for François: lookup the line number in the relevant source file and compare
> the source line.
That's pretty hard for three reasons:
* The test reports are analyzed on the TestBot server which does not
have access to the Wine source (giving it access to a Wine checkout
is on the todo list for other reasons).
* What it needs is the patched source code whereas what it would have
access to is the unpatched source.
* On the server multiple processes may need access to the patched source
code at the same time.
Also a number of tests loop over the test cases so that they all involve
the same lines but the failures are still distinct. For instance (from
the set of "new" failures in a recent "32 bit WineTest: base VMs" job):
d3d11.c:2729: Test failed: Test 22: Got unexpected hr 0x8007000e (format 0x9).
d3d11.c:2729: Test failed: Test 23: Got unexpected hr 0x8007000e (format 0x9).
d3d11.c:2729: Test failed: Test 24: Got unexpected hr 0x8007000e (format 0xf).
d3d11.c:2729: Test failed: Test 25: Got unexpected hr 0x8007000e (format 0x13).
These all come from the same line but it would be wrong to say that a
failure on iteration 22 is equal to the one on iteration 25.
Francois Gouget <fgouget at codeweavers.com>
More information about the wine-devel