[Bug 48091] New: Deduplicate identical consecutive failures

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=48091

            Bug ID: 48091
           Summary: Deduplicate identical consecutive failures
           Product: Wine-Testbot
           Version: unspecified
          Hardware: x86
                OS: Linux
            Status: NEW
          Severity: normal
          Priority: P2
         Component: unknown
          Assignee: wine-bugs at winehq.org
          Reporter: fgouget at codeweavers.com
      Distribution: ---

kernel32:debugger has an intermittent failure which results in a variable
number of duplicate consecutive failures:
https://www.winehq.org/pipermail/wine-devel/2019-November/153982.html

Specifically, reports are known to have had between 0 and 5 such lines in a
row:
debugger.c:320: Test failed: GetThreadContext failed: 5

Most cases of duplicate consecutive failure lines like this are bugs where the
test iterates on an array of test data and author forgot to include the loop
index in the failure message. Those are bugs and should be fixed if only so
that developers investigating the failure can figure out which array entry
caused the failure (maybe a job for Coccinelle?).

But this kernel32:debugger failure iterates over the test's threads and it does
not look like there is anything it could add that would make the failure
message unique without also making it random, which would break the new failure
detection (e.g. if the thread handle were included in the message).

Currently, whenever these failures happen more times in a test than in the last
WineTest run the extra failures are reported as new. After bug 47998 is fixed
such errors will only be reported as new if their number exceeds the maximum on
record.


So there are four options:

1. Fix this kernel32:debugger bug and hope there are no other such cases (see
bug 48052).

2. Tweak kernel32:debugger so it aborts the loop on the first failure. Knowing
that the failure can happen a random number of times is useful in that it means
it does not just happen on the first or last thread. But maybe it's not that
important (see also bug 48052).

3. Rely on the fix for bug 47998 to make such false positives rare enough that
they are not a nuisance.

4. Deduplicate identical consecutive failure lines.
   In this case it should be done while extracting the failures from the test
report. That's because once the failures have been extracted, any trace that we
present between them has been lost. But such traces provide context and thus
should prevent the deduplication from happening. This is also a place where
line numbers can be taken into account (they are ignored when comparing
failures since patches will change them).

-- 
Do not reply to this email, post in Bugzilla using the
above URL to reply.
You are receiving this mail because:
You are watching all bug changes.