[Bug 48209] New: Ignore random parts of test failure messages

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=48209

            Bug ID: 48209
           Summary: Ignore random parts of test failure messages
           Product: Wine-Testbot
           Version: unspecified
          Hardware: x86
                OS: Linux
            Status: NEW
          Severity: normal
          Priority: P2
         Component: unknown
          Assignee: wine-bugs at winehq.org
          Reporter: fgouget at codeweavers.com
      Distribution: ---

The TestBot needs to detect if a patch introduces NEW test failures. To do so
it compares the failure messages with those from reference WineTest runs. But
in some cases part of the message changes from one run to the next, causing the
failure to look new.

Of course in all cases the ideal solution is to fix the test so it does not
fail.

Barring that there are also cases where the varying value is not really useful
(e.g. handle or pointer values) and in that case it should just be removed from
the message.

But there may be cases where knowing the value is still useful. So one solution
would be to tag such values so the TestBot knows it can ignore them when
comparing failure messages. For instance they could be enclosed in double
braces, curly braces, or other.

-- 
Do not reply to this email, post in Bugzilla using the
above URL to reply.
You are receiving this mail because:
You are watching all bug changes.