[Bug 48655] New: Automate checking the WTBS emails

[WineHQ Bugzilla]

https://bugs.winehq.org/show_bug.cgi?id=48655

            Bug ID: 48655
           Summary: Automate checking the WTBS emails
           Product: Wine-Testbot
           Version: unspecified
          Hardware: x86
                OS: Linux
            Status: NEW
          Severity: normal
          Priority: P2
         Component: unknown
          Assignee: wine-bugs at winehq.org
          Reporter: fgouget at codeweavers.com
      Distribution: ---

See bug 48653 for a description of the WTBS and how to check the jobs created
when running it on the TestBot.

When a 'wine-devel' job completes the TestBot sends one or two emails.
* One of the emails simply sends the task reports to the patch author.
* The other is only sent if a task generated new errors.

The point of the TestWTBS.emails script would be to check that:

1. The TestBot sent the task report email for every Wine patch and not for any
patch email. This would be based off of the WTBS.Patch.Disposition field.

2. The TestBot sent a 'found new errors' email only when new errors are
expected. This would be based off of the 'WTBS.*.NewFailures' property or the
expected errors list.

However for both these checks the script would need access to the emails sent
by the TestBot. The simplest option would be for the tester to set the
WinePatchToOverride property and save the emails to a separate mbox and give
the patch to that mbox to the script.

As in the TestWTBS.patches case, the script would only care about a few lines:

* The 'From' lines to delimit the emails.

* The 'Subject' lines to match them to TestBot jobs and thus to the set of WTBS
properties.
  Note that this would not rely on the 'TestBot job xxx results:' part since
the new failures email are lacking it. Instead it would match the rest of the
subject to the job's Remarks field.

* The 'I think I found new failures' line to identify emails reporting new
errors.


The script could later be expanded to check the content of the email or
attachments more in details if needed.

-- 
Do not reply to this email, post in Bugzilla using the
above URL to reply.
You are receiving this mail because:
You are watching all bug changes.