WineTest failure patterns preview

Francois Gouget fgouget at codeweavers.com
Fri Apr 23 07:54:29 CDT 2021


I did not get to send this patch set yet so here's a preview:

* Windows test patterns
  http://fgouget.free.fr/tmp/winepatterns/patterns-tb-win.html

* Wine test patterns
  http://fgouget.free.fr/tmp/winepatterns/patterns-tb-wine.html

So for each test unit that had failures you have: 
* one line per test configuration (report tag)
* one row per WineTest build (daily commit)

Each cell has a one character description of the test result (with more 
details in the tooltip) and is color coded to help identify patterns. 
Each 'failure' cell also links to the corresponding report page.


The main goal of these pages for me is to simplify detecting new 
failures in the WineTest results... and not just in cases where there 
was no failure before since there are so many cases with preexisting 
failures already. In the patterns an increase in the number of failures 
shows up as a color change which makes them easy to spot.

A secondary goal is to allow comparing the before/after results when a 
test configuration (VM or otherwise) is modified.

A side effect is that these can also help identify the different failure 
modes. For instance comctl32:monthcal fails on wednesdays, except in 
Korean where it's all the time:
http://fgouget.free.fr/tmp/winepatterns/patterns-tb-win.html#comctl32:monthcal


To help with these tasks the page also sorts the test units to show 
those that look like they have new failures first. So starting the 
review from the top is most likely to point to the new issues.


-- 
Francois Gouget <fgouget at codeweavers.com>



More information about the wine-devel mailing list