Wine Test Bot 1.7.14 analysis
jwhite at codeweavers.com
Mon Mar 10 07:26:41 CDT 2014
On 03/10/2014 06:08 AM, Henri Verbeet wrote:
> On 10 March 2014 03:13, Jeremy White <jwhite at codeweavers.com> wrote:
>> I wrote a script to analyze winetestbot results on all of the testbot vms
>> except the win8 vms (they are just too broken to try to analyze right now).
>> I'm trying to get a handle on the nature of the bot failures; this current
>> script looks for consistent failures (partly because a consistent failure
>> that goes green is a win, and I badly want to track wins).
>> My results are here:
> Note that e.g. the win2000 testbot doesn't have results for all runs.
> It looks like this causes the script to classify some failures that
> should be "fixed" as intermittent failures. That might in turn cause
> someone to draw wrong conclusions about e.g. the ddraw tests, if they
> didn't pay enough attention to wine-patches.
Yes; not just win2k, but the win7u bot is unreliable, and one of the
other win7 bots and one of the vista bots have a few drop outs as well.
But my code, in theory, skips holes in the data, so long as the data
stays in line.
In other words, a pattern like this:
where S is success, F is failure, and - is missing data, is considered
'fixed'. A pattern like this:
is considered 'consistently failing'. All other patterns are considered
Note that it's only against the newtb vms; so you'll see the claim that
d3d9:stateblock is fixed, but there is one non newtb machine where it
More information about the wine-devel