Sources of noise on test.winehq.org?

[Paul Vriens]

Dan Kegel wrote:
> (This was last discussed in February, e.g.
> http://www.winehq.org/pipermail/wine-devel/2009-February/073060.html )
> 
> The results on test.winehq.org seem more variable than one
> would expect, which makes it harder to gauge wine's progress.
> 
> I can think of two sources of noise:
> 1) 32 and 64 bit results are mixed together
> 2) we don't have a stable stable (sic) of machines
> running the tests
> 
> Removing these two sources of noise might be as simple as
> 1) omit 64 bit results, and
> 2) omit computers for which results are not
> consistently available throughout the time range
> being displayed
> 
> Shouldn't be too hard for someone to whip together an
> alternate report that did that.  Wish I had the time to...
> - Dan
> 
> 
So what constitutes a stable machine? I for one am using VMware for most 
of my boxes which are clean installs but up-to-date with patches and 
such. After running winetest they will be reverted to the last correct 
snapshot (winetest still leaves a lot of rubbish around that could 
potentially influence a next run).

Side note:
One thing I'd also like to see (on first glance) on test.winehq.org is 
whether we are dealing with a real box or a virtualized one.

Side note 2:
Our index per build pages are almost 4MB in size. Splitting things up 
will also cut down on that.

-- 
Cheers,

Paul.