Sources of noise on test.winehq.org?
paul.vriens.wine at gmail.com
Thu Apr 9 05:41:18 CDT 2009
Dan Kegel wrote:
> (This was last discussed in February, e.g.
> http://www.winehq.org/pipermail/wine-devel/2009-February/073060.html )
> The results on test.winehq.org seem more variable than one
> would expect, which makes it harder to gauge wine's progress.
> I can think of two sources of noise:
> 1) 32 and 64 bit results are mixed together
> 2) we don't have a stable stable (sic) of machines
> running the tests
> Removing these two sources of noise might be as simple as
> 1) omit 64 bit results, and
> 2) omit computers for which results are not
> consistently available throughout the time range
> being displayed
> Shouldn't be too hard for someone to whip together an
> alternate report that did that. Wish I had the time to...
> - Dan
So what constitutes a stable machine? I for one am using VMware for most
of my boxes which are clean installs but up-to-date with patches and
such. After running winetest they will be reverted to the last correct
snapshot (winetest still leaves a lot of rubbish around that could
potentially influence a next run).
One thing I'd also like to see (on first glance) on test.winehq.org is
whether we are dealing with a real box or a virtualized one.
Side note 2:
Our index per build pages are almost 4MB in size. Splitting things up
will also cut down on that.
More information about the wine-devel