make test drill

[Stefan Dösinger]

Am Samstag, 6. Oktober 2007 20:28:36 schrieb Paul Vriens:
> Jeremy White wrote:
> > So, in a radical break from tradition, we're trying to
> > accomplish something useful at Wineconf.
> >
> > Specifically, we're making 'make test' work for everyone,
> > not just Alexandre.
> >
> > Maarten Lankhorst is maintaining a tree of all of our
> > test related patches.
> >
> > So, for those that want to play, the thing to do is to
> >  git fetch git://repo.or.cz/wine/testsucceed.git master:testsucceed
> >  git rebase testsucceed
> >
> > Then you can do a make test (with a clean .wine directory)
> > and share your results.  It's useful to use Dan's filter:
> >    egrep '__test|make.*ok|Backtrace' <your-log-file-here>
> >
> > Up through the end of day tomorrow, you can send me the results
> > of that egrep and I'll tally and triage.
> >
> > If you have a patch, send it in, and CC Maarten so he can
> > update it.
> >
> > The goal is to have the number of failures as close to 0
> > as possible, and for the non 0 cases, to at least understand
> > each failure.
> >
> > Cheers,
> >
> > Jeremy
>
> Hi,
>
> It's nice to fix all the test failures when running them in Wine. We
> however have several tests that still fail on Windows so how can we be sure
> the test results in Wine are correct? Some that are failing on a lot of
> Windows boxes:
I brought that up on WineConf, but I want to add it here too: It is a bit 
tricky to draw the line between a broken test and a badly set up Windows 
installation. There are situations where applications are broken on a 
specific Windows installation. As an example, d3d tests may fail when run in 
VMware. VMWare has a D3D to GL wrapper simmilar to Wine. This is some work in 
progress piece of code which fails to run many apps. So if the d3d test fails 
in VMWare then it is likely VMWare's bug. The problem can occur in other 
parts too, like an application overwriting system global libraries(aka DLL 
hell), or malware, copy protection rootkits, security apps, system tuners, 
etc. For example, there is a problem with Steam on Wine that occurs on 
Windows too, triggered by various tuning apps.

So a failure on Windows doesn't necessarily mean that the test is wrong.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part.
Url : http://www.winehq.org/pipermail/wine-devel/attachments/20071009/faecc473/attachment.pgp