Wine regression testing: PIT_

[Molle Bestefich]

Mike McCormack wrote:
> To do a valid regression test, you really want to run it on a system
> that worked in 2003, so you probably should be compiling it and running
> it on Slackware 9.0 or rather than Slackware 10.1, or whatever system
> you were using back then.

Probably.  I don't require 100% accuracy however, so I won't be doing that.

It's obviously too much of a hassle to go find out which operating
systems were in use back then and install those on my PC just because
I really "should" in order to get picture perfect results, isn't it?
99.9% of the time, it's stuff in Wine that's broken, not stuff in Linux, right?


> For example, if you tried to use Wine from a couple of years ago on the
> latest Fedora Core, even if you get it to compile, it won't run, because
> of problems with exec-shield and who knows what else.

That's exactly what should be fixed.
Stuff that prevents older versions from running *at all*.

You know where the problems are;
I know where *some* of them are;
I'm willing to do volunteer work to rectify the situation for everyone
else that wants to do Wine regression testing, but need cooperation
from you guys before I can do any good.


wino at piments.com wrote:
> It seems that the result is, as you say, regression testing is a PITA and
> as a result often gets skipped.

I see that we agree :-)


Marcus Meissner wrote:
> We adapt to the upstream packages. We cannot control their changes.
>
> Like alsa using "interface" as keyword in their headers.
> Or glibc changing.

No, but we can adapt to them.

And if we are serious about wanting people to do regression testing
and finding the patches that breaks things, we SHOULD adapt to them.

It seems that you and others have lots of examples of things that
break Wine but is not Wine's fault.
Let's get those things *fixed* in older release tarballs, so that
people can do regression testing.

We actually /ask/ people to do regression testing, here f.x.:
http://www.winehq.org/site/docs/winedev-guide/x1344
to find a patch that breaks things.  Some developers has probably
recommended that approach on the mailing lists too, I know I have.

The approach is useless however, until these simple fixes are applied
to the tarballs (preferably through the versioning system).