Wine regression testing: PIT_

[wino at piments.com]

Very good points. It seems a lot of this has been accepted over the years  
on the basis that wine was alpha software and in that context , the past  
is the past - get the new release etc.

It seems that the result is , as you say, regression testing is a PITA and  
as a result often gets skipped. The code base has certainly advanced a lot  
but every release seems to create as many problems as it solves.

I managed to get an app working in Feb, luckily I tarballed the whole  
.wine installation since I am now unable to reinstall this app under any  
version of wine and make it work.

Wine is now officially beta and this is probably the stage at which some  
serious project management has to come into play.

Maybe someone has to take a week off from coding and review the overall  
picture. The core devs have a huge task so it must be made easier for  
people like yourself who clearly have the expertise to contribute to get  
involved.


A working wine installation seems to have the life expectancy of a  
butterfly. If any wider interest is to be generated: like people who  
actually want to use the code rather than rewrite it , it has to stabalise  
and be done more rigourously (for example documentation syncronisation -  
last time I looked the man page was still telling me how to setup the  
config file, and winever simply reports CVS , irrespective of what build I  
install from.). Hopefully beta is where this will start to happen.

Now is a crucial time to Linux , it has reached take-off velocity.

If wine is to contribute to the migration path, and there is a the  
opertunity for a great contribution, it seems to me this has to be  
adressed now.

I dont mean to imply nothing is being done behind the scenes, I'm sure you  
guys are already working on these issues.

Hopefully 0.9 will be a turning point for Wine . Many thanks for the work  
that has gone into getting this far.

Best regards.


On Wed, 26 Oct 2005 12:04:46 +0200, Molle Bestefich  
<molle.bestefich at gmail.com> wrote:

> Regression testing with Wine is a pain in the butthole.
>
> For very bad reasons: Stuff that is simple to fix, but haven't been.
>
> I'd like to help improve that situation.
>
>
> Example:
> ========
>
> I want to find which patch ruins an application.
> The application, according to one note, worked in 2003.
> I decide to try and see how it really runs with the 2003 version of
> Wine mentioned in the note.
>
> I then spend several days pulling various releases, just trying to  
> compile them.
> They all fail miserably, because some fix required to compile
> correctly with newer Linux versions is /missing/.
>
> For example,
>  - any Wine before 2004-01-02 won't work because it won't compile
> against newer ALSA versions.
>  - any Wine before 2003-12-01 needs patches to the build system before
> 'make depend' will even run.
>
> I've been at it for a week and I've managed to fix the above two.
> _They're really simple things, but they take a lot of time to figure  
> out._
>
>
> Currently I'm working with two other problems:
>  - Wine fails to correctly invoke the debugger. Winedbg shows 'Usage:
> blah blah'.  Seems 'wine' calls it with incorrect parameters.
>  - Wine fails to start any application, complaining about 'kthreads'
> something or the other.
>
> I imagine that if I spend a few more weeks on the above problems, I
> can find patches in Wine CVS that fixes the exact problems I'm seeing.
>
> I've already wasted a week of my time on problems that someone else
> already fixed.  Spending two or more weeks doing this is not exactly
> on my fun-things-to-do-before-I-die list.  I'd like to see a general
> solution to this entire problem.
>
>
>
> What can be done to alleviate?
> ==============================
>
> Once I find a patch which makes an earlier version actually compile,
> I'd like to propose that for a backport.
>
> If approved/applied, the next person that checks out the release would
> check out a version containing the backported patches.
>
> That would fix the problem and allow people to do proper regression
> testing, so developers and QA people can spend their time doing
> development and QA instead of redoing already done effort trying to
> get an older version to run.
>
> I'm not sure how this would work with CVS.  I'm from an SVN world
> where tagging is a 0-cost tree-copy operation, and any patches can be
> applied to the tag with history and all showing that the tag's been
> modified.  If it can't work with CVS, perhaps the tags can be created
> in Troy's SVN repo and patches applied there?  Not sure.  Suggestions,
> please, people.
>
>
>