Wine regression testing: PIT_

[Molle Bestefich]

Jonathan Ernst wrote:
> > The approach is useless however, until these simple fixes are applied
> > to the tarballs (preferably through the versioning system).
>
> How do you decide what things have to be fixed in old tarballs ? Do you
> test each old tarballs every now and then and apply the corresponding
> patches to them ?

Sure, I wouldn't mind checking out alpha releases from the past
year or so and testing if they compile.

> What if someone wants to use an old tarball in an old system that would
> be broken because of this patching ? You would have to maintain a lot of
> old tarballs version...

They'd check out the original tagged version.

> I don't think it is feasible.

I'll show you.

The KDE project tagged version 3.4.9.2 on 2005-10-11.
Since then they've spent two days doing cleanups and merging hotfixes
to that tag.

Too see what has happened to the 3.4.9.2 tag (only), do:
# svn log --stop-on-copy svn://anonsvn.kde.org/home/kde/tags/KDE/3.4.92/

You'll see that the tagging operation is r469461 and the hotfixes etc.
are ensuing numbers.

To check out the HEAD of the tag, including hotfixes:
# svn co svn://anonsvn.kde.org/home/kde/tags/KDE/3.4.92/ my_kde

To check out the original tag, no fixes:
# svn co -r469461 svn://anonsvn.kde.org/home/kde/tags/KDE/3.4.92/ my_kde

Easy :-).

This is just an example to show it can be done all while keeping the
source in the repository (where it belongs, IMHO) and making checkout
of both the original tag and the fixed version as easy as sunday
morning.

I know that SVN doesn't fly around here, so please don't hit me on the
head for that one - it's just that I have no experience with CVS..


gslink wrote:
> This is the only solution that has worked in the past.  It requires a
> test suite be built that can easily test Windows, not Wine, and it
> requires that the tests be run before the next version comes out.

Not realistic.  Building a test suite will take years.
I disagree that it's the only solution that will work.



Saulius Krasuckas wrote:
> Then that person won't use patching.  Actually, using Molle tactics mean
> to patch tarball only when the problems arise.  Such script even may use
> some database to collect patches for an old distros and a variety of a
> kernels.

Right.  Except I'm still hoping that an entire new system isn't needed
for this.  Although it would obviously be more flexible, it would
probably be overkill?

> If such patcher-system/script isn't acceptable in the Wine tree, then it
> can go as a separate project, named for example Wine2Old, OldWine, etc.

Feasible albeit not as pretty.  And vastly more time consuming.



Lionel Ulmer wrote:
> > The application, according to one note, worked in 2003.
>
> The question now is: why was this application not tested more often to get
> the exact date of regression ?

I'm guessing that most people give up before I do =).

Or maybe they just haven't the time.  Which is where I'd like to help
regarding this thread..



Jonathan Ernst wrote:
> I just got a new idea:
>
> Instead of appliying new patches to old versions which is unmaintainable
> imho,

Bah :-).  Perhaps with CVS.  I don't think so but I don't really know.

> we could have VMWare installs of Wine and people could download
> the VMWare image of any Wine release and play it for free using the
> VMWare player !

Excellent idea!!

> I guess the download size will be quite big, but a small distro with
> only X, Wine, Gnome and KDE and required dependencies for example could
> fit on half a CD I guess.

Do you know of any miniature distro(s) that carry these and
(preferably) existed all along with Wine?  If you have suggestions,
I'd like to give it a shot right away.

We'd need a web page somewhere telling people what image works with
which Wine alphas/betas.