Google Summer of Code Test Suite project

[Corey McClymonds]

On 3/19/08, Lei Zhang <thestig at google.com> wrote:
> On Wed, Mar 19, 2008 at 3:01 PM, Corey McClymonds <galeru at gmail.com> wrote:
>  > I would like to work on the Regression Suite project, but I'm not
>  >  quite sure of the scope.  Would I be choosing a good 30 or 40
>  >  applications that run well on wine, using wpkg to install them, and
>  >  then test for failures?  Because that would seem as though I am only
>  >  doing installer testing, instead of actual application testing.  I
>  >  would also like to do some sort of automated regression testing for
>  >  actual features you can easily test, if that would be possible.
>  >
>  >  Corey
>
>
> Let's just say you're going to write an installer tester. How do you
>  define a failure? If you're  doing silent installers, do you just look
>  at the return value from the installer? If you installed app foo, and
>  the installer created file bar on Windows with a checksum of qux, but
>  not in Wine, does that count as a failure? How about if a registry key
>  is missing on Wine?
>

It depends if any functionality is lost.  You could argue that if
anything is different, then wine isn't implementing something
correctly, but I would be fine with it completing and working as
intended.  I would probably do checks on all 3 things you talked
about, and likely anything else that could be easily checked for by a
computer.