Babystep: Testing framework

[Francois Gouget]

On Thu, 10 Jan 2002, Andriy Palamarchuk wrote:

>
> --- Jeremy White <jwhite at codeweavers.com> wrote:
> > However, having the diff feature allows us to more
> > rapdily adapt existing programs to become tests.
> > Since it's done (and it's trivial code), I don't see
> > the harm in leaving in the feature.  We can hide
> > it/discourage in in the (as yet unwritten) doco if
> you like.
>
> Some disadvantages of the diff approach was discussed
> before. I just realized the problem which will make
> using this approach practically impossible. The
> problem - variations of output as result of:
> 1) using different Windows versions.

   .ref -> deneric reference
   .win95 -> win95 reference
   .win98 -> win98 reference
   etc.
   Okay, if the difference is between nt and win 95/98/me then it may
get a bit hairy.


> 2) using TODO tests. The problem becomes even worse if
> more than one Win32 implementation project (e.g. ODIN)
> starts to use the test, because list of TODOs is
> project-specific

   .ref.diff files. If the diff between the .out and the relevant .ref
file matches that diff file, then all is good.


[...]
> Now, imagin combinations of these :-)

   WONTFIX will not be very practical. But are they really needed? I
would say that TODOs should be enough.


> Explicit check, on other hand, nicely comments all
> these conditions in one place - in code.

   Except that in some cases they will make the checks pretty complex.


--
Francois Gouget         fgouget at free.fr        http://fgouget.free.fr/
In theory, theory and practice are the same, but in practice they're different.