Babystep: Testing framework

[Andriy Palamarchuk]

--- Francois Gouget <fgouget at free.fr> wrote:

> > 2) using TODO tests. The problem becomes even
> worse if
> > more than one Win32 implementation project (e.g.
> ODIN)
> > starts to use the test, because list of TODOs is
> > project-specific
> 
>    .ref.diff files. If the diff between the .out and
> the relevant .ref
> file matches that diff file, then all is good.

like:

.win95.wine
.nt4.wine
.win95.odin
.nt4.odin

?

> > Now, imagin combinations of these :-)
> 
>    WONTFIX will not be very practical. But are they
> really needed? I
> would say that TODOs should be enough.

WONTFIX is not TODO, it is DONE :-) I don't want to
use TODO for this, because it will show up as
unfinished work.

> > Explicit check, on other hand, nicely comments all
> > these conditions in one place - in code.
> 
>    Except that in some cases they will make the
> checks pretty complex.

Arguable. IMO to retrieve the same information from
diffs - you need to visit:
test1.c
test1.win95 
test1.win98 
test1.nt4
test1.w2k
test1.wine
test1.win95.wine   - TODOs
test1.win98.wine   - TODOs
test1.nt4.wine     - TODOs
test1.w2k.wine     - TODOs

In many of these files you need to filter hundreds of
lines of code to correspond the check and its output. 

This holds true even for checks which work in the same
way on different Windows platforms, have no TODO and
WONTFIX attributes, e.g. for 99.9% checks.

How much lines the explicit check with all these
attributes will take? 2 additional lines in the worst
case, nothing - in 99.9% of cases.

Andriy Palamarchuk

__________________________________________________
Do You Yahoo!?
Send FREE video emails in Yahoo! Mail!
http://promo.yahoo.com/videomail/