[ANN] Conformance testing campaign

[Ferenc Wagner]

Jakob Eriksson <jakob at vmlinux.org> writes:

> First - can I leave out the "of 55" tests?  It is kind of
> difficult for me calculate how many tests there are in
> advance. Of course, I can do it - but it feels better if I
> know it is for a good cause.

You know, it was not that handy for me, either.  I do not
need that, but the numbering does not make too much sense
without the total -- all the fuss happened to provide sort
of a "progress bar" for Dimitry.  You will do it, won't you?
:)

> So, what I am proposing is something similar to this flow,
> had it been done as a BAT:
>
> echo kernel32.dll:atom (test begin)>>results.txt
> kernel32_crosstest.exe atom>>results.txt
> echo kernel32.dll:atom done>>results.txt

All right.  Instead of (test begin) simply put starts for
consistency.

> Also - those not compiled, how about just not include them
> in the report?  This I am more flexible about though.

Not really the same.  Tests can also be missing because the
tester had to remove them, and I would like to discriminate
in this respect.  See XP_robi for example.

>> If you could arrange that crashing tests die silently
>> (ie. without silly dialog boxes), that would be really
>> great!
>
> Hm...  I don't know.  Is that possible?

No idea.  But the best would be to recover the error code
and/or message somehow, and include in the report.  Like
waitpid() on Posix.  Is the tester has to click a couple of
times, then be it...  It is surely impossible from a batch
script, but since you have a C framework, let us try to make
the most out of it!

> BTW, is "ver-sion" a pun?  Good one.  :-)

Heh, I think it is.  Glad you like it.
                                                Feri.