[ANN] Conformance testing campaign

[Jakob Eriksson]

Ferenc Wagner wrote:

>Jakob Eriksson <jakob at vmlinux.org> writes:
>
>  
>
>>First - can I leave out the "of 55" tests?  It is kind of
>>difficult for me calculate how many tests there are in
>>advance. Of course, I can do it - but it feels better if I
>>know it is for a good cause.
>>    
>>
>
>You know, it was not that handy for me, either.  I do not
>need that, but the numbering does not make too much sense
>without the total -- all the fuss happened to provide sort
>of a "progress bar" for Dimitry.  You will do it, won't you?
>:)
>  
>

For the sake of Dimis continued mental health, OK then. :-)



>>So, what I am proposing is something similar to this flow,
>>had it been done as a BAT:
>>
>>echo kernel32.dll:atom (test begin)>>results.txt
>>kernel32_crosstest.exe atom>>results.txt
>>echo kernel32.dll:atom done>>results.txt
>>    
>>
>
>All right.  Instead of (test begin) simply put starts for
>consistency.
>  
>

ok.


>>Also - those not compiled, how about just not include them
>>in the report?  This I am more flexible about though.
>>    
>>
>
>Not really the same.  Tests can also be missing because the
>tester had to remove them, and I would like to discriminate
>in this respect.  See XP_robi for example.
>
>  
>

ok. I am not sure how to solve this right now, but I will figure 
something out.
Can't be that hard.


>>Hm...  I don't know.  Is that possible?
>>    
>>
>
>No idea.  But the best would be to recover the error code
>and/or message somehow, and include in the report.  Like
>waitpid() on Posix.  Is the tester has to click a couple of
>times, then be it...  It is surely impossible from a batch
>script, but since you have a C framework, let us try to make
>the most out of it!
>  
>

Sure, I have no idea how at this time though...


regards,
Jakob