do ERR messages imply bugs?

[Ben Klein]

2009/4/13 chris ahrendt <celticht32 at aol.com>:
> Is there a guide documenting what each test is supposed to do etc?

Source code.

Before you say that's an unacceptable answer, the sheer number of test
cases (especially considering those that keep getting added) would
make it impractical to the point of impossibility to maintain proper
documentation on each test. And when the tests are only intended for
developers anyway, the source code is perfectly suitable as
documentation. The tests are relatively simple, and it's clear which
case is being test by the use of ok().

> I run the test set to see where the current version of wine is on any of my
> particular hardware.

Then you should learn to interpret the ERRs correctly.

> I am not allowed to develop wine due to NDA's , and so forth I am under. And
> have discussed
> with the Alexandre and Dan concerning these NDA's and what I can do to help.
>
>>>
>>> That does not make any sense...  There has to be a consistent way for
>>> developers and users
>>> to report or work on issues..
>>>
>>
>> It's called bugzilla.
>>
>
> I believe you may be missing my point Ben. By consistent I mean an ERR means
> this and only means this. Warning, info etc...

As far as I'm concerned, that already exists, even if it's not written
explicitly and finitely in the dev guide. ERRs caused by the test
suite can be ignored as long as the tests run correctly. By the same
logic, ERRs caused by real applications can be ignored as long as the
apps still run correctly. In the former case, there is no motivation
for the devs to fix them. In the latter, there is minimal motivation,
and the devs have bigger problems to worry about than things that
aren't actually *breaking* anything at the moment.

> From what I  have been following here and also seeing in the code an error
> doesn't always follow the
> coding standard set forth in the developers guide.

I don't think you understand the nature of the test suite. There are
some tests that are 100% valid logic and are expected to create no
ERRs or WARNs. These are called positive cases. There are some tests
that are not 100% valid logic, and do things wrong deliberately, and
are expected to create both ERRs and WARNs. That's an "and" there, not
an "or". Both ERRs and WARNs. This cannot change.

>>> and like Vincent says the use has gotten a little skewed.
>>
>> Or been misinterpreted ... though a review of the developers' guide
>> description of ERR wouldn't hurt.
>>
>
> I would agree it may be... and maybe a review is in order..

So who wants to volunteer? :)


2009/4/13 chris ahrendt <celticht32 at aol.com>:
> An Excellent point was mad in one of the bugs I reported :
>
> Comment #25 from Austin English <austinenglish at gmail.com>
>
> The problem is that this isn't a 'normal' application doing weird things.
> It's our testsuite, which does some _really_ strange stuff, e.g., lots of
> corner case testing.
> Our implementation code, however, knows this is weird, and prints an error.
> The problem is that there's no way to tell, e.g, dlls/mshtml/htmldoc.c that
> what's
> being tested is dlls/mshtml/tests/doc.c, rather than foobar.exe. It's a bit
> misleading, which makes it a bit harder for users.

This is EXACTLY what I've been saying. You ignore me, and listen to
Austin. Absolutely fantastic.

> So, there has to be a happy middle in this...
> why not have in the test suite some printf or the like that says to the
> effect that err output is to be expected or the like.. Most users if they
> see ERROR on the screen
> be it in a test case or an application will think its a bug and not just
> ignore it.

Most users DON'T RUN THE TEST SUITE. It's not useful for users, it's
ONLY useful for developers, and developers know how to interpret these
ERRs correctly. You don't.