do ERR messages imply bugs?
jjmckenzie51 at earthlink.net
Sun Apr 12 21:10:54 CDT 2009
Ben Klein wrote:
> 2009/4/13 chris ahrendt <celticht32 at aol.com>:
>> Vincent Povirk wrote:
>>> On Sun, Apr 12, 2009 at 5:24 PM, Ben Klein <shacklein at gmail.com> wrote:
>>>> 2009/4/13 Vincent Povirk <madewokherd+8cd9 at gmail.com>:
>>>>> But the description doesn't say "invalid conditions". It says "serious
>>>>> errors in Wine". That's something that should never happen in tests,
>>>>> as it would imply that the state of the libraries we're testing (and
>>>>> thus the test) is invalid.
>>>> How about "serious errors in Wine, or in the test cases, sometimes
>>>> deliberate errors in the tests"? As Vitaliy points out, some tests
>>>> deliberately do invalid things to make sure they fail. We can't ONLY
>>>> test on things that succeed!
>>> I'm not against changing the description. If it's OK to have err
>>> messages for cases that we know fail (but don't crash or prevent the
>>> library from continuing to function), then the Developer's Guide is
>>> wrong and should be changed. I also don't care how long it takes to
>>> make the change, as long as there's a consensus that it's the guide
>>> that's wrong and not the reality of what's in Wine.
>> This is EXACTLY the point I am trying to get to.. if they are normal and not
>> ERRORS but warnings then they should be thus and noted in the developers
>> guide. Right now Wine isn't even following its own guidelines in this case.
> No. Not warnings. Errors. They are errors. There is no way to
> distinguish an error caused by a real application from an error caused
> by a Wine test.
If the situation is an error and it is expected, the test should handle
ok (some_test_here_that_should_fail, "The test that should fail, did/n")
I'm guessing that most of the tests that should fail, do. I don't know
if there is a failure like there is an ok.
> If you don't like it, run all your tests with WINEDEBUG=-all.
And that will prove nothing. Tests should be run with debugging on.
You are really being sarcastic, right?
As to the discussion, I will add my .02 Eurocent here:
Fixme: Code that is not implemented that should be.
Warning: Code that encountered an unexpected or new condition that is
not normal. Data should not be corrupted by this event.
Error: Code encountered a condition that will result in corruption of data.
It appears that we have 'error creep' and that is not good from a user
point of view and it is really irritating from a support point of view.
During testing an error could be either a Warning or an Error. Tests
should not be run against non-existent Wine code, but should against
existing Windows code. The situation with testing is that unexpected or
improper behavior of code should be an error. There is no such thing as
a warning during a test run. Either the test passes, which means that
Wine is acting the same as a certain version of Windows, or it is not.
Now, the problem is that we are sending cryptic messages to end users,
most of which they can and should ignore. Err messages piling up on
their terminal windows should be a cause for concern. If we know that a
condition does not cause data corruption, then we should not be marking
it as an error, but maybe a warning or if the code is non-existent or
improper, a fixme.
Can we start to clean up some of the most obvious "it is not an error
but the message says it is" so that we can calm down users who encounter
More information about the wine-devel