To error out or to skip tests?

[Reece Dunn]

2009/11/6  <Joerg-Cyril.Hoehle at t-systems.com>:
> Ge,
>
>> to be honest I'm a bit put off by your aggressive tone.
> Please accept my apologies. It was not meant to be agressive.
> I was certainly disappointed by having submitted tests that
> I knew worked on 3 real machines: w95, w2k, xp, then turn to
> test.winehq and see everything red.
>
> BTW, dsound is also affected: both dsound and winmm have had little
> testing coverage despite the appealing green color.
>
>>I'm simply not interested in sound
> Please weight providing as much test data as possible to the Wine
> project against your lack of interest in sound issues.
> BTW, the default sound tests make no noise.
>
>>Anyway, the end result is that there will always be ESX VMs
>>that can't run sound tests.
> Note taken.
>
> The question remains:
> a) turn sound tests red on machines without sound (with 1 error only);
> b) turn them blue (clearly marking skipped tests)
> c) turn them green -- status quo (for dsound.ok and wave.ok).

 309     err = mciSendString("open waveaudio!tempfile.wav alias
mysound", buf, sizeof(buf), NULL);
 310     ok(!err,"mci open waveaudio!tempfile.wav returned error: %d\n", err);
 311     if(err) {
 312         skip("tempfile.wav was not found (not saved), skipping\n");
 313         return;
 314     }

Here, you are doing both (generating an error and skipping)!

Also, how do you know that this is due to tempfile.wav being missing
(the skip is invoked for any error case)? Especially given the line:

   mci.c:261: Tests skipped: Please install audio driver. Tests will fail

in the test results.

Shouldn't this be something like:

      err = mciSendString("open waveaudio!tempfile.wav alias mysound",
buf, sizeof(buf), NULL);
      if(err == MCIERR_INVALID_FILE) {
          skip("tempfile.wav was not found (not saved?), skipping\n");
          return;
      }
      ok(!err,"mci open waveaudio!tempfile.wav returned error: %d\n",
dbg_mcierr(err));

I think that it is safe to say that if there is no audio driver
present, all subsequent mci tests will fail and should also be skipped
(ideally with just the single warning, e.g. by returning FALSE from
that function and wrapping the other tests in an if).

  27 static const char* dbg_mcierr(MCIERROR err)
  ...
 108      default: return "unknown error";

This is not really helpful, it should contain the error number.

    mci.c:239: Test failed: mci set format tag pcm returned error: 282

This (and the other error numbers) does not mean much to me. There is
a dbg_mcierr function that was created; shouldn't that be used here
(as the error is returned from mciSendString)?

- Reece