Testing edge cases and undocumented behavior

Joerg-Cyril.Hoehle at t-systems.com Joerg-Cyril.Hoehle at t-systems.com
Tue Aug 30 06:05:10 CDT 2011


Hi,

Michael Stefaniuc wrote:
>What you do want to test is:
>- the de jure API (as documented), and
>- the de facto API (as used by applications

You plead for operational profile testing.  I say that the API "as
used by applications" is unknown to me because the 100000 apps out
there don't send me email about their API usage.  So I'm left with
extremely few apps to test, and my judgement.

My judgement tells me to complete your list:
- the expected corner cases of the API or the programming language.
  + e.g. buffer sizes 0 or even < 0 with C
  + reference counts with COM
  + class inheritance with C++


For instance, Scott Ritchie's example is not unusual:
+ *  If lpszStr is Null, returns how long a formatted string would be.
This is a very common pattern.  Probably MSDN forgot to mention it.
Expect apps to use that when they find out it works.

What I don't understand about Scott's patches is why there's half a
dozen of them.  One patch for the code, one for the tests about
StrFromTimeInterval, possibly even join them.


>An extreme case would be to try to test how much time/CPU cycles a
>function call takes and trying to replicate that in Wine.
>In general that doesn't matter but is is observable and testable.

I have audio in mind so I don't care about "in general".
With audio, it matters, as seen in the MS compatibility kit:
http://technet.microsoft.com/en-us/library/cc766308%28WS.10%29.aspx
See IgnoreMCISTOP.  Complete the list further:
- the timing of the audio API
- the timing of events (e.g. mmdevapi EVENTCALLBACK) and
- the timing and invoking thread of callbacks (bugs #3930, 27795)

Actually, I've been thinking about performing timing tests of the
mmdevapi Start and Stop methods.  One noteworthy point is that I *may*
perform such tests to validate my beliefs about how mmdevapi works
(e.g. is it so fast as if it solely flips a bit polled by the timer job?),
but that does not necessarily imply that I'll add tests to the source.
I've performed many more tests than you'll eventually find in tests/.

Regards,
 Jörg Höhle


More information about the wine-devel mailing list