kernel fix for SB16 fullduplex system lockup

[Andreas Mohr]

[wine-devel copy of my LKML mail]

Hi all,

when running a program on Wine, suddenly my 2.4.18 froze completely.
This has been caused by the program submitting an odd number of output
bytes to the soundcard in 16bit output mode, thus causing the sound
driver to loop endlessly on the last remaining byte.
When digging deeper, I found that this odd number of output bytes had
been caused by the SoundBlaster 16 driver always returning one half of
the output bytes that it could take (in the SNDCTL_DSP_GETOSPACE ioctl
result), thus always cutting the remaining free buffer in half, thus
hitting an odd number after some time (1988 bytes -> 994 -> 497).
The SB16 driver is doing an internal 16bit -> 8bit conversion in case
it's running in fullduplex mode, and it simply returned the remaining
free bytes in the DMA buffer, which should have been twice that number
due to the 16bit -> 8bit conversion in fullduplex mode.
Also, it's doing 8bit -> 16bit conversion in the 8bit output case,
which SNDCTL_DSP_GETOSPACE ioctl didn't account for either.

In short: When using SB16 in fullduplex mode, the SNDCTL_DSP_GETOSPACE
ioctl didn't account for the 16/8bit format conversions that the SB16
does in this case, thus leading to an odd-numbered input count freezing
the box due to a second "odd remainder" endless loop bug.

Thus this patch does:
- add two flags DMA_CONV_16_8, DMA_CONV_8_16 to struct dma_buffparms
  to check for format conversion in dma_ioctl()/SNDCTL_DSP_GETOSPACE
  and thus fix SNDCTL_DSP_GETOSPACE result to account for format
  conversion
- prevent endless looping (lockup) on single remaining byte by
  discarding odd remainders of odd output counts in case of 16bit format

This WORKS (well, for me at least :-).

My previous LKML mails about this problem were:
 950     May 25 Andreas Mohr    (  80) SB16: 2.4.18 lockup on odd-numbered 16bit
 951     May 26 Andreas Mohr    (  59) SNDCTL_DSP_GETOSPACE bug for SB16 (was: S
I'm not in the least experienced with sound card programming, and I
didn't get much LKML help on this issue, so please review and let me know
the result, if possible.

CC'd to Andrea, the original author of SB16 fullduplex support.

Thanks !

-- 
Andreas Mohr                        Stauferstr. 6, D-71272 Renningen, Germany