OLEAUT32: fix detection of GIF palette index >= 128 for transparency

[Alex Villaci­s Lasso]

This patch was created in an attempt to trace down the incorrect 
displaying of bitmaps in the Toolbar control of MSCOMCTL.OCX (possibly 
unrelated to the toolbar control of comctl32.dll), which is frequently 
used in Visual Basic applications.

The symptoms: applications which use said Toolbar control to display 
buttons with icons which use transparency show incorrectly drawn icons 
in Wine. For example, MP3GainGUI shows icons with an ugly black 
background, instead of the intended gray (in my setup) application 
background, and big black blotches bleed through the intended bitmap.

I am currently trying to find the cause (and solution) to this issue. 
Since MP3GainGUI uses GIF images with transparency to provide the 
Toolbar button bitmaps, my first test was to create a sample VisualBasic 
application which assigns a GIF picture with transparency to the Picture 
property of the main form. The form is supposed to show a gray 
background (the default brush in VisualBasic for forms) with the chosen 
picture. This is how it shows up in native Windows98. Sure enough, the 
use of the builtin oleaut32 causes the picture to show up with a white 
background (that is, no transparency is being used).

I traced this problem to a one-line bug in the GIF loading code. The 
code is supposed to initialize a transparency flag to -1 when no 
transparency is used, or else the palette index of the transparency 
color. The sample image I was using indicates that the transparency 
index is 255. Due to an unintended sign extension, the transparency 
index (held in a Byte array of basic type "signed char") is 
sign-extended to the variable "transparency" of basic type "signed int". 
This causes the flag to be assigned the value -1, which happens to be 
the same value used as a no-transparency indicator. So transparency was 
not detected in my sample image. The attached patch fixes it by 
inserting an explicit cast to unsigned char.

The bad news is that this fix uncovers an even greater bug: builtin 
oleaut32 is unable to create a valid AND mask for a bitmap with 
transparency. The end result is that the AND mask (which is now created 
in the first place) holds a garbage bitmap, rather than a valid outline 
of the sample image. At first glance, the AND mask creation code seems 
correct, but it was 2:00 AM and I could not investigate the issue any 
further last night. So I submit this patch in the belief that it fixes 
an actual bug, and will engage tonight in further bug-hunting for the 
AND-mask bug in oleaut32, unless the Wine gurus out there can fix the 
bug faster for me, or else point me to an already existing patch.

Changelog:
* Fix failure to notice the use of a GIF palette index greater or equal 
to 128 for transparency


-------------- next part --------------
A non-text attachment was scrubbed...
Name: wine-oleaut32-olepicture-transparency_unsigned.patch
Type: text/x-patch
Size: 412 bytes
Desc: not available
Url : http://www.winehq.org/pipermail/wine-patches/attachments/20050329/ee1437dd/wine-oleaut32-olepicture-transparency_unsigned.bin