[PATCH] winex11.drv: Fix drawing of layered windows with a client window.

[Giovanni Mascellani]

Hi,

Il 27/04/21 11:33, Rémi Bernon ha scritto:
> As far as I can tell from a quick test with OpenGL rendering, there is 
> two different behavior on Windows:
>
> I'm only considering the layered attribute and calls on top-level 
> windows, as I couldn't make it work on child windows, to the contrary 
> to what MSDN says is supported on Windows > 8.
>
> 1) When UpdateLayeredWindow is called, the displayed image is the one 
> that has been provided in the call, regardless of any later drawing 
> call indeed, be it GDI or OpenGL. If the window has child windows, 
> they aren't displayed either, regardless of the API they use too.
>
> 2) When SetLayeredWindowAttributes has been called, the window is 
> displayed, with the transparency computed according to the attributes. 
> If the window has child windows, they are displayed normally, but with 
> the same transparency transformation. Drawing with GDI or OpenGL on 
> the top-level or the child windows works as for a normal window, 
> except for the transparency.

You're right, I was forgetting about SLWA. With this in mind, I can 
reproduce the behaviors you describe, except that I didn't test with 
child windows. On the other hand, I also checked that D3D9 are on the 
same level as GDI and OpenGL (i.e., they are ignored after ULW). So, I 
let me summarize again the Windows behavior. Windows can be of three 
different mutually exclusive types: not layered, ULW-layered and 
SLWA-layered. Creating a window with WS_EX_LAYERED makes it ULW-layered, 
otherwise it is not layered (on Windows a ULW-layered window appears 
immediately, even if it's empty, and it receives events; on Wine it 
doesn't). Then the painting behavior is:

* If the window is not layered, ULW and SLWA return error and do 
nothing. The windows receives WM_PAINT as usual. GDI, OpenGL, D3D9 work 
as usual and are on the same level, meaning that each of them can 
overwrite what the others painted. I suppose Vulkan and other D3D 
versions are analogous. Setting WS_EX_LAYERED on the window makes it 
ULW-layered. Window decorations are rendered as usual.

* If the window is ULW-layered, it doesn't receive WM_PAINT and all the 
other APIs are ignored (at least, they don't draw; I don't know if they 
produce side-effects, if they have any). The only way to put something 
in the window is calling ULW. Clearing WS_EX_LAYERED on the window makes 
it non layered, and calling SLWA makes it SLWA-layered (and generates a 
WM_PAINT event). Window decorations are not rendered.

* If the window is SLWA-layered, it receives WM_PAINT as usual. All the 
other APIs (i.e., not ULW) work as in the non layered case, except that 
the post-processing set with SLWA (color keying and global alpha) is 
applied. ULW returns error (there is no way to switch directly to 
ULW-layered: you have to clear and set WS_EX_LAYERED) and clearing 
WS_EX_LAYERED of course makes the window not layered. Calling again SLWA 
(possibly with different settings) does not cause a new WM_PAINT event, 
but correctly updates the window, which probably means that Windows 
keeps an offscreen copy of the original buffer. Window decorations are 
rendered as usual (and are affected by the SLWA alpha).

We already have a "layered" bit in x11drv_win_data, but I don't think we 
have a bit to distinguish the two layered variants. I think we should 
have one, which we would use to deny painting by other APIs when the 
window is ULW-layered (and, conversely, to allow ULW to paint on the 
client window, i.e., what my patch was trying to do; or maybe in this 
case we should just unmap or destroy the client window). Does this look 
sensible? (I'm not meaning that this bit is the only thing we need to be 
like Windows, but that at least it is a step forward)

> Here I would expect the surface flush to only work once until a later 
> GL swap buffer / VK present override it (although I'm not completely 
> sure when the Expose events are generated).
>
> It may also work differently depending on if the client windows are 
> off-screen or not (depending on whether it's a client window of a 
> child or if the top-level has child windows).
>
> Although this patch possibly fixes some bad cases I think the layered 
> window implementation is actually incorrect in a more general way.
>
> To make things simpler I think we should maybe make the off-screen 
> GL/VK rendering the default and do the present ourselves on the 
> top-level windows, and put the GL/VK on-screen /only/ when we're sure 
> we can (which is hopefully still going to be quite often). 

I am not sure I am following you. In which case would off-screen 
rendering help?

Thanks, Giovanni.