OpenGL in child windows

[Chris Robinson]

Here's something I've been working on for about the past week. It's my attempt 
to get OpenGL to render into an X11 child window that overlays the Win32 
window. These patches are more proof-of-concept than anything I'd attempt to 
get into Wine.. just to show that it seems to work. If the method is good 
enough, these would need to be rewritten.

The biggest problem facing these patches that I ran into are threading issues. 
Since each window is basically tied to the thread that it was created in but 
can be used out of it, and each thread uses its own X11 display connection, 
there's quite a few hoops to jump through.

The problem is that the OpenGL X11 window can be created at basically any time 
after the window was created, in any thread (it's currently done in the call 
to SetPixelFormat, the earliest it'll know which visual/pixel format is 
wanted, and can create a colormap to be able to use it in a child window, 
regardless of the visual of its parent). If that call is not done in the 
thread the win32 window was created in, it would have trouble getting some of 
the data structures it needs. Add on top of that, that the X11 window can be 
parented to yet another win32 window (the one that has the "whole" X11 
window), which can be in yet another thread, it leaves a lot of problem 
trying to keep syncronization.

Most of the placement syncronization is done by adding a couple fields to 
x11drv_win_data to hold info for the X11 child window's parent, that can be 
used to properly offset the X11 child in relation to the X11 parent, 
according to the Win32 window's placement. It also needed a couple 
driver-specific WM_ messages to be able to get the data it needs in case the 
creation is done in a seperate thread.

Because the X11 window uses a thread-specific display connection, and the 
child window needs to share the display connection, I needed to add a 
Display* to the X11DRV_PDEVICE struct, which is used wherever a GLX call 
needs an explicit display connection. This works well most of the time, but 
has a problem when a GL context (tied to an HDC in Windows) is in use when 
the thread that created the Win32 window that the HDC came from is killed. 
When the thread is killed, it closes the connection which leaves GLX using an 
invalid Display.

I have to ask, is there a reason each thread uses its own display connection 
(which are all different from gdi_display), and leaves the window data to 
only be available from the thread the window was created in? Besides the 
aforementioned problem with the GL context, I'd imagine it would be 
problematic even in current Wine.. for example, if something creates a window 
in one thread, passes it to another, then kills the thread it was originally 
created in while still using the window.

I currently made it so each thread shares the same Display connection (which 
is different from gdi_display). This seems to cause no noticeable problems, 
and it fixes the GL context display issue.

Beyond that, the only problem I can recall off-hand is that the OpenGL window 
will draw overtop of all Win32 windows that are sharing the same X11 parent 
window. However, considering that the current method with the scissors hack 
causes the whole X11 window to clear on a buffer flip, and also draws on top 
of anything in its window space anyway, this is still a nice step forward. 
And there are also a couple syslevel potential-deadlock warnings generated 
from the calls to create the window, however I have yet to see a problem 
arise from all the programs I've tested (the warnings can't stay obviously, 
but they can be figured out later).


What these patches end up accomplishing is that it allows for the possibility 
to use alternate visuals, so the X11 driver would not need to force a 
specific double-buffered visual on initialization, and it allows programs to 
use program-specified color, stencil, depth, aux, multisampling, and 
single/double-buffer configurations without relying on the main window to 
have it too (this isn't done in these patches, but it does allow for it to 
work). Also, the scissors hack is no longer needed. All calls to the 
Wine-specific extension for it can be removed (they aren't removed here, 
instead just piped directly to the appropriate gl* function with no 
alteration, since that would require changes to opengl32). OpenGL in Win32 
child windows is improved by being restrained to the Win32 windows' extents, 
as seen in the screen-grab here in VLC using the OpenGL output renderer:

http://kcat.strangesoft.net/deskshot.jpg

It uses proper color ordering (unlike the GDI renderer), is properly centered, 
can be scaled with no performance impact, and doesn't clear the whole window. 
Performance is good, and it is actually quite useable.

Terragen (which has a real-time 3D window), Terragen 2 (which has two or three 
OpenGL child windows), Milkshape 3D, and GoogleEarth all work. Terragen 2 has 
some issues with its right-click menu that comes up from the OpenGL child 
window (the mouse doesn't interact with it), however I'm not sure if this is 
an issue with my patches since Milkshape3D also has a right-click menu from 
an OpenGL child and it works fine. In any case, there's no other 
input-related issues to speak of. Clicking and dragging the globe in  
GoogleEarth works, as does the keyboard controls in Terragen, and the other 
Win32 windows are properly receptive to all input as well. It also behaves 
properly with composite effects (such as the transparency, stretching, and 
warping effects from Beryl).


All testing and comments on the patches are welcome. If the code is doing 
something it shouldn't be, I'm more than willing to figure out alternative 
methods if needed (hopefully with help). With luck, they can be gotten to a 
point soon where they can be rewritten properly to get into Wine.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ogl-child-windows.tar.bz2
Type: application/x-tbz
Size: 19495 bytes
Desc: not available
Url : http://www.winehq.org/pipermail/wine-devel/attachments/20070109/9f4ffe6d/ogl-child-windows.tar-0001.bin