Wayland driver development - December 2021 update and next steps

[Alexandros Frantzis]

On Mon, Dec 13, 2021 at 05:34:30PM +0100, Jacek Caban wrote:
> Hi Alexandros,

Hi Jacek!

> It's good to hear that the driver is progressing well.
> 
> On 12/10/21 6:56 PM, Alexandros Frantzis wrote:
> > The vast majority (but not all) of the commits in the series affect only
> > the driver itself. However, there is still a lot of surface area, mostly
> > in terms of interfaces used to interact with Wine core, that could lead
> > to broken code when upstream is updated. This has been more pronounced
> > the last few weeks with the various changes in the driver interface.
> 
> 
> I was responsible for those changes. This is just a preparation for a huge
> change that drivers will need to implement as a part of PE separation.

It was a relief to remove from driver all the registry related manipulations for
the display setup. Thanks :)

> While following interface changes shouldn't be too hard, doing the final
> separation will be more tricky. Let me roughly describe what will need to be
> done.
> 
> 
> A majority of the driver will ultimately live in Unix library (not PE-in-ELF
> like now) and will be executed behind syscall barriers. The main practical
> implication is that while it will have full access to Unix libraries, it
> will not be able to (easily) call PE libraries. This requires changes all
> over the place (see dlls/win32u for an example and "#pragma makedep unix" in
> general):
> 
> - Instead of kernel32/kernelbase system functions, the driver can use
> ntdll.so directly.
> 
> - Instead of kernel32 runtime functions, like memory allocators for memory
> not exposed outside Unix space, it will need to libc functions.
> 
> - For synchronization primitives like CRITICAL_SECTION, it can use ptherad.
> 
> - For gdi32 and user32, it can use win32u.so. win32u.so, however, is still a
> work in progress. While it should be already be able to cover gdi32 usage,
> some user32 are still missing in win32u.
> 
> - The above are the more straightforward parts. Doing that will live some
> other calls that will be more tricky. Some parts of the driver may need to
> be in PE file, for example.
> 
> 
> We don't do that for in-tree drivers yet, mostly because win32u needs to be
> more complete before we can have a complete solution. But since your branch
> is a new and actively developed code, it may make your work easier if you
> take that into account sooner.

Thanks for all the information. It helps put the changes into
perspective.

Moving in this direction earlier rather than later for the Wayland
driver makes sense to me. Are all these changes safe to do before the
full transition?  Also is it possible at this point to mix and match
calls, e.g., to win32u NtUser* and also to user32.dll (for any functions
not yet in win32u), or can such a partial transition lead to issues?

> > * Display mode changes (emulated with compositor scaling, rather than
> >    real display hardware mode changes).
> 
> 
> This essentially sounds like "full screen" hack that Proton uses. Given that
> there apparently is a demand for such feature, I wonder if it would make
> sense to share such emulation between drivers. I imagine that this could be
> a null driver's behaviour that other drivers could choose to utilize.
> 

I haven't seen the details of how Proton deals with full screen scaling,
but, at least in the Wayland driver, it's all done using Wayland specific
functionality, so it's not clear to me at this point how much sharing is
possible.

In particular, in the Wayland driver the work of scaling is relegated to
the compositor. The application renders normally, assuming whatever mode
is active from the Win32 perspective, and the driver tells the
compositor to scale the surface accordingly to provide the illusion of a
mode change.

> > There are some known cases where the driver doesn't work properly, that
> > I think deserve an explicit mention since they may trip people up.
> > 
> > The first is when applications try to render to window created by
> > another process. Although this not a typical scenario, it's how
> > Chrome/Chromium and thus applications using CEF, including some game
> > store apps, work. For now, the workaround is to instruct the application
> > to use "in-process" GPU handling (with the "--in-process-gpu"
> > command-line) or turn-off GPU rendering completely (typically with
> > "--disable-gpu --disable-software-rasterizer"). For example, one can run
> > GOG Galaxy with: "GalaxyClient.exe --in-process-gpu [--use-angle=d3d11]"
> 
> 
> Yeah, we had those problem with winemac.drv for years. At some point I wrote
> a hack that used window surfaces backed by shared memory that could be used
> in child process and parent windows' process later copy that into the actual
> window surface. It works well enough for a number of scenarios, but it's not
> really enough for Vulkan/OpenGL.

I think such an approach would be a somewhat viable option for Wayland,
given that its presentation model is based on submissions of client
allocated buffers, either SHM buffers for software rendering, or dmabufs
(DMA buffers) for Vulkan/OpenGL.

The driver could manually render to a (offscreen) buffer and then submit
that itself, or pass it to another process (e.g., the one owning the
window) for presentation. The driver would essentially need to
reimplement much of the Wayland WSI swapchain/EGL swapbuffers
functionality, including buffer allocation, handling of fences, frame
callbacks etc. Then there is all the infrastructure around IPC of these
buffers.
 
> I would hope that a cross-driver solution could be found. winex11.drv also
> has some problems with its support for cross process rendering. At the time,
> we talked about an OpenGL-based compositor, although Vulkan would be
> probably the first choice those days. It seems to me that for cases when
> driver can provide support for Vulkan top level windows' surfaces, the rest
> could be implemented in win32u. Child windows Vulkan/OpenGL rendering could
> be rendered off-screen and then composed with the rest top level window. OOP
> rendering could probably be implemented on top of that by setting up a
> top-in-the-process window surface and additional IPC to parent window's
> process. I think that Windows has something like that and exposes some of it
> with DComposition APIs, so aligning such solution with that would be
> interesting. Anyway, I didn't look deep enough to be sure about any of that,
> but that's roughly how I imagine that this could be solved with a lot of
> work, but not much involvement on driver side.
> 
> 
> Thanks,
> Jacek

Thanks,
Alexandros