[RFC] Listening to keyboard in background, RawInput and DInput

[Rémi Bernon]

Hi everyone,

I'm trying to solve a problem that looked simple at first but ended up 
being much more complicated than I expected. I would like to expose the 
issue and the solutions I'm thinking about here so that you can give me 
your opinion on it.


The issue:

On Linux/X11, when an application that uses DInput for keyboard input 
loses focus - with Alt-Tab - then gets focus back, it sees the Alt key 
as still being pressed until you press and release it again.

The reason is that X11 sends a KeyPress event to the application for the 
Alt while it still has foreground, but as soon as you press Tab the 
window manager takes a keyboard grab and intercepts all other key events 
until you release the Alt key. And the release event is not received 
either - so even if you don't actually change to another window, this 
event will not be seen.

There are two ways that I saw to workaround this missing release event. 
The first one is a KeymapNotify event that is received upon gaining 
focus, which could be used to send missing input. And the other is 
XInput2 RawKeyPress/RawKeyRelease events that would allow to listen to 
keyboard events while in background.


Regarding the input stack:

On the Windows side, AFAIU there're two different ways to get keyboard 
(or mouse) input. One is the WM_KEYPRESS/WM_KEYRELEASE messages, and 
their corresponding low-level hooks. And there's the WM_INPUT messages 
for raw input. I could see that both the low-level hooks and the raw 
input allow an application to get keyboard input while in background.

(I understand that internally the WM_KEYPRESS/WM_KEYRELEASE messages are 
probably implemented on top of the raw input, then provided to the 
low-level hooks and then discarded if the application does not have 
foreground, but in Wine it's all on the same level.)

Then there's DInput. In Wine it is implemented using low-level hooks, 
but I believe that it is using raw input on Windows - it is mutually 
exclusive with WM_INPUT events, and you can even see the raw input 
devices that DInput registered by calling GetRegisteredRawInputDevices 
after activating the DInput keyboard / mouse devices.


The problem:

When trying either of the two ways to get the missing release events, 
while keeping the current input stack unmodified, I faced some 
difficulties. The main reason AFAIU is because of duplicate key input 
being sometimes sent to wineserver, which can make the keystate 
inconsistent.

As each window sends input event separately, we rely on the host to only 
send unique KeyPress/KeyRelease events. This is correct in general as 
only the window with the input focus receives such events, but as soon 
as we add the KeymapNotify or the raw input events, it is not anymore.

It could be possible to add some de-duplication of the keyboard events 
in wineserver as it is done for some mouse motion, but that doesn't seem 
quite right.


The proposal:

I believe the right way to do would be to change DInput to be 
implemented on top of raw input, and listen to the host raw input events 
from the desktop window thread to receive them in background if possible.

It would make the desktop window the unique source of raw input 
messages, which would ensure their uniqueness, and they would be 
received whether wine windows are in foreground or not. In this case, 
normal input events would only translate to WM_KEY* messages and 
low-level hooks.

If not possible, then we keep the current way where normal input events 
generate both raw input messages and the normal messages and hooks.

This would also fix the values reported by DInput, that are currently 
modified by cursor speed and acceleration although they aren't on Windows.

This would also make wine windows capable of listening to keyboard (and 
mouse) input while in background, and make windows keylogger software 
work better. Not sure if this is an improvement or not.

However it doesn't change the fact that the low-level hooks aren't 
called when wine is in background. If we want that they it would require 
to translate the raw input to normal input ourselves instead of relying 
on the host events, but it would also be possible.


tl;dr: I would like to change DInput keyboard/mouse to be implemented on 
top of raw input and listen to raw input while wine is in background. A 
lot of work for a small issue, what do you think?
-- 
Rémi Bernon <rbernon at codeweavers.com>