Accessibility in wine

[Reece Dunn]

On 12 October 2010 14:21, Seth Shelnutt <shelnutt2 at gmail.com> wrote:
> I can't seem to find much info on this. What is wine's support for MSAA or
> the newer UIAutomation? I see that there is the oleacc dll which is the main
> dll for MSAA. According to the API site, is seems there are still 10 stubs.
> Looking at the source code there hasn't been any updates in over a year. We
> don't have the UIAutomation dll yet. Does anyone know the status of
> MSAA/UIAutomation support in wine?
>
> I'm trying to get NVDA working and still running into some issues with it
> requiring UIAutomation.dll, when according to them, that dll is optional.
> After I get that worked out I'm hoping to see what needs to be done to get
> MSAA supported under wine.

The current wine implementation supports GetRoleText and the like on
oleacc.dll, but does not support IAccessible (MSAA) or UIAutomation
functionality on the Win32 objects (windows, list boxes, text edit
controls, buttons, etc.).

UIAutomation consists of 4 parts:
   1/  a COM API to the accessibility functionality
   2/  UIAutomation support for all Win32 controls
   3/  an MSAA <=> UIAutomation bridge (for applications that use MSAA)
   4/  a .NET binding to the UIAutomation objects

To get MSAA working in wine, you will need to (minimally):
   1/  complete the IAccessible core support -- LresultToObject and friends
   2/  add a WM_GETOBJECT handler to all the Win32 controls and expose
IAccessible COM objects (with tests!)
   3/  implement/flesh out the WinEvents API to get the IAccessible
objects for the events

Given the move to UIAutomation, it would be easier in terms of
implementation to:
   1/  expose UIAutomation objects via the WM_GETOBJECT message
   2/  use the MSAA <=> UIAutomation bridge to support MSAA

A question here is whether the wine MSAA/WinEvents/UIAutomation
implementation should bind to the Gnome/KDE accessibility APIs
(GAIL/???). The benefit here is that the Windows applications will
integrate into the native Linux a11y technologies. The problems here
are (a) getting Alexandre's buy in to use the GAIL APIs if present and
(b) whether these are going to change in the move over to Gnome3.

In terms of applications to test, Firefox supports at least MSAA
bindings as do Qt applications. Others will to, to varying degrees
(some relying on the native Win32 support), but FF and Qt implement
their own IAccessible bindings.

In terms of testing this via applications (in addition to the
regression tests that will need to be provided), there are various
assistive technologies that make use of MSAA/UIAutomation -- JAWS,
Dragon Naturally Speaking, etc. I am not familiar with NVDA.

- Reece