Wine GPU decoding

Tue Apr 1 09:45:00 CDT 2014

Hi Michael,

I had a quick look over your patches. As Henri pointed figuring out the
interaction between dxva2 / d3d is important. I would like to add to also
study the interface between the GPU driver (maybe that's through an awful
ExtEscape). The current code uses the VAAPI X11 APIs fro dxva2.dll. This
code should be moved in some way or another to the display driver, winex11.

Interaction between the display driver and dxva2 could be inspired by the
Windows way (if documented well), but it may be too complex and you need
your own mechanism. Depending on how dxva2 and d3d can interact, you may
want to use OpenGL and probably need your own WGL extension (e.g. something
similar to the VDPAU opengl interop extension).

Whichever video decoding library is better is hard to say. Over the years
vdpau felt like the better library to me (it felt like it took years to get
libva in some usable shape and I'm not sure how stable it is now.) You
mention that vdpau on Intel uses opengl. From my understanding it does use
the hw decoder and then uses some form of gl/glx buffer sharing, which is
not bad. They also defined a reasonable GL extension, would could be ported
to Wine. Some more testing may be needed. If worst came to worst, maybe
allow multiple APIs to be used, though less than ideal, but we have
multiple backends for other features (xrandr / xvidmode).

On a sidenote, OpenCL also allows sharing of buffers with the video
decoders using directx. It may give some clues on how it works in windows
and may give clues on how to do it for wiine. Long-term we may also want to
support this opencl extension, assuming it is widely used (I doubt that
right now).

I would urge to focus on the driver and d3d interaction.

Thanks,
Roderick

On Tue, Apr 1, 2014 at 6:50 AM, Michael Müller <michael at fds-team.de> wrote:

> Hi Henri,
>
>
> > Which APIs did you consider for this? What were the various issues?
> > What made you choose VAAPI in the end? Do you have tests for how the
> > dxva / d3d9 interface is supposed to work?
>
> I mostly considered VAAPI and VDPAU for this as they both offer support
> for multiple vendors.
>
> VDPAU has native support for nvidia, amd (only open source driver) and S3
> but not for intel. There is a OpenGL backend for VDPAU which can be used on
> Intel graphic cards but I expect the video decode engine of Intel to reach
> a better performance than an implementation which is completely based on
> OpenGL.
>
> VAAPI has native support for intel, crystal HD decoder, S3 and can also
> use VDPAU and the amds proprietary XvBA  interface. It simply supports
> everything VDPAU does plus Intel and the proprietary AMD driver. I think
> that a library which offers support for the most graphic cards is the best
> possible option for Wine as we do not want to implement these decoders
> multiple times.
>
> The only issues I encountered so far with VAAPI is that not all backend
> support all commands and you sometimes get an unimplemented error. This is
> not a big problem in the most cases as you can use other commands to
> achieve the same, the only real issue I found so far is that the vdpau
> wrapper does not allow setting or querying for the native image format of a
> codec. I can force a yuv 420 format, but i can not set or query whether it
> is stored as NV12 or YV12. When mapping the image it is possible to define
> a image format but I don't know whether a conversation is done or if is the
> raw decoded data. It may be necessary that we actually take a look at vdpau
> library and hard code some values if the vdpau backend is used to avoid
> conversations between formats.
>
> There are currently no tests so far. The reason for this is that mingw
> does not support the dxva header files and you can not use the hardware
> decoder in a VM. So I basically wrote some test code in MSVC and tested it
> on an old laptop which is running Windows 7 with a nvidia 9800 GTS.
>
> VLC wrote some header files for mingw which they use to cross compile VLC,
> but it does not offer everything we need ( http://www.videolan.org/
> developers/vlc/modules/codec/avcodec/dxva2.c ). Maybe whe can use the
> wine header file or ship a more complete version to allow cross compiling.
>
> If you want to know how dxva2 is used by applications I would suggest you
> to take a look at these 3 files used by VLC  to decode mpeg2 using dxva2:
>
> http://www.videolan.org/developers/vlc/modules/codec/avcodec/dxva2.c
> http://www.ffmpeg.org/doxygen/1.0/dxva2__mpeg2_8c-source.html
> http://www.ffmpeg.org/doxygen/1.0/dxva2_8c-source.html
>
> VLC initializes DXVA2, creates the surfaces and passes them to avcodec for
> decoding and storing the result images. Since you need a complete mpeg2
> bitstream decoder to gather the information for decoding it is not very
> easy to create a small example code.
>
> @Stefan Dösinger: I will try to do some tests this evening when I am at
> home.
>
> Michael
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.winehq.org/pipermail/wine-devel/attachments/20140401/f81fb363/attachment.html>