[PATCH v7 4/4] winevulkan: Implement VK_KHR_external_memory_win32 for images.

[Derek Lesho]

On 5/25/21 11:52 AM, Zebediah Figura (she/her) wrote:

> How do you plan to implement the following features:
For what it's worth, all of the ideas I express in this email don't 
apply this patch, but rather planned follow-up patches.  As it relates 
to Vulkan, I don't think there's any approach we take this path makes 
more of a headache.  For example, if we do decide to instead create 
custom object types for resources, all we have to do is extend the 
create_gpu_resource function in that direction.
>
> * named NT paths
>
> * security descriptors
>
> * KMT handles 

I intend to take the approach of wrapping the FD object behind a device 
object, like you suggested in your previous mail.  Device objects can be 
named, and while support for access rights on device file objects 
doesn't seem to be implemented right now, it shouldn't be too hard to 
add this functionality.

https://source.winehq.org/git/wine.git/blob/HEAD:/dlls/ntoskrnl.exe/ntoskrnl.c#l530

The device objects will be created via IOCTLs to a master "Utility" 
device object, which will handle KMT lookup and user data storage.  For 
unnamed device objects I'm thinking about using 
FILE_AUTOGENERATED_DEVICE_NAME, retrieving the name of the device object 
IoCreateDevice returns, then proceeding as normal and opening/returning 
the handle.


>
> * CL_KHR_d3d11_sharing

OpenCL lacks any equivalent to GL's EXT_external_objects, which is 
"modern" approach to shared resources, meaning it is unable to directly 
share resources with Vulkan.  If needed in the future, I imagine it 
wouldn't be too hard to convince the OpenCL spec people to draft an 
equivalent extension.

To use the extension as it exists in OpenGL right now, we can just use 
the publicly exposed D3D API (GetSharedHandle/CreateSharedHandle for 
D3D11) to get a HANDLE to the device object, then run an IOCTL fetching 
the FD we feed into the extension.  Theoretically we could even use 
EXT_external_objects as an implementation detail for openCL, sharing the 
resource through two steps (first through EXT_external_objects then 
through CL_KHR_gl_sharing), but I imagine there would be some hitch we 
run into there, plus nothing uses OpenCL so why bother.

>
> * CL_KHR_gl_sharing
This doesn't involve what I am working on, both openGL and openCL are 
both host libraries, and should be able to share without any help from wine.
>
> * Shared resources in d3d9-11, when using the OpenGL backend

As mentioned, the EXT_external_objects extension exists, ( 
https://www.khronos.org/registry/OpenGL/extensions/EXT/EXT_external_objects.txt 
), which accepts and produces file descriptors compatible with the ones 
Vulkan produces and accepts.  The interactions here seem to be well defined.

>
> * Shared resources in d3d9-11, when using the Vulkan backend

The only additional challenge that faces both D3D implementations of any 
backend is that along with referencing the raw GPU payload, certain 
metadata is also stored in the resource objects.  However, this metadata 
is not stored by the host implementation (either GL or Vulkan), meaning 
it has to be tracked by us somehow.  For this, inside the device objects 
there will be user data that translation layers can access through IOCTLs.

As to not artificially hamstring certain implementations, I plan to 
leave the structure of this user data translation-layer defined.  To 
this end, I think it would be useful to store a GUID key needed to 
set/get the user data, so that translation layers doesn't try to parse 
user data with an unknown format.  This still leaves open the 
possibility of converging onto a single key if possible.

>
> * Shared resources in d3d12
Nothing special here, as the D3D12 model for sharing seems to be quite 
similar to the Vulkan one in that you share heaps, not resources.  It 
looks like it's a 1:1 mapping from a D3D12 heap to a Vulkan Device 
Memory object.
>
> * Shared resources between d3d9-12 devices, when using wined3d or 
> libvkd3d on Windows
Putting aside how rare and questionable of a use-case this must be, I 
don't think this concerns any of my patches.  My future patches would 
provide a way for translation layers on wine to store metadata alongside 
the host handlers, it doesn't have any correspondence to Windows.  If 
you really wanted to (why?), you could probably create your own backing 
storage in a shared memory object and accept the caveats that come along 
with that.
>
> * Shared resources between d3d9-12 devices and OpenGL or Vulkan, when 
> using wined3d or libvkd3d on Windows

Again, a questionable and rare use-case not related to the patches I 
intend to send, but I'll try to brainstorm something.

If you don't control the code trying to import your D3D resources into 
Vulkan or GL, you'll have to either

1) Figure out how to register the resources in your translation layer 
with Windows in such a way that it thinks the resources belong to the 
APIs you are implementing (as the Windows driver for Vulkan or GL will 
have to recognize it as such), or instead

2) Try to create an implicit Vulkan layer (for GL it would be much 
harder) that intercepts calls trying to import D3D objects, and instead 
imports the NT/KMT handles your translation layer can retrieve from Vulkan.

>
> Please give detailed walkthroughs, including code if possible, or 
> detailed explanations of why any of these can't be done.
>
>
> I note your tests also don't seem very interesting; they only share 
> objects within the same device. Sharing objects across multiple 
> devices, and across multiple processes, strikes me as more interesting.

In my opinion, given that the semantics for VK_KHR_external_memory_fd 
and VK_KHR_external_memory_win32 are so similar, I wonder how relevant 
such tests would be, it seems like the kind of thing the Vulkan CTS 
should add for testing drivers.

I think the tests should be more for testing the properties of the 
HANDLEs we get, as this is what we are implementing ourselves.  Maybe I 
should add more tests looking at the locations of the named objects in 
the NT directory, or tests relating to types of the objects and their 
access permissions.