[PATCH v7 4/4] winevulkan: Implement VK_KHR_external_memory_win32 for images.

[Zebediah Figura (she/her)]

On 5/25/21 1:11 PM, Derek Lesho wrote:
> On 5/25/21 11:52 AM, Zebediah Figura (she/her) wrote:
> 
>> How do you plan to implement the following features:
> For what it's worth, all of the ideas I express in this email don't
> apply this patch, but rather planned follow-up patches.  As it relates
> to Vulkan, I don't think there's any approach we take this path makes
> more of a headache.  For example, if we do decide to instead create
> custom object types for resources, all we have to do is extend the
> create_gpu_resource function in that direction.
>>
>> * named NT paths
>>
>> * security descriptors
>>
>> * KMT handles
> 
> I intend to take the approach of wrapping the FD object behind a device
> object, like you suggested in your previous mail.  Device objects can be
> named, and while support for access rights on device file objects
> doesn't seem to be implemented right now, it shouldn't be too hard to
> add this functionality.
> 
> https://source.winehq.org/git/wine.git/blob/HEAD:/dlls/ntoskrnl.exe/ntoskrnl.c#l530
> 
> The device objects will be created via IOCTLs to a master "Utility"
> device object, which will handle KMT lookup and user data storage.  For
> unnamed device objects I'm thinking about using
> FILE_AUTOGENERATED_DEVICE_NAME, retrieving the name of the device object
> IoCreateDevice returns, then proceeding as normal and opening/returning
> the handle.
> 
> 
>>
>> * CL_KHR_d3d11_sharing
> 
> OpenCL lacks any equivalent to GL's EXT_external_objects, which is
> "modern" approach to shared resources, meaning it is unable to directly
> share resources with Vulkan.  If needed in the future, I imagine it
> wouldn't be too hard to convince the OpenCL spec people to draft an
> equivalent extension.
> 
> To use the extension as it exists in OpenGL right now, we can just use
> the publicly exposed D3D API (GetSharedHandle/CreateSharedHandle for
> D3D11) to get a HANDLE to the device object, then run an IOCTL fetching
> the FD we feed into the extension.  Theoretically we could even use
> EXT_external_objects as an implementation detail for openCL, sharing the
> resource through two steps (first through EXT_external_objects then
> through CL_KHR_gl_sharing), but I imagine there would be some hitch we
> run into there, plus nothing uses OpenCL so why bother.
> 
>>
>> * CL_KHR_gl_sharing
> This doesn't involve what I am working on, both openGL and openCL are
> both host libraries, and should be able to share without any help from wine.
>>
>> * Shared resources in d3d9-11, when using the OpenGL backend
> 
> As mentioned, the EXT_external_objects extension exists, (
> https://www.khronos.org/registry/OpenGL/extensions/EXT/EXT_external_objects.txt
> ), which accepts and produces file descriptors compatible with the ones
> Vulkan produces and accepts.  The interactions here seem to be well defined.

According to my reading, neither GL_EXT_external_objects nor 
GL_EXT_external_objects_fd provide for export of an object from an 
OpenGL context; they only allow importing objects created by Vulkan.

> 
>>
>> * Shared resources in d3d9-11, when using the Vulkan backend
> 
> The only additional challenge that faces both D3D implementations of any
> backend is that along with referencing the raw GPU payload, certain
> metadata is also stored in the resource objects.  However, this metadata
> is not stored by the host implementation (either GL or Vulkan), meaning
> it has to be tracked by us somehow.  For this, inside the device objects
> there will be user data that translation layers can access through IOCTLs.
> 
> As to not artificially hamstring certain implementations, I plan to
> leave the structure of this user data translation-layer defined.  To
> this end, I think it would be useful to store a GUID key needed to
> set/get the user data, so that translation layers doesn't try to parse
> user data with an unknown format.  This still leaves open the
> possibility of converging onto a single key if possible.
> 
>>
>> * Shared resources in d3d12
> Nothing special here, as the D3D12 model for sharing seems to be quite
> similar to the Vulkan one in that you share heaps, not resources.  It
> looks like it's a 1:1 mapping from a D3D12 heap to a Vulkan Device
> Memory object.
>>
>> * Shared resources between d3d9-12 devices, when using wined3d or
>> libvkd3d on Windows
> Putting aside how rare and questionable of a use-case this must be, I
> don't think this concerns any of my patches.  My future patches would
> provide a way for translation layers on wine to store metadata alongside
> the host handlers, it doesn't have any correspondence to Windows.  If
> you really wanted to (why?), you could probably create your own backing
> storage in a shared memory object and accept the caveats that come along
> with that.

I suspect those of us who have worked with Direct3D for multiple decades 
would take issue with "rare and questionable", but regardless, part of 
the point here is to ask: *is* it possible to use a native set of APIs, 
like perhaps the D3DKMT*() functions from gdi32, to share resources? 
I.e. should the implementation live in gdi32 instead of 
winevulkan/opengl32/wined3d? Just from a quick scan I see several which 
look like the ones we want, and they're even documented.

>>
>> * Shared resources between d3d9-12 devices and OpenGL or Vulkan, when
>> using wined3d or libvkd3d on Windows
> 
> Again, a questionable and rare use-case not related to the patches I
> intend to send, but I'll try to brainstorm something.
> 
> If you don't control the code trying to import your D3D resources into
> Vulkan or GL, you'll have to either
> 
> 1) Figure out how to register the resources in your translation layer
> with Windows in such a way that it thinks the resources belong to the
> APIs you are implementing (as the Windows driver for Vulkan or GL will
> have to recognize it as such), or instead
> 
> 2) Try to create an implicit Vulkan layer (for GL it would be much
> harder) that intercepts calls trying to import D3D objects, and instead
> imports the NT/KMT handles your translation layer can retrieve from Vulkan.
> 
>>
>> Please give detailed walkthroughs, including code if possible, or
>> detailed explanations of why any of these can't be done.
>>
>>
>> I note your tests also don't seem very interesting; they only share
>> objects within the same device. Sharing objects across multiple
>> devices, and across multiple processes, strikes me as more interesting.
> 
> In my opinion, given that the semantics for VK_KHR_external_memory_fd
> and VK_KHR_external_memory_win32 are so similar, I wonder how relevant
> such tests would be, it seems like the kind of thing the Vulkan CTS
> should add for testing drivers.
> 
> I think the tests should be more for testing the properties of the
> HANDLEs we get, as this is what we are implementing ourselves.  Maybe I
> should add more tests looking at the locations of the named objects in
> the NT directory, or tests relating to types of the objects and their
> access permissions.
> 
> 

The point is more of a smoke test, because it's easy to get it wrong, 
and not too hard to verify that we got it right.

It'd also be nice to demonstrate what the behaviour of KMT handles is 
across processes.