[RFC PATCH 0/5] XAudio PE conversion.

[Rémi Bernon]

On 6/23/21 7:11 PM, Henri Verbeet wrote:
 > On Wed, 23 Jun 2021 at 17:23, Zebediah Figura (she/her)
 > <zfigura at codeweavers.com> wrote:
 >> I don't think this is a very good idea, for several reasons:
 >>
 > Yeah, I'm definitely not a fan of the proprietary/"modern" approach of
 > just copying everything into the same source tree, instead of building
 > proper interfaces and infrastructure.
 >

I think this approach is meaningful and valuable when two projects are 
separate.

It's not and it gets in the way when the two are just meant to live 
together and are developed in a tightly coupled cycles, especially by 
the same people, following the same development processes (less true for 
FAudio).

On 6/23/21 7:15 PM, Zebediah Figura (she/her) wrote:
> On 6/23/21 11:33 AM, Rémi Bernon wrote:
>> On 6/23/21 5:22 PM, Zebediah Figura (she/her) wrote:
>>> On 6/23/21 7:37 AM, Rémi Bernon wrote:
>>>> I'm sending this as an RFC for now as there's been some discussion
>>>> already about how to handle PE dependencies. I think the approach I'm
>>>> taking here, by including FAudio headers directly in the source and
>>>> loading it dynamically is the easiest way, and the least disruptive for
>>>> building and running Wine, if not the cleanest. All the other solutions
>>>> have higher friction, either on third party packages, or on the system
>>>> distribution.
>>>
>>> I don't think this is a very good idea, for several reasons:
>>>
>>> * firstly, it's a huge amount of imported code;
>>>
>>
>> I honestly don't think it is.
> 
> It's 3482 lines of code; I'm not sure what you count as huge if not 
> that...?
> 

XAudio source, which is mostly just plumbing around FAudio is 6k LoC?

deflate.c is 3k?

I don't know but this is not huge, and there's no logic there, merely 
type and function declarations.

>>> * secondly, the headers can easily change in the near future (and FAudio
>>> has usually taken a loose approach to compatibility);
>>>
>>
>>   From what I could see it's usually okay, with new functions and
>> structures added when they require a new ABI.
>>
>> I don't see how this is so different from the current situation, as the
>> current source shows it, we only check for the new function presence in
>> the public headers.
>>
>> The struct changes could be a problem, but only if we don't control the
>> distribution of the PE FAudio. If we consider FAudio to be provided by
>> Wine Mono for instance, it's merely a matter of making sure Wine Mono
>> updates also update the headers here.
> 
> I'm not terribly concerned about API breakage, but I'm a lot more 
> concerned than I would be with some other libraries.
> 
> More to the point, I don't want to be constantly updating headers when 
> new features are added.
> 
> vkd3d is in a similar situation, and also evolving very quickly.
> 

For FAudio it's not changing so much as far as I can see.

>>> * thirdly, at least some distributions actually explicitly don't like
>>> this [1].
>>>
>>
>> This more precisely applies to sources, and sources which are already
>> packaged as separate Debian packages. This is not the case here, and
>> going that way is an entirely separate approach which IMHO has a good
>> number of downsides.
> 
> Maybe Debian disagrees, but I'm not sure why headers don't count here. 
> Note also that Debian generally has separate packages specifically for 
> the headers.
> 
> I know that dependency management (and ABI compatibility, for that 
> matter) has fallen out of style in some circles, in favor of statically 
> linking everything or shipping all your dependencies, but I really don't 
> think that's a good thing, it has easily visible negative effects, and 
> while I don't really want to get into an argument over it I also don't 
> want to see us adopt that philosophy without question.
> 

Again I'm not against proper dependency management, but I don't think 
it's a hard rule either and like everything it's a matter of pros and 
cons. It actually probably makes more sense for FAudio than it does for 
vkd3d, as FAudio is currently already used for porting XNA games.

Forcefully splitting everything into dynamic libraries just for the sake 
of defining proper interfaces at every level doesn't make much sense. We 
don't have proper dependency management for Wine internals, we create 
interfaces as we need to, and we change them as we need.

>>> As I've said, I think it's past time we ask distributions what they
>>> actually want, and work with them accordingly. My guess is it'll be some
>>> variation of pkg-config.
>>>
>>
>> I don't think this is only a matter of discussion with the distros.
>>
>> Having the dependencies provided by the system creates a much higher
>> friction when building and running Wine from source.
>>
>> Although I like the idea in theory, it's something I don't look forward
>> to. Multi-arch is already so much of a mess than people started relying
>> on container to build things in a sane environment.
>>
>> So far Wine will be the first and probably only user ever of these new
>> MinGW dependencies, and I don't think it's nice to delegate this to 
>> distros.
> 
> We're not the only user of PE libraries, or Debian wouldn't already be 
> shipping some. See [1] for example.
> 

Yes they ship some, and I'm sure they do to make development of Windows 
applications easier on Linux. FAudio doesn't make anything easier here, 
people would target XAudio libraries directly, other dependencies may be 
more useful (libjpeg, etc...).

> More to the point, all I'm really trying to advocate for is *asking 
> first*. I don't necessarily claim that distributions *want* to ship 
> mingw dependencies, but we haven't asked them, and in the case that they 
> *do* want to—which seems more than plausible to me—we should work with 
> them rather than doing everything ourselves. The point is, nobody's 
> asked yet.
> 

Thing is that I'm personally not going to ask because I'm not already 
convinced it's a good idea, and I'm not sure I want to use my system 
distribution for this case.

> For that matter, as to my own opinions on what Should Be Done, I'm not 
> even attached to asking distributions to ship PE dependencies. I'd be 
> happy to do that from winehq hosted repositories. I *am* attached to 
> using deb/rpm/pkgbuild packages rather than inventing our own, on the 
> other hand.
> 
> FWIW, multiarch isn't actually a mess anymore; I can build wine just 
> fine without a container on Debian [2] and Arch, and even if it was 
> there is a right solution, which is *fix multiarch*.
> 
> [1] https://packages.debian.org/sid/libz-mingw-w64-dev
> 

Yes it's getting better, but people moved on already.

And to the contrary to multi-arch, which has a well established way of 
building, with packages being built as part of the other architectures 
supported by Debian (for instance), MinGW is not and never will be a 
supported architecture.

Packages will need to be defined and maintained separately, with an even 
less standardized filesystem location. Some use 
/usr/<mingw-arch>/{include,lib}, while multi-arch usually uses 
/usr/{include,lib}/<arch>/, but that may vary.

> [2] With the exception of GStreamer :-(
> 

So still mess, and it's been a while.

>> We can probably use (cross-)pkg-config for the build-time discovery
>> (however bad it will be to integrate to configure, I tried but quickly
>> bailed out), but then comes the problem to how to load the DLLs at 
>> runtime.
> 
> Right, it doesn't work currently, I also tried to see if it would work 
> with system libz. Unfortunately I couldn't easily make it work. But it 
> seems plausible that we'd want to make it possible for it to work. Maybe 
> that means tweaking MinGW pkg-config a bit.
> 

At least on my debian I can use prefixed pkg-config, like 
x86_64-w64-mingw32-pkg-config and that works fine, but Wine configure 
overrides PKG_CONFIG_PATH (and it probably shouldn't), and in any case 
this needs to be implemented somehow in m4 language which I absolutely 
don't master.

>> We usually compile-in the SO name, because dlopen has a set of builtin
>> search paths, but that won't work with PE file, unless we add such paths
>> to Wine loader.
> 
> That sounds like a good idea to me.
> 
>> I can also pretty easily see DRMs or anti-tamper mechanisms trigger if
>> an unknown module is loaded from some non-standard location.
> 
> I think there are solutions to this. One that comes to mind is 
> symlinking. If we end up using a special reparse tag for system32 DLLs 
> (to avoid copying them into the prefix while still pretending that 
> they're in the prefix to a Win32 application) we could use the same 
> solution for PE dependencies.
> 
> (Also, doesn't this patch series do exactly that anyway?)
> 

Yes, it loads the DLL from Wine Mono installation as a fallback. IMHO it 
would be better if FAudio was in system32. And even better if we could 
build it as a builtin DLL and have it installed already as part of Wine 
build / installation.

>> Overall this seems way more complicated, and IMHO too much for this
>> particular case.
>>
>>
>>>> Upstream FAudio now has a pure Win32 backend, which makes it much 
>>>> easier
>>>> to build as PE. This was never a requirement, and this series should
>>>> work too with the SDL2 backend, as it tries to load Wine Mono SDL2 and
>>>> FAudio as a fallback after trying from the prefix system libraries.
>>>>
>>>> It could help the transition though, and is going to be required anyway
>>>> to support WMA decoding. Right now, with or without the Win32 backend,
>>>> this would also introduce a functional regression as although supported
>>>> by the new FAudio Win32 backend, it also depends on the WMA decoder MF
>>>> transform to be implemented, which isn't the case yet in Wine.
>>>
>>> What if Wine is built without MinGW? Do we care about that case? If we
>>> don't care about it for XAudio, what about other libraries?
>>>
>>
>> Is this still something we should worry about with the PE conversion? I
>> don't see any sane path ahead if we add that problem on top, unless we
>> just include FAudio source and be done with it.
> 
> Well, I'm not sure it'd be too hard to maintain alternate 
> dlopen/LoadLibrary paths, for instance, and to use different paths to 
> search for the library depending on whether we're using mingw.
> 

It's not only about that. XAudio also has a nasty thread swap mechanism 
to support the case where the FAudio processing thread is created by 
SDL2, and is not a Wine thread. Of course this could be ifdefed too, but 
I think everybody will be happy if we could drop it.

>> To be completely honest I don't really understand the state of XAudio.
>> Having just this thin shell just makes everything more complicated than
>> it needs.
>>
>> The DLLs are pretty useless without FAudio, so either we should delegate
>> the build of fully functional XAudio DLLs to FAudio which it already
>> (partially) supports, or we should be able to build fully functional
>> XAudio DLLs directly in Wine tree.
>>
>> This remarks only applies to some of the dependencies, not ones where
>> they are only part of the functionality, or, like winex11.drv just not
>> possibly integrated, but it also definitely applies to, say, vkd3d.
> 
> I think FAudio and vkd3d are the particularly important cases, but other 
> PE dependencies could (possibly) include libmpg123, libgsm, libpng, 
> libjpeg, libtiff, libgnutls, libfreetype, libxml2, libOSMesa, libz.
> 
>> I don't completely understand [1] why it's separate and IMHO it makes
>> integration and working on both Wine and vkd3d just horribly painful.
>>
>> [1] Well I understand source ports, but I don't see why having it live
>> in Wine would make source ports more complicated. I understand wined3d
>> is already used that way, and it should be pretty much possible to do
>> the same for vkd3d.
> 
> I'm not deeply involved in either decision, but I'd offer as an 
> explanation that Wine is comparatively very heavy-weight. It takes up a 
> lot of space, consumes a decent number of resources while running, 
> requires that it be the one to set up the process and launch your main 
> executable, and it's not exactly trivial to compile winelib applications.
> 

I didn't mean using Wine to run an application,

> One could make an argument for having the FAudio or vkd3d source live in 
> the wine tree, though, which strikes me as a bit more reasonable. I 
> can't as easily come up with reasons not to do that.
> 

but this instead. For vkd3d in particular, Wine is its primary user, and 
shares the developers too.

Having well-defined interfaces and proper ABI compatibility is something 
I perfectly understand, when a piece of software needs to be distributed 
and used separately. The only reason I can understand for Wine to go 
through that interface, is that it makes a good test case, but then it 
also makes integration, testing, and d3d12 updates in Wine more 
complicated than it needs to be.

The vkd3d internals also then cannot benefit from Wine facilities, such 
as debug logging, and have to duplicate them in vkd3d source, with some 
alternate variable to know about.

Wouldn't it be easier to just use D3D12 interface as the public 
interface to vkd3d? And just build d3d12.dll / d3d12.so there directly 
instead of having a small piece still living in Wine? Same for XAudio.

And having vkd3d live in Wine source wouldn't make it impossible to 
build a standalone SO version of it with well defined interface too, to 
support source ports, either with some configure flags, or using a 
separate configure / Makefile in d3d12 folder.
-- 
Rémi Bernon <rbernon at codeweavers.com>