Request for discussion: Using PE libraries for Wine dependencies

[Zebediah Figura]

On 4/13/20 2:17 PM, Jacek Caban wrote:
> On 13.04.2020 19:38, Zebediah Figura wrote:
>> On 4/13/20 11:38 AM, Jacek Caban wrote:
>>> Hi Zebediah,
>>>
>>> I really think we should at least experiment and evaluate using winelib
>>> first. My previous mail didn't really explain details behind this and
>>> was a bit messy, so let me try now.
>>>
>>> First, the name mingw or mingw-w64 is used to name different things
>>> depending on the context. For sake of this mail, let me use it to call
>>> the complete toolchain for (cross, but not only) compiling a Window
>>> target. mingw based on GCC contains following parts from different
>>> projects (LLVM mingw variant is similar, except that llvm/clang is
>>> always a cross compiler, so this part doesn't need to be built with
>>> mingw in mind and instead a fairly regular build can be used):
>>>
>>> - GCC + binutils tools built with mingw as a target (from gcc and
>>> binutils tree)
>>> - libgcc cross compiled for mingw target (from gcc tree, but depends on
>>> crt part of mingw-w64)
>>> - crt headers (from mingw-w64 repo)
>>> - crt static libraries and startup files (libmingwex, libmingw32,
>>> libmsvcrt, etc, from mingw-w64 repo)
>>> - platform headers (from mingw-w64 repo)
>>> - platform importlibs (from mingw-w64 repo)
>>> - optionally winpthreads, which messes with crt headers and crt libs
>>> (from mingw-w64 repo)
>>> - libstdc++ and a few more component not important for sake of this
>>> conversation
>>>
>>> Wine is not a typical mingw user, because it actually implements the
>>> platform for which it builds. As such, we need to provide majority of
>>> the stuff that other projects would use from mingw-w64. Initially, Wine
>>> used an exotic mix for crt: it used its own headers and msvcrt
>>> importlib, but mingw-w64 for the rest of static libs and startup files.
>>> It meant that Wine dependent on mingw-w64 internals that it shouldn't
>>> touch. mingw-w64 assumes that crt headers + crt libs are shipped
>>> together and it has the right to do so. It worked, but it was fragile.
>>> Anything that Wine didn't take into account (like configuring mingw-w64
>>> to use ucrt by default) could cause a problem. It's probably just a
>>> matter of time when wine-5.0 will not build with future mingw-w64
>>> releases and it will not be mingw-w64's fault.
>>>
>>> We looked for a solution to that problem. One way to solve this would be
>>> tightening projects relationships. It's wasn't clear how technically
>>> that could look, but we had to And that's true for both way of doing it. put that idea aside for non-technical
>>> reasons anyway.
>>>
>>> The other solution is to not mix crt parts at all. This required Wine to
>>> provide the few missing bits (mostly startup files) by itself. I'd say
>>> that it makes sense: we provide similar things for ELF builds anyway and
>>> we provide msvcrt implementation and headers, it feels natural to
>>> provide static library bits as well. Since February (or August last year
>>> for majority of the codebase), Wine does not use any part of mingw-w64
>>> repo directly at all. We always use our own headers and -nodefaultlibs.
>>> Wine cross toolchain dependency is reduced to just GCC + binutils +
>>> libgcc (and mingw-w64 as build time libgcc dependency) from the list
>>> above. That's what you get with winegcc -mno-cygwin (without
>>> -mno-cygwin, we still use the exotic mix for mingw targets and we still
>>> use that for PE-only builds).
>>>
>>> So now that we have this powerful tool and successfully use it for Wine
>>> build, it feels natural and technically right to use it for the rest of
>>> Wine ecosystem. What you propose would instead bring back mixing crt
>>> parts. Linking current Wine against external mingw-compiled (as opposed
>>> to winegcc -mno-cygwin) library is essentially linking a static lib with
>>> a different ABI. Headers should mostly work, but it's still shaky.
>>>
>>> I didn't try winegcc yet with non-Wine code myself mostly because I
>>> still think there are things that we should get right first. My ucrt
>>> work was partially motivated by this. We should be able to default to
>>> ucrt in winegcc now (we already do that for Wine in makedep, so I'd say
>>> we're ready for that). There are a few other things, but we're getting
>>> there.
>> Admittedly I'm really unhappy with the way that discussion went; it
>> seemed to die immediately before any discussion was made about *how* we
>> could share code. I do think it was a mistake to try to take mingw under
>> the umbrella of Wine—as I see it, mingw targets the Win32 API, and Wine
>> is just one implementation—
> 
> 
> That's one way to look at it. I'd say that implementation and SDK for 
> the implementation fit naturally together (just like Microsoft provides 
> both implementation and SDK of Win32 API).

To a degree, though

(1) most things built with mingw are not built to run on Wine;

(2) anything with mingw has the ability to run on any implementation, at
least in theory;

(3) Wine isn't even the only open-source implementation of Win32;

(4) I imagine we'd all be using Microsoft SDK headers if they weren't
copyrighted.

>> but I don't particularly understand how that
>> precludes more sharing of headers, libraries, and even CRT bits. I know
>> that you've done a lot of work to disentangle us from mingw libraries,
>> but is there any technical reason why we can't go the other direction?
> 
> 
> I don't see how that could be done without bad effects on the workflow. 
> What would you do if you wanted to add a trivial test for a new API? One 
> that currently would take a few minutes to write and a single patch to send.

That is a good point. In my experience mingw is pretty quick to review
and commit patches, but awfully slow to release new versions. Maybe
that's something that could be improved, though.

>> Even setting that aside, broadly a mingw library should work on Windows,
>> so I'd expect it to work on Wine too (and since we're not talking about
>> cygwin here, I think we can assume they won't do dumb things like
>> alloca() several pages in DLL_PROCESS_ATTACH and expect that memory to
>> remain untouched).
> 
> 
> I think I explained why it would be wrong for static libs. If you mean 
> when built as a separated DLL, it would work in most cases, yes. That's 
> assuming that library's API doesn't expose things like fds, FILEs, etc.

Sure, I was assuming dynamic libraries.

As long as we link to the right CRT version I guess we shouldn't have
trouble. I have to presume that's something that any mingw user would
have to worry about anyway.

>>> On 13.04.2020 01:18, Zebediah Figura wrote:
>>>> So, I figure it's probably worth quantifying what we'd gain this way
>>>> (as well as how many libraries we'd potentially have to build or
>>>> maintain). As I see it, the libraries that can be built as PE (unless
>>>> I misunderstand what any one of them requires, so please correct me if
>>>> I'm wrong) are:
>>>
>>> Some of those are not obvious and will need a separated discussion, but
>>> let's not distract from the main topic for now. I think that the list
>>> will look somewhat like this.
>>>
>>>
>>>> On 2/14/20 12:43 PM, Esme Povirk (they/them) wrote:
>>>>> Options for distributing dependencies:
>>>>>    * Distribute headers and libs in a tarball built from wine-pedeps.
>>>>> The binaries would be part of an addon, also built from wine-pedeps.
>>>> I personally don't like this. I think there's value in
>>>> (1) not requiring the user to install libraries they don't need (like,
>>>> say, libkrb5),
>>>
>>> I think that fully functional Wine is the right default. If you think
>>> about addons as a part of Wine ecosystem, it's not much different from
>>> requiring the user to install jscript.dll even if they don't need it.
>> Yes, I suppose I had better accept that :-/
>>
>>>> (2) not prompting for every add-on (personally I feel that two is
>>>> already too many),
>>>
>>> Sure, it would be nice to improve user experience with Wine addons. The
>>> reason I'm not motivated to work on it myself is that it's really just a
>>> fallback. If distro does its job well (and I think most of distros do),
>>> users don't need that anyway. IMHO we should concentrate on making it
>>> easy for distros to ship addons themselves, but fallback improvements
>>> are welcomed as well.
>>>
>>>
>>>> (3) letting the distribution choose what to distribute by default.
>>>
>>> Sure, it should be possible to disable things at build time. Just like
>>> it's already possible to disable jscript.dll at build time.
>>>
>>>
>>>>>    * Rely on the host distribution to package mingw headers, libs, and
>>>>> binaries for the dependencies we need. The binaries would then be
>>>>> distributed with Wine. Fedora is the only distribution I'm aware of
>>>>> that has an extensive library of mingw packages. To make this easier
>>>>> on other distros, I could maintain wine-pedeps as an alternative
>>>>> source of headers, libraries, and binaries at build time. The trouble
>>>>> with this approach is that it's not clear how we can keep Wine's
>>>>> libraries independent from the application.
>>>> This was the idea that I proposed when the discussion came up
>>>> internally. It strikes me as the architecturally "correct" thing to
>>>> do. We're essentially building some of our files with a different host
>>>> architecture, so just as we'd link separately to x86_64-pc-linux-gnu
>>>> libraries and i686-pc-linux-gnu libraries, so too would we link
>>>> separately to x86_64-w64-mingw32 libraries (as well as i686-w64-mingw32).
>>>
>>> Yes, some distros provide such packages and it's nice. I happened to
>>> contribute to that on mingw-w64 side. The purpose of those packages is,
>>> however, different than what we need for Wine. It's meant to provide a
>>> nice way for cross compiling projects with open source dependences. And
>>> once you cross compile such an application and want to ship it, you need
>>> to copy required DLLs or statically link dependencies. It's a very
>>> different goal than assembling Windows distro that we need.
>> It's not clear to me that it is that different, though. We're also
>> cross-compiling high-level code with open-source dependencies, like
>> windowscodecs. The only difference is that we're also shipping the
>> reimplementation of the lower-level code and the loader. To illustrate
>> that it's pretty much the same thing, I think it should even be possible
>> to e.g. use wine's windowscodecs on a real Windows box, after copying
>> over the dependencies. Frankly, the only difference I see with regard to
>> wineprefixes is that we don't necessarily have to copy the system
>> libraries into the wineprefix.
> 
> 
> I don't negate that it will work. I'm just saying that I don't think 
> it's technically the best way of doing it.
> 
> 
>>> What you propose is essentially forcing packagers to assembly and
>>> Windows (Wine) distro and I don't like that. Linux distros should
>>> concentrate on providing good native userspace and it's Wine's
>>> responsibility to use that to provide Windows-compatible environment.
>>> The fact that we will leverage PE build of some project should be
>>> something that packager can control, but he should not be forced to take
>>> care and maintain 20 packages only for Wine.
>> Sure, I don't see a problem with letting the packaging be Wine's
>> responsibility, if Wine is the only user. That doesn't mean that we
>> can't follow the existing model for mingw packages, though.
>>
>>> We definitely don't want such Wine dependences to be used in place of
>>> application provided DLLs with the same name. Renaming the library in
>>> build time seems like way cleaner way than any runtime heuristics that
>>> this thread led to.
>> I'm not sure I agree with this (obviously we do want to use native
>> versions where the application provides them, but I'm not sure I see how
>> renaming is necessarily cleaner), but it's pretty much a separate issue
>> so I'll leave it aside for now.
>>
>>> Another thing where your solution can become a nightmare is ABI
>>> incompatibility. We don't control the dependencies and some of them are
>>> not ABI stable. We just need to accept it and make sure our solution
>>> takes that into account. So for example, when something like this
>>> happens (I don't think this is an isolated example, but I happen to know
>>> about it and it's hilarious: they had an incompatible MSVC and mingw
>>> ABI, so decided to break and uglify mingw ABI to follow MSVC and hack
>>> around a problem that I believe is imaginary):
>>>
>>> https://bugzilla.libsdl.org/show_bug.cgi?id=4771
>>>
>>> What I propose is to have a solution where we just bump just one
>>> constant in one repo, which would make Wine use wine-sdl2-2.dll instead
>>> of wine-sdl2-1.dll and that's it. All distros are be happy.
>>>
>>> What you propose is that now tens of distros would need to notice the
>>> problem and express it in their package dependences. It means tens of
>>> action tickets spread around the ecosystem and a few Wine NOTOURBUG bugs
>>> reported upstream. And all of that work is just to provide a library
>>> that's otherwise internal to Wine anyway.
>> I think I may have poorly communicated—I don't think it necessarily has
>> to be the distribution's job to package mingw libraries. Of course it's
>> nicer if they do—and if there's already distribution libraries I think
>> we can use them—but I'm fully expecting that we'll have to package most
>> libraries for most distributions, for a time if not in perpetuity. If
>> we're the only user of those libraries I think that's reasonable.
> 
> 
> Well, my hope is that distros will like it and adopt it themselves. If 
> building the whole thing is as easy as pulling wine-meta-distro and running:
> 
> ./configure; make; make install
> 
> and get fully functional Wine with all PE dependences, I hope distros 
> will choose to use it. We've been there when we introduced Wine Gecko. I 
> think the ecosystem is in much better shape now and adaptation may be 
> much smoother.
> 
> 
> Still, we will need to provide those dependences in our packages as 
> well. But if we make it easy for distro packagers, it will be easy for 
> us as well (or, well, the another way around).

Right, I forsee it'd be as simple as that using mingw structure as well.
You'd just also have to install the package(s). We could even make a
'wine-common' dummy package that pulls in all of the dependencies, I guess.

>> Hence ABI incompatibility is dealt with the same way it usually is, by
>> requiring specific library versions. If we have to build and ship
>> separate versions because the distribution's version is out of date,
>> well, that's the life we already have to live :-/
> 
> 
> If a solution helps to avoid dealing with that, I'd say it's a good 
> thing. Also, on Linux, there are common practices like different .so 
> suffices for ABI version. There is no such thing on Windows. What I 
> proposed is really doing an analogous thing.

The way I see it, if we don't rely on mingw libraries / packages /
structure, we have to build a specific version of everything anyway. If
we do, we *might* have to build a specific version, if a library does
break ABI.

>>>> I don't know what we'd do about Mac, but along the same lines, I'd
>>>> think we'd want to extend/analogize the current solution regarding
>>>> bitness to the whole architecture triple.
>>>
>>> Well Mac would be even easier to target than it is now if we provide
>>> right tools (because some of those dependences are tricky on Mac
>>> already). And if some packagers really insist on splitting those
>>> packages, they can still do that. with what I proposed. They can use our
>>> build system as a 'driver' or do it manually, that's all fine. We'd be
>>> not forcing one way or another.
>>>
>>>
>>> So let me give a few examples of different choices packagers can do. I
>>> will call wine-core Wine as we know today (which will not change
>>> architecturally) and wine-meta-distro the proposed separated repo for
>>> dependencies build system allowing a simple and straightforward way to
>>> bootstrap complete Wine+dependences package, depending only on cross
>>> compilation tools.
>>>
>>>
>>> 1. Not care, provide just wine-core package (addon headers are just a
>>> build time dependency). Wine will download and cache addons in runtime.
>>>
>>> 2. Use wine-core the way distro currently do to provide Wine package.
>>> Use wine-meta-distro to build Wine addons as one but separated package.
>>>
>>> 3. Use wine-meta-distro to produce both a fresh wine-core and wine
>>> addons package in a single step for a single package.
>>>
>>> 4. Like 2 or 3, but distro wants some customization itself. They can do
>>> that, for example, by pointing wine-meta-distro to its own version of
>>> relevant package, pass additional config options or do simple source
>>> code modifications.
>>>
>>> 5. Split Wine into 20 packages. A distro can use wine-meta-distro to do
>>> build config stuff, so that they don't need to change their scripts
>>> every time something changes in any of dependent packages, but otherwise
>>> treat every dependency as a separated package.
>>>
>>> 6. Split into 20 packages and deal with them without wine-meta-distro
>>> packages. This will require packagers to redo all the work that
>>> packagers choosing 1-5 will be getting for free.
>>>
>>>
>>> Your proposal seems to be similar to something between 5 and 6. I don't
>>> expect packagers to be thrilled with that idea if they have 2-4 as
>>> alternatives, by the key point is: let's give them a choice.
>> Pretty much 5, I think. If I understand correctly, 6 is only true when
>> distributions are already shipping mingw packages (and only for those
>> packages), in which case they're not really redoing any work.
> 
> 
> The other important part of 5 is that you can build all those packages 
> in a combination of versions that are tested and guaranteed by community 
> to work (unless packager chooses to do otherwise). My understanding is 
> that you meant packagers to do arbitrary version choices by default.

I'm not admittedly sure what's meant by arbitrary version choices, or
perhaps this paragraph in general, but I'm imagining that mingw packages
will be no different than non-mingw packages in most respects, so
we/distributions can do whatever testing is desired.

>> I don't work in distributions, but I don't particularly understand why 5
>> is any harder than 2-4, or what's gained from having 2-4 as an option.
> 
> 
> The gain from 2-4 is simplicity that I mentioned earlier, which I think 
> is crucial if we want wide adaptation. What's gained by 5?

Essentially, that it's the way things are already done.

I guess I'm also still not understanding why 2-4 is simpler, except I
guess it doesn't require any packages to be installed as build
dependencies—which doesn't seem like a huge advantage, since that's
basically just the norm for package building.

> 
> 
> Jacek
> 
> 
> 

Ultimately I guess I should summarize my interest in all this:

* I'm generally in favor of working with an existing system rather than
discarding it and reimplementing it;

* I'd really rather not have to build twenty libraries as part of
building Wine (so I really don't like Rémi's proposal, or any proposal
involving submodules);

* the less we have to rebuild (or re-download!) every time a core CRT
header or static library gets changed, the better;

* while I understand that addons "should" be the responsibility of the
distribution, currently my distribution doesn't handle them, and I do
have to click "yes" every time; I'd rather not have to click "yes" more
times (or be forced to change distributions, or have to upstream such a
package);

* I'd rather not needlessly duplicate libraries in general, hence I
don't like the idea of using static libraries or building Wine-specific
versions. Wine may be the only user of most of these, though, so this
point is not particularly salient.