wine modules
Bernhard Kölbl
besentv at gmail.com
Thu Apr 14 12:38:50 CDT 2022
Very nice sum up of the whole situation. Sounds perfect for the wiki.
Bernhard
Eric Pouech <eric.pouech at orange.fr> schrieb am Do., 14. Apr. 2022, 17:26:
> I tried to write down what I grasped for the current modules' evolution
>
> since there's always questions floating around this subject, might be
> worthwhile to document it a bit
>
> it's likely full of errors, shortcuts, misconceptions... so don't hesitate
> to comment/amend
>
>
> Wine "old" module model:
> -----------------------
> Wine used to generate a module (DLL or EXE) as:
> - an .so (ELF) module, compiled with a native compiler
> - this .so module embeds also PE module information, so that Wine's loader
> exposes to Windows application the PE module only.
>
> +--------------------------+
> | FOOBAR.dll.so (ELF) |
> | +----------------------+ |
> | | PE information | |
> | +----------------------+ |
> +--------------------------+
>
> Wine loader is also able to load "pure" PE modules from .dll/.exe files.
>
> A built-in module (FOOBAR.dll.so) can use standard ELF dynamic linking to
> load
> also ELF libraries (eg winepulse needs to loads libpulse.so). This is
> needed to
> implement in Wine most of the integration into the Unix world; e.g:
> graphical
> (X11 or image coding/decoding jpeg...), audio/video (pulse, gstreamer), and
> other utilities (xml/xslt, database connectivity...) to name a few.
>
> +--------------------------+ +-----------+
> | FOOBAR.dll.so (ELF) | -------------> | helper.so |
> | +----------------------+ | ELF dyn link +-----------+
> | | PE information | |
> | +----------------------+ |
> +--------------------------+
>
> But, this had some limitations: some applications (version checking,
> anti-crack...) were trying to open directly the FOOBAR.dll file (as it
> exists on
> Windows) to read some elements inside. Wine implemented "fake" files to
> mimic
> that PE file content.
>
> +--------------------------+ +-----------+
> | FOOBAR.dll.so (ELF) | -------------> | helper.so |
> | +----------------------+ | ELF dyn link +-----------+
> | | PE information | |
> | +----------------------+ |
> +--------------------------+
>
> +--------------------------+
> | FOOBAR.dll (Fake PE) |
> +--------------------------+
>
> But this doesn't take into account 32 vs 64 bit applications (letting
> aside the
> 16bit part). So in fact, the global picture is closer to this:
>
> +------------------------------+ +-----------+
> | FOOBAR.dll.so (ELF 64bit) | -------------> | helper.so |
> | +------------------------+ | ELF dyn link | (64 bit) |
> | | PE information (64bit) | | +-----------+
> | +------------------------+ |
> +------------------------------+
>
> +------------------------------+
> | FOOBAR.dll (Fake 64 bit PE) |
> +------------------------------+
>
> +------------------------------+ +-----------+
> | FOOBAR.dll.so (ELF 32bit) | -------------> | helper.so |
> | +------------------------+ | ELF dyn link | (32 bit) |
> | | PE information (32bit) | | +-----------+
> | +------------------------+ |
> +------------------------------+
>
> +------------------------------+
> | FOOBAR.dll (Fake 32 bit PE) |
> +------------------------------+
>
> This requires that both 32 and 64bit .so libraries to be installed on the
> host.
>
> But again, this started to show some limitations:
> - some Linux distros and Apple are willing to drop the 32bit support from
> their
> packages, so the 32 bit helper.so libraries might not be available in the
> future
> - Apple evolution to support M1 family chip also adds another dimension to
> the
> above picture (as a an application can either be compiled for ARM/M1 or
> Intel
> chip, so there's also a need to support invocation from both processors
> (the
> ABI is different, to start with).
>
> Wine "new" PE model: (PE migration...)
> -------------------
> In this model, Wine requires a cross-compiler to generate directly a PE
> file for
> each module.
>
> +-----------------+
> | FOOBAR.dll (PE) |
> +-----------------+
>
> One of interesting benefit of this choice, is that the PE module can now
> simply
> link with Window's libc implementation (either msvcrt or ucrt). In the
> "old"
> model, the default was to use the unix libc libraries, yet providing some
> other
> potential differences.
>
> For modules not requiring system .so files, this migration is rather
> simple. There are still some elements to look at [1]. But most of the hard
> job
> is done by the compiler and the Wine infrastructure.
>
> For modules requiring linking to .so files, the scheme has to be extended
> by
> generating an additional ELF library, tied to the PE module.
> +-----------------+ +-----------------+
> +-----------+
> | FOOBAR.dll (PE) | | FOOBAR.so (ELF) | -------------> |
> helper.so |
> | | --------> | (unixlib) | ELF dyn link
> +-----------+
> | | | |
> +-----------------+ +-----------------+
>
> Addind this extra module in the middle allows to:
> - do the translation from a entities defined in PE world into entities
> defined in the ELF world (ABI are different, structures layout are
> different)
> - this provides an abstraction of the library for the PE world.
> - decide what to migrate from the existing code into the PE part (remember
> existing library in old model lives in the ELF world)
>
> One can see the FOOBAR.so like a Wine-"system driver" for providing the
> required
> support from the host.
>
> But, it brings constraints too. The ELF part cannot call back into the PE
> part
> of the module, nor into other PE modules.
> If this ELF parts requires some Wine "kernel features", the it can (using
> ELF
> dynamic linking) use the APIs provided by ntdll.so ELF library.
>
> +-----------------+ +-----------------+
> +-----------+
> | FOOBAR.dll (PE) | | FOOBAR.so (ELF) | -------------> |
> ntdll.so |
> | | --------> | (unixlib) | ELF dyn link
> +-----------+
> | | | | -------------> |
> helper.so |
> +-----------------+ +-----------------+
> +-----------+
>
> Note: likely some other modules .so should be available too (like win32u)
> (to be
> confirmed)
>
> Currently, the 32 <> 64 bit is still duplicated:
>
> +-----------------+ +------------------------+
> +--------------------+
> | FOOBAR.dll (PE) | | FOOBAR.so (ELF 64 bit) | ------------->
> | ntdll.so (64 bit) |
> | 64 bit | --------> | (unixlib 64 bit) | ELF dyn link
> +--------------------+
> | | | | ------------->
> | helper.so (64 bit) |
> +-----------------+ +------------------------+
> +--------------------+
>
> +-----------------+ +------------------------+
> +--------------------+
> | FOOBAR.dll (PE) | | FOOBAR.so (ELF 32 bit) | ------------->
> | ntdll.so (32 bit) |
> | 32 bit | --------> | (unixlib 32 bit) | ELF dyn link
> +--------------------+
> | | | | ------------->
> | helper.so (32 bit) |
> +-----------------+ +------------------------+
> +--------------------+
>
> When all modules are converted to the new scheme, this shall look like:
>
> +-----------------+ +------------------------+
> +--------------------+
> | FOOBAR.dll (PE) | | FOOBAR.so (ELF 64 bit) | ------------->
> | ntdll.so (64 bit) |
> | 64 bit | --------> | (unixlib ) | ELF dyn link
> +--------------------+
> | | +----> | (64 + 32 bit) | ------------->
> | helper.so (64 bit) |
> +-----------------+ | +------------------------+
> +--------------------+
> |
> +-----------------+ |
> | FOOBAR.dll (PE) | |
> | 32 bit | ---+
> | |
> +-----------------+
>
> This will require that FOOBAR.so correctly handles the "syscall" for both
> 32 bit
> and 64 bit unixlib calls.
> When this is in place, no support for 32-bit .so files will be necessary
> (both
> the packages that provide the .so files, but also for the development
> packages.
>
> Unixlib in a nutshell
> ---------------------
> A unixlib interface is just a table of entry points. Both ends (calling PE
> side, and callee on ELF side) share:
> - the semantic attached to an entry point a given index in the table
> (usually defined as a value inside an enum)
> - the signature of each entry point is always the same:
> NTSTATUS the_func(void *args);
> - args is a pointer to a structure containing input and output
> parameters to this very entry point.
> There's a specific structure for each function (in some rare cases,
> several
> functions can use the same structure).
>
> Invoking on PE side looks like:
> | struct xxx_yyy arg;
> | /* fill in arg */
> | NTSTATUS status = __wine_unixlib_callinto(foobar.so, /* pseudo code <g>
> */
> | 23 /* index of entry */,
> | &arg);
>
> An on ELF side:
> | static NTSTATUS my_func_23(void* parg)
> | {
> | struct xxx_yyy *arg = parg;
> | /* do the hard work */
> | }
> |
> | /* other functions */
> |
> | entry_point_table[] = {
> | /* ... */
> | my_func_23, /* index 23 */
> | /* ... */
> | };
> |
> | /* a bit of black magic to register the entry_point_table */
>
> Keep in mind:
> - the same structure can have different sizes for fields, and different
> layouts
> when compiled for PE or ELF, 32 or 64 bit, or different CPUs.
> So the ELF part must handle those discrepancies if any.
> - pointers will be of different sizes...
> - allocation of memory can be harder to implement on ELF side (as one must
> ensure that allocated memory can be accessed from 32 bit PE part)
> - exception handling?
>
> Actually, there will be two distinct tables:
> - one used for the invocation from 64 bit PE side,
> - the second one used for the invocation from 32 bit PE side.
>
> Upon invocation, their argument will be mapped to the 32 bit or the 64 bit
> layout of the argument structure.
> This allows either to adapt the implementation to any difference between
> the 32
> and 64 bit worlds.
>
> So the interface between the PE module and its .so counterpart has to be
> designed with care:
> + as already stated, no callback from ELF part into PE part is available,
> + some objects can be defined differently from both sides:
> - integers (int) doesn't have the same size
> - pointers are either 32 bit or 64 bit entities... so if memory has to be
> returned to PE side, it must be in lower part of memory (at least for
> 32 bit
> "syscalls")
> - ...
>
> So the migration of a module from the "old" model to this PE model
> requires to:
> - define the API between the PE and the ELF parts
> - ensure that code in the ELF doesn't call back into PE APIs
> - call through the Unixlib mechanism between the PE and ELF part.
> - the Unixlib call mechanism is available even if the module is generated
> in the
> old model, hence allowing to put the unixlib API in place still using
> the old
> model. When all is migrated to the Unixlib interface, module can be
> generated
> using the new module.
> (it may be that after reading this you need either/or a) a break, b) a
> deep
> breath, c) check your aspirin stock and d) restart from the beginning)
>
> TODO:
> - debugging
> - sketch the ARM/Intel on Darwin
>
> Additional bits:
> ---------------
> From the .so files that Wine used to link to, some of them are not tied to
> the
> physical system, and can be available directly as PE modules (eg. JPEG
> decoder,
> XML reader/writer...). The most important of those libraries are
> cross-compiled,
> and their source code inserted into Wine tree (see libs/).
> It's also possible to load the system-wide equivalent PE modules when
> installed.
>
> It's still not clear to me how the new module wow64 will be activated.
>
> [1]: on 64 bit, Windows chose the LLP64 integer model, while Unix chose
> the LP64
> one (sur. The main difference is that long is 32 bit on Windows and 64 bit
> on
> Unix.
> In the old model, as compilation was native, the Windows types were
> redefined to
> fit the LP64 model (like DWORD was an unsigned int to get the right size).
> With the migration to the new model, as compilation of PE modules is done
> on
> LLP64 models, DWORD (and friends) are now defined as on Windows as an
> unsigned
> long.
> This is called the migration to long types (90% of modules migrated) and
> takes
> place along side the PE migration.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.winehq.org/pipermail/wine-devel/attachments/20220414/3b4aa758/attachment.htm>
More information about the wine-devel
mailing list