Discussing address space for 32bit apps

[Henri Verbeet]

On Sun, 11 Nov 2018 at 14:53, Axel Davy <davyaxel0 at gmail.com> wrote:
> Wine emulates windows behavior of restricting virtual address space for
> 32 bits apps to the low 2GB.
>
>
> One issue though is that in this process space, some of the space is not
> directly used by the application. Indeed the linux libs loaded by wine
> (including things like pulseaudio) and the wine dlls take space in the
> process space.
>
>
> I am a gallium nine developer, and this has been particularly an issue
> for us for some games, as in addition to all the libs loaded without
> gallium nine, some extra libs are loaded.
>
> Some games will crash very fast. For example Mass Effect 1 complains
> being out of virtual mem when loading a save. We have a few opened
> issues related (https://github.com/iXit/Mesa-3D/issues/309 for example).
>
>
> We thought this would be fixed by having gallium nine use wine's
> HeapAlloc for all its allocations, but this hasn't helped. Indeed the
> problem doesn't seem to be gallium nine allocating memory, but gallium
> nine code (and other linux libs) being mapped on the process space and
> thus preventing that space to be used for allocations.
>
> A tester who doesn't use pulseaudio said he didn't run into issues with
> Mass Effect 1. Indeed the fact pulseaudio takes almost half the space
> allocated by all the linux libs, which could explain that.
>
> I also hit the Mass Effect 1 issue with wine without gallium nine, after
> an hour of play, which further proves this is not gallium nine specific,
> though gallium nine makes it worse.
>
> Making the game exe Large Space aware fixes the issue, as wine will not
> prevent allocation to be in the low 2GB.
>
>
> What are your thoughts on this issue ?
>
The short version is "Yes, it's hard, and don't use PulseAudio."

There's ultimately not a lot we can do about this, other than being
careful about our own address space usage, and trying to convince
libraries we depend on to do the same. PulseAudio is one offender, but
e.g. LLVM isn't great for address space usage either. If it's any
consolation, the situation is much worse on MacOS than on Linux.

> I understand it could be important for some apps to have the code it
> calls live in the low 2GB, but all the code of internal libs (libGL,
> libX11, gallium nine's d3dadapter) and maybe some allocations
> (pulseaudio, etc) could be mapped on the upper 2GB.
>
We do release some address space again after the application is loaded
and initialised (see virtual_release_address_space()/attach_dlls() in
ntdll). The way blocking the upper 2GB works though, it would be hard
to allow things like libGL to be loaded there while at the same time
disallowing e.g. d3d9.dll.so to be loaded there. At some point the
idea was floated to implement our own ELF loader to have more control
over where in the address space things are loaded, but that never
happened. Perhaps glibc could be convinced to add some RTLD_ flag for
us. Even then, I'd expect the larger issue to still be libraries like
PulseAudio being wasteful with address space, simply because they
primarily target 64-bit systems. Using separate debug symbols (or just
stripping them completely) also helps a little bit.

Henri