Debugging Wine thoughts

[Chris Robinson]

On Wednesday 10 September 2008 10:44:09 pm Damjan Jovanovic wrote:
> For example applications don't expect to see pointers into the upper
> 1-2 GB of the 4 GB virtual memory address space because on Windows the
> kernel's memory is mapped there. But, ld-linux.so.2 could load
> libraries there, including libraries hosting Wine's DLLs, and pointers
> to memory in those would leak into the Windows code.

AFAIK, Wine doesn't load .dll.so files using the standard dl lib functions. At 
least, the dlopen/dlsym functions don't recognize the .dll.so files in a 
winelib app. What it does, again AFAIK, is mmap the lower 2-3GB range so it 
can put kernel32/etc where some apps expect it to be, and to mimick Windows' 
allocation algorithms.

However, because it's all premapped, further libc malloc calls can't use that 
same range, and will quickly run out of allocatable memory. This causes 
problems particularly with video cards that have 512MB VRAM or more, since 
there's not enough room to map and/or mirror the card resources.


An idea I had and mentioned on IRC a couple times is to have libwine expose 
functions that can be used by drivers and other native modules to 
allocate "win32 memory" instead of using the standard libc functions. It 
would be pretty easy for a driver or such to do:

void *hdl = dlopen("libwine.so", RTLD_NOLOAD);
void *(*mallocfunc)(size_t) = (hdl ? dlsym(hdl, "wine_malloc") : NULL);
void (*freefunc)(void*) = (hdl ? dlsym(hdl, "wine_free") : NULL);
if(!mallocfunc || !freefunc || dlerror() != NULL)
{
    mallocfunc = malloc;
    freefunc = free;
}
..use mallocfunc/freefunc for memory..

This will, of course, rely on drivers to be aware of Wine and handle it.

An alternative idea Alexandre had was to override libc's mmap, so anything 
loaded in the process would automatically use the new function (and thus not 
need any Wine-specific code). However, my attempts at doing that caused glibc 
to crash on init.