32bit Wine on macOS Catalina

[Eddie Hillenbrand]

Hi all,

Reading everything I can find about Wine on macOS, seems like there
still isn't a good solution for 32bit on Catalina and beyond.[1] I'm
currently running Catalina and don't intend to upgrade my Mac or the OS
anytime soon.

My goal is to compile everything on Catalina and to run 32bit Wine on
Catalina. I can compile 64bit Wine without a problem and 32bit Wine
compiles against the MacOSX10.13.sdk.

I can run 32bit Wine with the `no32exec=0' boot argument and
DYLD_ROOT_PATH pointing at a directory containing `/System' & `/usr/lib'
from a machine running 10.13. It crashes because of kernel changes that
are incompatible with libraries from High Sierra. Specifically, (10.15)
libdispatch redefined work queue priorities, so the (10.15) kernel
starts the work queue thread with an invalid priority as understood by
(10.13) libdispatch.

    Exception Type: EXC_BAD_INSTRUCTION (SIGILL)
    Exception Codes: 0x0000000000000001, 0x0000000000000000
    Exception Note: EXC_CORPSE_NOTIFY

    Termination Signal: Illegal instruction: 4
    Termination Reason: Namespace SIGNAL, Code 0x4
    Terminating Process: exc handler [22976]

    Application Specific Information:
    dyld2 mode
    BUG IN CLIENT OF LIBDISPATCH: Corrupted priority

    [...]

    Thread 1 Crashed:
    0   libdispatch.dylib 0x0643aef5 _dispatch_worker_thread3 + 180
    1   libsystem_pthread.dylib 0x068c3fa5 _pthread_wqthread + 1356
    2   libsystem_pthread.dylib 0x068c3a32 start_wqthread + 34

Initially, I thought using DYLD_ROOT_PATH with 10.13 system directories
was the way to go. So I modified libdispatch to use the new priority
definitions and replaced `/usr/lib/system/libdispatch.dylib' in the
DYLD_ROOT_PATH with my modified version. This got past the work queue
crash, but a few other crashes emerged that needed to be squashed in
libdispatch, eventually I made it to NSApplication initialization.

Unfortunately, the application initialization crashes when it tries to
initialize the user defaults subsystem. I was hoping I would only have
to solve a few crashes in some of the open source libraries. Of course
this wasn't the case.

Since I had made it past the initial app bootstrapping and into ObjC
frameworks I was no longer in open source code. I could have kept
playing Whac-A-Mole, as it is especially easy to monkey patch ObjC code
with method swizzling. I considered going down this path, but before
going there, I decided to try a DYLD_ROOT_PATH with system directories
from 10.14.

I'm glad I tried using 10.14 dylibs before sinking more time into
patching and hacking my way through 10.13 dylibs. The 10.14 dylibs don't
crash, instead the WindowServer refuses to allow a connection from a
32bit process.

    _RegisterApplication(), FAILED TO REGISTER PROCESS WITH
    CPS/CoreGraphics in WindowServer, err=-304

I suspected this might be an AppKit limitation so I tried using the
deprecated Carbon libs and got the same error. Then I tried using
private WindowServer APIs directly, which didn't error in the same way,
but also didn't draw anything.[2] Compiling the same code for 64bit,
however, draws as expected.

A way forward for 32bit Wine is possible. Just do drawing in a 64bit
process. It might sound like a hack, but drawing or interacting with the
WindowServer is already done using IPC—it is just transparent to the
developer. I already made a proof of concept, so I know the technique is
solid.

Additionally, on Catalina at least, this removes the need for the
DYLD_ROOT_PATH since the 32bit process would, in theory, only need to
link with libSystem and it ships with 32bit binaries.

    $ sw_vers -productVersion
    10.15.7
    $ lipo -info /usr/lib/libSystem.dylib
    Architectures in the fat file: /usr/lib/libSystem.dylib are: x86_64
    i386

And by extension, most everything in the umbrella framework
`/usr/lib/system/*' also contains 32bit binaries.

My intention is to put the winemac.drv in its own 64bit process. I don't
really have a question, this is more a statement of intent, but I'd
welcome any feedback, thoughts, or concerns. If nothing else this email
memorializes the things someone is likely to encounter if they try to
make 32bit Wine run on Catalina or anyone that tries to run any 32bit
app on Catalina.

Eddie


[1] I'm aware of CrossOver, etc.

[2] I did eventually get one of the private functions to return -304
    which is probably significant as it matches the
    _RegisterApplication() error.