[PATCH] ntdll: Avoid the lower 2GB region for large allocations on win64.

[Elaine Lefler]

Fixes a crash in PaintTool SAI when allocating more than 2GB of memory.

Signed-off-by: Elaine Lefler <elaineclefler at gmail.com>
---

>From what I can tell, Sai basically requests an initial block of
memory from VirtualAlloc, then increments that pointer and calls
VirtualAlloc again with a fixed address when it wants more ram.
Blindly assuming that memory space is non-fragmented enough for the
call to succeed. This is definitely incorrect program behavior, but it
works on Windows, and after this tweak it's working on Wine too.

Heads up: I initially disabled the code path for all allocations, not
just large ones. On macOS this works fine. On Linux it's broken.
Somewhere, presumably in the X11 driver, we're truncating pointers to
32 bits. The change is safe for Windows apps, since Windows itself
seems to avoid the lower 4GB region, even for functions like HeapAlloc.

I can't find any apparent cause for the truncation, other than
converting HWNDs to XIDs, which doesn't seem to be the problem. I
wonder if there might be a confusion between "long" and "long long"
somewhere. Anyway, window data should never be large enough to trigger
the new behavior, so the broken behavior stays under containment until
it can be fixed properly.
---
 dlls/ntdll/unix/virtual.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/dlls/ntdll/unix/virtual.c b/dlls/ntdll/unix/virtual.c
index 94b300c5057..eac9bcb1f83 100644
--- a/dlls/ntdll/unix/virtual.c
+++ b/dlls/ntdll/unix/virtual.c
@@ -1917,7 +1917,11 @@ static NTSTATUS map_view( struct file_view **view_ret, void *base, size_t size,
         alloc.top_down = top_down;
         alloc.limit = (void*)(get_zero_bits_mask( zero_bits ) & (UINT_PTR)user_space_limit);
 
-        if (mmap_enum_reserved_areas( alloc_reserved_area_callback, &alloc, top_down ))
+        if (
+#ifdef _WIN64
+                size < 2 * 1024 * 1024 &&
+#endif
+                mmap_enum_reserved_areas( alloc_reserved_area_callback, &alloc, top_down ))
         {
             ptr = alloc.result;
             TRACE( "got mem in reserved area %p-%p\n", ptr, (char *)ptr + size );
-- 
2.32.0 (Apple Git-132)