Problems with VirtualAlloc/Lock

[Michael Ost]

On Mon, 2005-03-14 at 20:13, you wrote:
> > * VirtualLock does nothing in Wine
> VirtualLock does nothing in win95,98,ME as well :)
> 
> I bet the correct behaviour for wine is to do anything in VirtualLock
> only if you set windows version to NT/2000/XP. Did you do it?

Good point. But the setting is "winxp". I played with the setting and it
doesn't change the behavior.

> Anyway, mlock() seems to work fine, so this should be implementable.

That would be the natural call to use. The absence of mlock in the wine
source tree was one of my early clues that there was a problem! %)
 
> > * Wine makes no distinction between MEM_RESERVE and MEM_COMMIT
> Assuming that we're talking about VirtualAlloc().

Yes, but I wasn't specific enough. VirtualAlloc(MEM_RESERVE) and
MEM_COMMIT are handled differently in the code. I don't understand the
code, the design, or Linux mmap() sufficiently to say if they work
correctly or not. They both end up calling mmap() which looks right.

What I was trying to say is that in winxp MEM_RESERVE and MEM_COMMIT
will VirtualAlloc() and/or VirtualLock very different amounts of memory
than wine, even on the same (dual booting) machine.

Here were my results when I tried to see how much winxp and wine would
allocate and lock. The machine has 368M of RAM on it and a ~1GB paging
file in Windows.

winxp: 
1. VirtualAlloc(MEM_RESERVE) ->  2,086,912k
2. SetProcessWorkingSetSize(260,000k) + VirtualAlloc(MEM_COMMIT) +
VirtualLock() -> 259,072k
3. VirtualAlloc(MEM_COMMIT) ->  1,011,712k

wine:
1. VirtualAlloc(MEM_RESERVE) ->  1,020,928k
2. SetProcessWorkingSetSize(260,000k) + VirtualAlloc(MEM_COMMIT) +
VirtualLock() -> 1,020,928k
3. VirtualAlloc(MEM_COMMIT) ->  1,020,928k

What I meant by "makes no distinction" is that wine always lets you
MEM_RESERVE or MEM_COMMIT 1,020,928k no matter what. 

In winxp, 
* VirtualAlloc(MEM_COMMIT) seems bounded by the size of the paging file
* VirtualAlloc(MEM_RESERVE) seems just to be 2GB
* VirtualLock() is bounded by GetProcessWorkingSet
* GetProcessWorkingSet is bounded by how much RAM you have

> > * Wine has no implementation of Windows' process working sets
> Linux doesn't either. At least I don't know of a documented API that
> could 
> reliably implement semantics of Windows working sets. Maybe it could be 
> implemented on other OSes like some BSDs if they have such APIs. Maybe 
> there's something in /proc/sys/vm or somesuch to tweak those, but then
> it 
> would probably be system-wide. I'm no expert here I'm afraid :(
I was wondering about getrtlimit/setrlimit. The upper bound of the
working set size seems to map well to RLIMIT_MEMLOCK. However I can't
see what the parallel would be for the lower bound.

Also RLIMIT_AS could be the limit for how much memory you can
MEM_RESERVE. 

These are process based values. They are available in Linux. Not sure
about other platforms. 
 
> > * Wine limits MEM_RESERVE to 1GB, but WinXP goes up to 2GB
> Wine? Maybe linux, but I doubt wine would do anything like that.
Yep, it does. Give it a try. As far as I can tell:

int total = 0;
for (;;) {
  LPVOID* leak = ::VirtualAlloc(NULL, 1048576, 
    MEM_RESERVE, PROT_NOACCESS);
  if (leak) {
    total += 1048576/1024;
    printf("Allocated %ldk\r", total);
  }
  else {
    printf("\n");
    PrintWindowsError("VirtualAlloc", ::GetLastError();
    break;
  }
}

will stop at 1,022,976k (1GB) on any wine machine. In winxp it goes up
to 2GB, as it ought to. 
 
> > Further, these DLLs are audio processing plugins. The apparent fact
> that
> > VirtualLock doesn't _actually_ lock memory into RAM for real time
> I didn't look into the code, but anyway a prerequisite for it would be
> to set 
> windows version to NT/2000/XP. Or at least wine should behave like that,
> assuming that MSDN doesn't lie here.
> 
> Apart from that, I bet you didn't raise ulimits for your processes. On
> my FC3 
> system, ulimit -l gives a meager 32 pages. So that's how many mlock()
> could 
> lock anyway.
There's not a problem there. My ulimit -l is "unlimited". 

BTW we are using an old kernel which required a hack to mlock to allow
us to lock more than half the ram (regardless of the ulimit), but that
doesn't change any of this behavior.
 
Thanks for your feedback... mo