PROT_EXEC mmap/mprotect, i386 PAE + NX broken, x86-64 2.6.17-rc2

[Alistair John Strachan]

On Sunday 23 April 2006 16:35, Jesse Allen wrote:
> On 4/22/06, Alistair John Strachan <s0348365 at sms.ed.ac.uk> wrote:
> > On Saturday 22 April 2006 18:38, Jesse Allen wrote:
> > [snip]
> >
> > > It has MEM_EXECUTE correctly set. I think that loader should be
> > > considered buggy.
> >
> > This executable works on Windows, therefore it should work in Wine. It's
> > really that simple. If Windows doesn't enforce DEP/NX, neither should
> > Wine. Or, at the very least, it should be configurable.
>
> What about x86-64 versions of Windows? I think it allows you to turn
> it on and off, but we should find out. I still think the modified
> executable is buggy, but if Windows does it, we could do it too.

My *guess* is that Windows x64 Edition will enforce DEP/NX for 64bit 
applications, but will do the same as XP SP2 for 32bit applications. That is, 
for 32bit applications, you can choose to enforce DEP/NX, and "whitelist" 
applications (selectively disabling DEP/NX), or have _only_ Windows component 
DLLs secured by DEP/NX (the default).

Linux, until 2.6.17-rc, also did this. Andi Kleen suggested on LKML that there 
are userspace tools for Linux which allow NX to be disabled per-binary at 
runtime, but I suspect such utilities would require privileges. It would be 
suboptimal to mandate their use with Wine.

-- 
Cheers,
Alistair.

Third year Computer Science undergraduate.
1F2 55 South Clerk Street, Edinburgh, UK.