Possibly crazy idea to deal with memory layout problems once and for all

[Troy Rollo]

On Saturday 18 February 2006 22:47, Mike McCormack wrote:

> It has a few problems though.  Firstly, we'd miss mmaps done with system
> calls.

Yes, but how many apps actually do this?

> Secondly, we'd have to make assumptions about what areas of 
> memory the kernel would let us map, and what areas of memory were
> already used in the address map.

It is possible to discover this by attempting to map all areas. It would not 
need to be done page-by-page because we could use something akin to a binary 
search (actually if we really wanted to be rude about it we could grovel 
at /proc/self/maps, which would work (absent a debugger), but is non-portable 
- of course this whole technique is non-portable).

If the system call fails because part of the address space it was attempting 
to allocate was already used, the same technique could be used to discover 
what was mapped and mark that space as temporarily "poisoned" (if we later 
run out of address space, or perhaps at the time we free neighbouring 
addresses, we could attempt to reclaim it).

> I had a go at creating a kernel based PE loader for Linux 2.6 by forward
> porting parts of David Howell's Wine kernel module.  It currently
> compiles, but that's about all.

Aren't kernel modules considered a last resort? Would it even win anything 
given that we still call native libraries which are then likely to call 
libc's mmap (directly or indirectly)?