To set video memory size set VideoMemorySize registry value. It should be string and vram size in Megabytes. See this for details: http://wiki.winehq.org/UsefulRegistryKeys