msi:RECORD_StreamFromFile bug, help needed

[robert.van.herk at serioustoys.com]

Hi all,

I think I found a msi.dll, record.c, RECORD_StreamFromFile.
This function calls GlobalAlloc, with a file size as an argument. However, I believe that this can easily fail (which it does in my case) when the file size is too large.

In my case, my msi builder (WiX) calls MsiRecordSetStreamW, which calls static function RECORD_StreamFromFile.
WiX calls MsiRecordSetStreamW on some file that seems to be 0x11b2acc5 bytes (296 megs) large. Trying to allocate the block of 296 megs causes the error:
 
036:Call msi.MsiRecordSetStreamW(00000003,00000002,0cc8078c L"C:\\users\\robert\\Temp\\3c5c43d5\\1bfe39fa\\#product.cab") ret=009a4c3a
0036:Call KERNEL32.CreateFileW(0cc8078c L"C:\\users\\robert\\Temp\\3c5c43d5\\1bfe39fa\\#product.cab",80000000,00000001,00000000,00000003,00000000,00000000) ret=202868cf
0036:Ret  KERNEL32.CreateFileW() retval=00000100 ret=202868cf
0036:Call KERNEL32.GetFileSize(00000100,0033ef58) ret=202868ef
0036:Ret  KERNEL32.GetFileSize() retval=11b2acc5 ret=202868ef
0036:Call KERNEL32.GlobalAlloc(00000000,11b2acc5) ret=202869fb
0036:Ret  KERNEL32.GlobalAlloc() retval=00000000 ret=202869fb         <--- fail

I believe the call to GlobalAlloc stems from RECORD_StreamFromFile. This tries to copy the full contents of the file in memory, and then create an IStream from that memory using CreateStreamOnHGlobal:

/* read the data in a file into an IStream */
static UINT RECORD_StreamFromFile(LPCWSTR szFile, IStream **pstm)
{
    ...
    handle = CreateFileW(szFile, GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, 0, NULL);
    ...
    sz = GetFileSize(handle, &szHighWord);
    if( sz != INVALID_FILE_SIZE && szHig    sz = GetFileSize(handle, &szHighWord);
    if( sz != INVALID_FILE_SIZE && szHighWord == 0 )
    {
        hGlob = GlobalAlloc(GMEM_FIXED, sz);        // <-- Fails in my case, because sz == 296 megs
        if( hGlob )
        {
            BOOL r = ReadFile(handle, hGlob, sz, &read, NULL);
            if( !r )
            {
                GlobalFree(hGlob);
                hGlob = 0;
            }
        }
    CloseHandle(handle);
    if( !hGlob )
        return ERROR_FUNCTION_FAILED;          // <-- hence, this error is raised.
    
    hr = CreateStreamOnHGlobal(hGlob, TRUE, pstm);
   ...
}

I want to fix this.

I guess that apart from that the allocation fails, I have the feeling that copying this entire 296 megs into main memory is anyway a bit heavy, just for creating a stream.
So I thought a better solution would be just to create a stream from the file. My first guess would be to use the SHCreateStreamOnFile function:

HRESULT SHCreateStreamOnFile(
  __in   LPCTSTR pszFile,
  __in   DWORD grfMode,
  __out  IStream **ppstm
);

But then I was thinking: this seems like such an obvious thing to do, that it is almost suspicious. Was there any reason for copying the file contents to main memory and then create a memory stream?

For instance, was the intention to have a non-mutable copy in memory, in case the backing file would be altered later on?

I guess if that's the case, I could just create a copy of the original file in a temp dir, and mark it as STGM_DELETEONRELEASE.

Before I wildly attempt a fix though, I would appreciate it if someone had an idea here...

Kind regards,
Robert van Herk