WIDL and Windows rpcrt4

[Rob Shearman]

2009/1/7 Ge van Geldorp <ge at gse.nl>:
> Hi Rob,
>
> Is WIDL supposed to generate code that's compatible with Windows RPCRT4.DLL?
> Because it appears this is currently not the case. When you look at
> http://test.winehq.org you'll see that the rpcrt4:server test fails on
> pretty much any Windows version.
>
> I investigated a bit more, the failures are caused by an exception 0x6e6
> ("An internal error occurred in a remote procedure call (RPC)" thrown from
> NdrPointerBufferSize() in server_c.c function get_name(). The
> mingw-crosscompiled executable doesn't handle that exception which causes
> the process to crash. Since this is in WIDL-generated code, I decided to
> take a look at what MIDL generates for the same IDL file.
>
> It turns out that MIDL treats the name_t struct as a simple structure, while
> WIDL treats it as a complex structure. That made comparison a bit difficult.
> Since WIDL never seems to generate simple structures, I tried to convince
> MIDL to generate a complex structure too. After a bit of fiddling around
> (I'm no IDL expert...) I came up with the following type, a mix of name_t
> and sun_t:
>
> typedef struct
> {
>  [string, size_is(size)] char *name;
>  unsigned int size;
>
>  [switch_is(s)] union
>  {
>    [case(SUN_I)] int i;
>    [case(SUN_F1, SUN_F2)] float f;
>    [case(SUN_PI)] int *pi;
>  } u;
>
>  int s;
> } complex_t;
>
> After staring at the WIDL-generated and MIDL-generated code for a while
> (which still differed quite a bit), I noticed something interesting: in the
> type format string, "unsigned int size" gets encoded as FC_ULONG by WIDL,
> but as FC_LONG by MIDL. I then went back to the server_c.c and server_s.c
> files generated by WIDL for the tests and changed the type format strings
> there (replaced the "unsigned int size" encoding from FC_ULONG by FC_LONG).
> With this change, the get_name() test passes :-). It then crashes later on
> with an access violation inside NdrVaryingArrayUnmarshall() in
> get_5numbers(), but let's take the problems one at a time...
>
> So it appears Windows RPCRT4.dll is not happy with FC_ULONG entries within a
> complex structure. I added a bunch of other primitive types to my complex_t
> structure to see how they are treated by MIDL. See attached complex.idl file
> and the MIDL-generated midl_complex_c.c file (also attached WIDL-generated
> widl_complex_c.c for completeness). It appears that MIDL simply drops the
> "unsigned" from any base type, at least within complex structures. "unsigned
> long" gets encoded as FC_LONG, "unsigned short" as FC_SHORT and "unsigned
> small" as FC_SMALL.
>
> Like I said, I'm no IDL expert. Does the stuff above make sense to you? Do
> you think it would be a good idea to change WIDL to generate the same
> encodings as MIDL?

Yes, and in fact Michael Karcher has already sent in a pair of patches
to do this:
http://www.winehq.org/pipermail/wine-patches/2009-January/067032.html
http://www.winehq.org/pipermail/wine-patches/2009-January/067031.html

However, they probably won't apply cleanly after my recent changes.
Michael, do you plan to rebase and resend these? If not, I can do
this.

-- 
Rob Shearman