Marshalling tutorial

[Mike Hearn]

On Sun, 2004-07-25 at 11:27 +0200, Ove Kaaven wrote:
> The answer is of course that COM doesn't assume that objects actually
> are thread-safe. Most real-world objects aren't, in fact, for various
> reasons. What these reasons are isn't too important here, though, it's
> just important to realize that the problem of thread-unsafe objects is
> what COM tries hard to solve with its apartment model. There are also
> ways to tell COM that your object is truly thread-safe (namely the
> free-threaded marshaller). In general, no object is truly thread-safe if
> it could potentially use another not so thread-safe object, though, so
> the free-threaded marshaller is less used than you'd think.

Right.

> Perhaps you should spell out "QueryInterface-d", instead of saying
> "QId".

Yeah, I suppose I should be as explicit as possible with this topic.

Actually I found out later that I was wrong anyway. You can't just take
a proxy and QueryInterface it to IRpcProxyBuffer as they have two
physically separate vtables. If you want to control the proxy you have
to use a separate vtable that is only ever returned by the CreateProxy
function and cannot be accessed any other way.

> It depends on the Windows version, I think. Windows 95 and Windows NT 4
> certainly had very different models when I looked. I'm pretty sure the
> Windows 98 version of RPCRT4 was able to dispatch messages directly to
> individual apartments. I'd be surprised if some similar functionality
> was not added to Windows 2000. After all, if an object on machine A
> wanted to use an object on machine B in an apartment C, wouldn't it be
> most efficient if the RPC system knew about apartments and could
> dispatch the message directly to it? And if RPC does know how to
> efficiently dispatch to apartments, why should COM duplicate this
> functionality? There were, however, no unified way to tell RPC about
> them across Windows versions, so in that old patch of mine, I let the
> COM/OLE dlls do the apartment dispatch, but even then, the RPC runtime
> was always involved. After all, it could be quite tricky to tell whether
> the call is merely interthread, without involving the RPC runtime...

Currently OXIDs are just ((PID << 32) | TID) so you could see if it was
inter-thread by examining that. In real DCOM though I don't know how
they do it - presumably by walking the proesses apartment list looking
for that OXID first.

> Yes, it's right, but as you mentioned, typelibs cannot encode all the
> information in the original IDL. InstallShield uses MIDL marshallers for
> the interfaces which cannot be represented perfectly by a type library.
> (You could of course ask why not use MIDL-generated marshallers all the
> way. I believe part of the reason for that is that on appropriate
> Windows versions, the marshallers generated from typelibs will be in a
> form that Microsoft claims will run faster than a big MIDL-generated
> marshaller, due to less RAM usage and CPU cache trashing.)

Right.

> And some related hacks in the typelib marshaller to cover up for the
> fact that the typelibs don't actually represent the IDL perfectly, by
> guessing at the extra information needed to successfully marshal the
> interfaces used by InstallShield.

To elaborate, the hack we're talking about is treating VT_VOID as an
interface pointer.

> All those hacks would no longer be needed after that old patch of mine,
> of course...

Yes but it's very large and none of us have been able to figure out how
to chop it up and submit it in a way that won't cause regressions:
really up until recently I didn't understand DCOM well enough to even
attempt this, I would just have made a big mess of it.

> I don't think you have covered the possibility of manually marshalling
> an interface from one single-threaded apartment to another, either
> (CoMarshalInterThreadInterfaceInStream). This is used to great effect in
> InstallShield to separate the worker thread and the UI thread; DCOM does
> all the hard work of letting the objects in the threads communicate
> easily and conveniently, even while the worker threads work and the UI
> thread remains responsive.

Yes, I did not cover this as I thought the email was long enough as is,
though the basic principles are still the same. Just the RPC transport
differs.

thanks for the review!
-mike