Marshalling tutorial

[Ove Kaaven]

ons, 14.07.2004 kl. 18.07 skrev Mike Hearn:
> You might be wondering about putting threads next to processes and
> machines in that last paragraph. You can access thread safe objects from
> multiple threads without DCOM normally, right? Why would you need RPC
> magic to do that?

The answer is of course that COM doesn't assume that objects actually
are thread-safe. Most real-world objects aren't, in fact, for various
reasons. What these reasons are isn't too important here, though, it's
just important to realize that the problem of thread-unsafe objects is
what COM tries hard to solve with its apartment model. There are also
ways to tell COM that your object is truly thread-safe (namely the
free-threaded marshaller). In general, no object is truly thread-safe if
it could potentially use another not so thread-safe object, though, so
the free-threaded marshaller is less used than you'd think.

> The backbone of DCOM is this RPC runtime, which is an implementation of
> DCE RPC [1]. DCE RPC is not naturally object oriented, so this protocol
> is extended with some new constructs and by assigning new meanings to
> some of the packet fields, to produce ORPC or Object RPC. You might see
> it called MS-RPC as well.

I think MS-RPC refers to MS's implementation of DCE RPC (i.e.
RPCRT4.dll), and possibly their API and extended IDL syntax. This is not
the same concept as ORPC. You can have MS-RPC without ORPC, and, with
some effort and wizardry, vice versa. It's just that the two work best
together, as that's what they're designed to do.

> COM PROXY/STUB SYSTEM
> 
> COM proxies are objects that implement both the interfaces needing to be
> proxied and also IRpcProxyBuffer. Likewise, COM stubs implement
> IRpcStubBuffer and understand how to invoke the methods of the requested
> interface.
> 
> You may be wondering what the word "buffer" is doing in those interface
> names. I'm not sure either, except that a running theme in DCOM is that
> interfaces which have nothing to do with buffers have the word Buffer
> appended to them, seemingly at random. Ignore it and *don't let it
> confuse you* :) This stuff is convoluted enough ...

The primary functionality of objects exposing IRpcProxyBuffer and
IRpcStubBuffer does involve marshalling into and unmarshalling from a
RPC buffer, if that helps. Their primary methods also get passed
IRpcChannelBuffer instances, which allows them to transmit and receive
RPC buffers. So they have *something* to do with buffers, at least, even
though it's not in the way the interface names suggest.

> Once CreateProxy has been called, the resultant object is QId to
> IRpcProxyBuffer, which only has 1 method, IRpcProxyBuffer::Connect [4]
> This method only takes one parameter, the IRpcChannelBuffer object which
> encapsulates the "RPC Channel" between the client and server.

Perhaps you should spell out "QueryInterface-d", instead of saying
"QId".

> RPC CHANNELS
> 
> Remember the RPC runtime? Well, that's not just responsible for
> marshalling stuff, it also controls the connection and protocols between
> the client and server. We can ignore the details of this for now,
> suffice it to say that an RPC Channel is a COM object that implements
> IRpcChannelBuffer, and it's basically an abstraction of different RPC
> methods. For instance, in the case of inter-thread marshalling (not
> covered here) the RPC connection code isn't used, only the NDR
> marshallers are, so IRpcChannelBuffer in that case isn't actually
> implemented by RPCRT4 but rather just by the COM/OLE DLLS (fixme: is
> this actually correct?).

It depends on the Windows version, I think. Windows 95 and Windows NT 4
certainly had very different models when I looked. I'm pretty sure the
Windows 98 version of RPCRT4 was able to dispatch messages directly to
individual apartments. I'd be surprised if some similar functionality
was not added to Windows 2000. After all, if an object on machine A
wanted to use an object on machine B in an apartment C, wouldn't it be
most efficient if the RPC system knew about apartments and could
dispatch the message directly to it? And if RPC does know how to
efficiently dispatch to apartments, why should COM duplicate this
functionality? There were, however, no unified way to tell RPC about
them across Windows versions, so in that old patch of mine, I let the
COM/OLE dlls do the apartment dispatch, but even then, the RPC runtime
was always involved. After all, it could be quite tricky to tell whether
the call is merely interthread, without involving the RPC runtime...

> In the case of InstallShield, it actually comes with typelibs for all
> the interfaces it needs to marshal (fixme: is this right?), but they
> actually use a mix of MIDL and typelib marshalling.

Yes, it's right, but as you mentioned, typelibs cannot encode all the
information in the original IDL. InstallShield uses MIDL marshallers for
the interfaces which cannot be represented perfectly by a type library.
(You could of course ask why not use MIDL-generated marshallers all the
way. I believe part of the reason for that is that on appropriate
Windows versions, the marshallers generated from typelibs will be in a
form that Microsoft claims will run faster than a big MIDL-generated
marshaller, due to less RAM usage and CPU cache trashing.)

> In order to cover up
> for the fact that we don't really use RPC they're all force to go via
> the typelib marshaller - that's what the 1 || hack is for and what the
> "Registering non-automation type library!" warning is about (I think).

And some related hacks in the typelib marshaller to cover up for the
fact that the typelibs don't actually represent the IDL perfectly, by
guessing at the extra information needed to successfully marshal the
interfaces used by InstallShield.

All those hacks would no longer be needed after that old patch of mine,
of course...

> WRAPUP
> 
> OK, so there are some (very) basic notes on DCOM. There's a ton of stuff
> I have not covered:

I don't think you have covered the possibility of manually marshalling
an interface from one single-threaded apartment to another, either
(CoMarshalInterThreadInterfaceInStream). This is used to great effect in
InstallShield to separate the worker thread and the UI thread; DCOM does
all the hard work of letting the objects in the threads communicate
easily and conveniently, even while the worker threads work and the UI
thread remains responsive.