ideas and questions for implementation of MessageMode in Named Pipes

[Luke Kenneth Casson Leighton]

On Thu, Feb 5, 2009 at 6:38 PM, Juan Lang <juan.lang at gmail.com> wrote:
>> and no, you _can't_ do "everything in wineserver", because you _still_
>> need a mechanism to be able to tell the client (ntdll) to "block".
>
> Well, if there are two fds still, but each end is either a client
> (reading or writing) and the other is the wineserver, I believe you
> could.

 that was "the plan"

>  That is, the server would only write complete messages to a
> fd, presumably prepended with the length.

 that would be fine, if it wasn't for one thing: i've seen NT send
_eight megabytes_ of data down a named pipe, on a spoolss service.  i
tried it out, once, with rpcclient.  the limit (on NT 4.0) was around
8 mb of PDUs.

now, what i _can't_ tell you is whether that eight megabyte limit was
in the RPC subsystem (or if it was in the SMB-NP subsystem), with the
MSRPC infrastructure breaking it down into PDU-sized chunks [of
MTU-size e.g. 1500] and i certainly can't tell you whether those PDUs
were put into a single 8mb buffer that was then handed to a single
NtWriteFile() call on a Named Pipe, or if it was done on a per-pdu
basis....

but, ultimately, i would be _very_ wary of getting wineserver to
malloc exactly the amount of memory that it was told was being written
down the pipe.

not least is the issue that any such large mallocs or writes would put
a _significant_ latency onto the response time of other wineserver
calls (which is something i wanted to raise as a issue, separately.
another time)

so, i'm doing it as a loop, and... *thinks*... i like the queue of
"struct fd"s more than i like the double-socket idea, the more i think
about it.


> A reading client would then
> block if there weren't any data to read, and there wouldn't be any
> data until a complete message were written there by wineserver.

 yes.  this would be _very_ nice.

 ... except... what do you do when there is another message being written?

NtWriteFile (puts the complete message into the fd)
NtWriteFile (what do you do now?  you can't block the writer)
NtReadFile (gets the first complete message)
NtReadFile (what do you do now?)

this is why having a queue of "struct fd"s, one per message, could end
up being the only viable solution.

>> which is why i asked if there was a way for wineserver to tell a
>> client wine thread/process to "go to sleep" [and didn't get an
>> answer].
>
> Right.  There isn't any such mechanism, at least as far as I know.

 bugger :)   mrrm... i encountered async_queue - that looks
suspiciously like it's a wake-up mechanism.

>>  _somewhere_ there has to be a "break" between "messages".
>>
>> and you can't stop people from sending data down the pipe (in NtWritePipe).
>>
>> therefore, logically, you have to have a "firewall" between "send" and
>> "receive".  and, because you _also_ need "blocking on read"
>> characteristics (across multiple processes and threads), the most
>> sensible way to implement that is with a unix filedescriptor on which
>> all of the clients (ntdll) can block.
>
> Yep, agreed.  This discussion is more about what the structure of the
> file descriptors is.  But in general, my opinion here doesn't carry
> much weight, Alexandre's does ;-)  And he's unlikely to voice an
> opinion except on patches.

 ha ha :)

 anyone who helps crystallise ideas is valuable.


>>  message-mode namedpipes falls into that category, which is why i'm doing it.
>
> And we appreciate your attention to it.  Really, we do :)

 thanks :)

>>  so - once i have _a_ working implementation, then please do not be
>> offended, or surprised, if, when asked to make further enhancements,
>> or if asked to fit specific criteria (such as doing things a "slightly
>> different way"), i decline to do so.
>
> Yep, that's been my assumption all along.  That's why I've nudged you
> with respect to tests a couple of times.  Thanks for putting some in
> the bug, by the way.

 no problem.  there will need to be more - a lot more.

 e.g. PeekNamedPipe actually does an optional read, and _then_ tells
you what's left.  wine's implementation, because nobody ever really
_does_ that "optional read", they only do the "peek".

>  Since I assumed it was unlikely you'd have the
> patience to work with AJ to get your patches accepted, I thought
> having some test cases, along with an implementation that isn't
> fundamentally flawed, would help point us in the right direction.

 ... well... now i'm... deeply impressed (with your foresight).

> Pointing out that handles can be shared between processes was a hint
> about one fundamental flaw that can trip you up.

 mrmmrmmm *grumble-about-how-nt-works* yeah that was appreciated,
although it won't be until i do some tests using dup on handles, or
simply blatantly passing the 32-bit handle pointer value between
processes (over a socket or a file), that i'll know what is and isn't
possible.

>  I'll try to look at
> what you're doing now and again to point out things that just can't
> work.

 he he.  appreciate it.

> For the most part, though, I'll assume you know what you're
> doing.

 *cackle*.  are you sure that's wise? :)


> Well, this is a bit off-topic, but he's a Linux kernel guy, not really
> part of the Samba team.

 ahh. thank you for clarifying.  i remember stephen from when he was
with ibm, he was always at the cifs conferences.

>  But I understand that you don't want to go
> the Linux-kernel-module-only route, and I respect that.

 *nods*.

 thank you, juan.

 l.