[PATCH 0/4] Server async restructuring

[Zebediah Figura]

Currently the only reliable way for server-side devices to become aware of
canceled asyncs is by checking for STATUS_PENDING in their reselect_async
callback. Named pipes and sockets both use this to free extra data (struct
pipe_message and struct accept_req respectively).

This is a bit architecturally unpleasant, and hard to follow, and caused me some
grief while trying to implement socket ioctls. After some discussion with Jacek
I decided to try to restructure things a bit, so that the socket provides a
callback which will be called as soon as we know the async is complete.

This requires some other changes, though:

* Not only is it possible for an async to outlive its object, but this even
  happens normally (consider e.g. STATUS_HANDLES_CLOSED)—even after the object
  is destroyed, a thread apc still holds a reference to the async. In case the
  completion callback needs to access object information (as with accept_req, to
  remove that structure from a per-socket list), we need to ensure that the
  object is not freed before then. The obvious way to do this is to take a
  reference to the object, i.e. patch 0003.

* However, this causes a circular reference, since we're effectively also taking
  a reference to the async on behalf of the object in alloc_accept_req() [and
  even if that were removed, we'd still be holding a reference to the async on
  behalf of the object via the async queue. We can't remove all such references
  as they are the only things keeping a pending async alive.] In order to break
  this reference loop, we make use of the close_handle callback to terminate all
  asyncs as soon as the last user-space handle is closed, rather than when the
  object is destroyed, i.e. patch 0002. Note that this doesn't mean they'll
  immediately be removed from the list, but they will be as soon as the apc
  object is done with them.

  [As an aside, this also seems to match the way Windows drivers work: according
  to [1] drivers should cancel pending asyncs in IRP_MJ_CLEANUP, and according
  to [2] when IRP_MJ_CLOSE is subsequently sent "all outstanding I/O requests
  have been completed or canceled."]

* Note that we can't easily avoid taking a reference to the socket object from
  the async. The earliest possible place we can call the async callback is in
  async_set_result(), once we're sure that the async is done and its results
  can't be altered. We could call it in async_terminate(), and make sure that
  async_terminate() is always called on object destruction, but there is a
  slight snag in that the client is technically allowed to restart any async.
  That won't actually happen in practice, for any of the asyncs I've seen or
  have planned, but it's an awful lot of implicit assumptions to remember.

As an added complication, there is an awful socket ioctl, viz. the WSAPoll()
ioctl, which can outlive the object it's called on. Neither of the applications
I've seen that use the ioctl depend on that behaviour, but this restructuring
does actually help make it possible to implement without reference leaks.

[1] https://docs.microsoft.com/en-us/windows-hardware/drivers/kernel/irp-mj-cleanup
[2] https://docs.microsoft.com/en-us/windows-hardware/drivers/kernel/irp-mj-close

Zebediah Figura (4):
  server: Hold a reference to the iosb in the accept_req structure.
  server: Terminate accept asyncs when the last socket handle is closed.
  server: Hold a reference to both sockets in the accept_req structure.
  server: Use a callback to free the accept_req structure.

 server/async.c | 15 ++++++++
 server/file.h  |  4 +++
 server/sock.c  | 98 ++++++++++++++++++++++++++------------------------
 3 files changed, 71 insertions(+), 46 deletions(-)

-- 
2.20.1