[PATCH v2 3/8] vfs: Add O_DENYREAD/WRITE flags support for open syscall

[J. Bruce Fields]

On Thu, Feb 07, 2013 at 06:32:38PM +0400, Pavel Shilovsky wrote:
> 2013/2/7 J. Bruce Fields <bfields at fieldses.org>:
> > On Thu, Feb 07, 2013 at 01:53:46PM +0400, Pavel Shilovsky wrote:
> >> Nothing prevents it. If somebody grabbed a share mode lock on a file
> >> before we call deny_lock_file, we simply close this file and return
> >> -ETXTBSY.
> >
> > But leave the newly-created file there--ugh.
> >
> >> We can't grab it before atomic_open because we don't have an
> >> inode there.
> >
> > If you can get the lock while still holding the directory i_mutex can't
> > you prevent anyone else from looking up the new file until you've gotten
> > the lock?
> >
> 
> Hm..., seems you are right, I missed this part:
> mutex_lock
> lookup_open -> atomic_open -> deny_lock_file
> mutex_unlock
> 
> that means that nobody can open and of course set flock on the newly
> created file (because flock is done through file descriptor). So, it
> should be fine to call flock after f_ops->atomic_open in atomic_open
> function. Thanks.

Whether that works may also depend on how the new dentry is set up?  If
it's hashed before you call flock then I suppose it's already visible to
others.

Not knowing that code as well as I should, I might test by introducing
an artificial delay there and trying to reproduce the race.

--b.