Workaround for AFS bug

[Francois Gouget]

We got reports that Wine would go into an infinite loop on systems that 
use an AFS file system. Strace logs indicated that the problem happened 
in read_directory_vfat() and implicated VFAT_IOCTL_READDIR_BOTH. So I 
wrote a test app which confirmed that on the user's system:
  * either AFS failed to set d_reclen to 0 to signal the end of the 
directory
  * or AFS did not support the VFAT_IOCTL_READDIR_BOTH ioctl at all but 
returned 0 (success) anyway

So this patch sets d_reclen to 0 before the ioctl() and checks whether 
it's still 0 afterwards.

For reference the user's kernel was an ia32 2.6.8 kernel as found in 
SUSE 9.1 and 9.2. I have checked the 2.4.26, 2.6.8 and 2.6.11.9 kernel 
sources but I have not found how this could happen so I could not 
produce a patch fixing the kernel.

If anyone uses AFS I would be interested if you could run the vfat_ioctl 
test and let me know if it gets into an infinite loop, and which kernel 
you are using.


Changelog:

  * dlls/ntdll/directory.c

    Francois Gouget <fgouget at codeweavers.com>
    On some systems (linux 2.6.8) AFS fails to set d_reclen to 0 or does 
not support VFAT_IOCTL_READDIR_BOTH but returns 0 (success) anyway. So 
set d_reclen to 0 before the ioctl() and check it afterwards to work 
around this bug.

-- 
Francois Gouget
fgouget at codeweavers.com

-------------- next part --------------
A non-text attachment was scrubbed...
Name: vfat_ioctl.c
Type: text/x-csrc
Size: 2181 bytes
Desc: not available
Url : http://www.winehq.org/pipermail/wine-patches/attachments/20050525/441dfc86/vfat_ioctl.c
-------------- next part --------------
Index: dlls/ntdll/directory.c
===================================================================
RCS file: /var/cvs/wine/dlls/ntdll/directory.c,v
retrieving revision 1.28
diff -u -p -r1.28 directory.c
--- dlls/ntdll/directory.c	20 May 2005 19:24:07 -0000	1.28
+++ dlls/ntdll/directory.c	25 May 2005 11:58:03 -0000
@@ -831,8 +831,11 @@ static int read_directory_vfat( int fd, 
     {
         off_t old_pos = lseek( fd, 0, SEEK_CUR );
 
+        /* Set d_reclen to 0 to work around an AFS kernel bug */
+        de[0].d_reclen=0;
         res = ioctl( fd, VFAT_IOCTL_READDIR_BOTH, (long)de );
-        if (res == -1 && errno != ENOENT) return -1;  /* VFAT ioctl probably not supported */
+        if ((res == -1 && errno != ENOENT) || de[0].d_reclen == 0)
+            return -1;  /* VFAT ioctl probably not supported */
 
         while (res != -1)
         {
@@ -859,8 +862,11 @@ static int read_directory_vfat( int fd, 
     }
     else  /* we'll only return full entries, no need to worry about overflow */
     {
+        /* Set d_reclen to 0 to work around an AFS kernel bug */
+        de[0].d_reclen=0;
         res = ioctl( fd, VFAT_IOCTL_READDIR_BOTH, (long)de );
-        if (res == -1 && errno != ENOENT) return -1;  /* VFAT ioctl probably not supported */
+        if ((res == -1 && errno != ENOENT) || de[0].d_reclen == 0)
+            return -1;  /* VFAT ioctl probably not supported */
 
         while (res != -1)
         {
@@ -1161,7 +1167,10 @@ static NTSTATUS find_file_in_dir( char *
         {
             KERNEL_DIRENT de[2];
 
-            if (ioctl( fd, VFAT_IOCTL_READDIR_BOTH, (long)de ) != -1)
+            /* Set d_reclen to 0 to work around an AFS kernel bug */
+            de[0].d_reclen=0;
+            if (ioctl( fd, VFAT_IOCTL_READDIR_BOTH, (long)de ) != -1 &&
+                de[0].d_reclen != 0)
             {
                 unix_name[pos - 1] = '/';
                 for (;;)