RESEND: Enable GCOV Code Coverage

Aaron Arvey aarvey at cs.hmc.edu
Tue Feb 1 01:22:51 CST 2005


This is the third or fourth resend... anything I'm unaware of?



We're using gcov and lcov to measure how well the Wine test suite
covers the wine source tree.  Here's our first cut at making it easy
to run wine compiled for coverage testing, and view the results
using the simple text interface of gcov.  A seperate patch for
lcov will be released soon.

Changelog:
  * configure.ac
    new option "--enable-gcov"
  * Make.rules.in
    new variable LDEXTRACFLAGS (for loader EXTRACFLAGS)
  * loader/Makefile.in
    used LDEXTRACFLAGS in place of EXTRAFLAGS
    (low level compilation issues when profiling code)



Index: Make.rules.in
===================================================================
RCS file: /home/wine/wine/Make.rules.in,v
retrieving revision 1.179
diff -u -u -r1.179 Make.rules.in
--- Make.rules.in	7 Oct 2004 03:12:44 -0000	1.179
+++ Make.rules.in	9 Nov 2004 05:18:32 -0000
@@ -104,8 +104,9 @@
 api_manext      = 3w
 conf_manext     = 5
 CLEAN_FILES     = *.o *.a *.so *.ln *.$(LIBEXT) \\\#*\\\# *~ *% .\\\#* *.bak *.orig *.rej \
-                  *.flc *.spec.c *.spec.def *.dbg.c *.tab.c *.tab.h @LEX_OUTPUT_ROOT at .c core
+                  *.flc *.spec.c *.spec.def *.dbg.c *.tab.c *.tab.h @LEX_OUTPUT_ROOT at .c core \
+                  *.bb *.bbg *.da

 OBJS = $(C_SRCS:.c=.o) $(EXTRA_OBJS)


Index: configure.ac
===================================================================
RCS file: /home/wine/wine/configure.ac,v
retrieving revision 1.324
diff -u -u -r1.324 configure.ac
--- configure.ac	4 Nov 2004 21:15:33 -0000	1.324
+++ configure.ac	9 Nov 2004 05:18:32 -0000
@@ -16,6 +16,7 @@
 AC_ARG_ENABLE(debug, AC_HELP_STRING([--disable-debug],[compile out all debugging messages]))
 AC_ARG_ENABLE(trace, AC_HELP_STRING([--disable-trace],[compile out TRACE messages]))
 AC_ARG_ENABLE(win64, AC_HELP_STRING([--enable-win64], [build a Win64 emulator on AMD64 (won't run Win32 binaries)]))
+AC_ARG_ENABLE(gcov, AC_HELP_STRING([--enable-gcov],[turn on code coverage analysis tools]))

 AC_ARG_WITH(opengl,    AC_HELP_STRING([--without-opengl],[do not use OpenGL]))
 AC_ARG_WITH(curses,    AC_HELP_STRING([--without-curses],[do not use curses]))
@@ -707,6 +708,7 @@
 dnl **** Check for gcc specific options ****

 AC_SUBST(EXTRACFLAGS,"")
+AC_SUBST(LOADEREXTRACFLAGS,"")
 if test "x${GCC}" = "xyes"
 then
   EXTRACFLAGS="-Wall -pipe"
@@ -768,6 +770,31 @@
     EXTRACFLAGS="$EXTRACFLAGS -gstabs+"
   fi

+  dnl Check for --enable-gcov and add appropriate flags for gcc
+  dnl Note that these extra switches are NOT applied to the loader
+  if test "x$enable_gcov" = "xyes";
+  then
+    dnl Check for -fprofile-arcs and -ftest-coverage option
+    AC_CACHE_CHECK([for gcc -fprofile-arcs support],
+		    ac_cv_c_gcc_fprofile_arcs,
+                    [WINE_TRY_CFLAGS([-fprofile-arcs],
+		                     ac_cv_c_gcc_fprofile_arcs="yes",
+				     ac_cv_c_gcc_fprofile_arcs="no")])
+    AC_CACHE_CHECK([for gcc -ftest-coverage support],
+		    ac_cv_c_gcc_ftest_coverage,
+                    [WINE_TRY_CFLAGS([-ftest-coverage],
+		                     ac_cv_c_gcc_ftest_coverage="yes",
+				     ac_cv_c_gcc_ftest_coverage="no")])
+    if test "$ac_cv_c_gcc_fprofile_arcs" = "yes" && \
+       test "$ac_cv_c_gcc_ftest_coverage" = "yes"
+    then
+      EXTRACFLAGS="$EXTRACFLAGS -fprofile-arcs -ftest-coverage"
+      dnl Turn off optimization so code coverage tool
+      dnl can get accurate line numbers
+      EXTRACFLAGS=`echo "$EXTRACFLAGS" | sed -e 's/-O[0-9]*//g'`
+    fi
+  fi
+
   dnl Check for noisy string.h
   saved_CFLAGS="$CFLAGS"
   CFLAGS="$CFLAGS -Wpointer-arith -Werror"
@@ -781,6 +808,11 @@
   fi
 fi

+dnl When adding in more 'EXTRAFLAGS' you must put them above this!
+dnl Strips out the code profiling abilities because the preloader
+dnl relies on low level assumptions which added code messes up.
+LOADEREXTRACFLAGS=`echo "$EXTRACFLAGS" | sed -e 's/-fprofile-arcs -ftest-coverage//g'`
+
 dnl **** Check how to define a function in assembly code ****

 AC_CACHE_CHECK([how to define a function in assembly code], ac_cv_asm_func_def,
Index: documentation/winedev-otherdebug.sgml
===================================================================
RCS file: /home/wine/wine/documentation/winedev-otherdebug.sgml,v
retrieving revision 1.1
diff -u -u -r1.1 winedev-otherdebug.sgml
--- documentation/winedev-otherdebug.sgml	26 Oct 2004 22:45:47 -0000	1.1
+++ documentation/winedev-otherdebug.sgml	9 Nov 2004 05:18:32 -0000
@@ -497,6 +497,179 @@
 	</listitem>
       </orderedlist>
     </sect1>
+    <sect1>
+      <title>Which code has been tested?</title>
+      <para>
+        Deciding what code should be tested next can be a difficult
+        decision.  And in any given project, there is always code that
+        isn't tested where bugs could be lurking.  This section goes
+        over how to identify these sections using a tool called gcov.
+      </para>
+      <para>
+        To use gcov on wine, do the following:
+      </para>
+      <orderedlist>
+        <listitem>
+         <para>
+           In order to activate code coverage in the wine source
+           code, use the <literal>--enable-gcov</literal> flag when
+           running <command>configure</command>.
+         </para>
+       </listitem>
+        <listitem>
+         <para>
+           Run any application or test suite.
+         </para>
+       </listitem>
+        <listitem>
+         <para>
+           Run gcov on the file which you would like to know more
+           about code coverage.
+         </para>
+       </listitem>
+      </orderedlist>
+      <para>
+        The following is an example situation when using gcov to
+        determine the coverage of a file could be helpful.  We'll use
+        the <filename>dlls/lzexpand/lzexpand_main.c.</filename> file.
+        At one time the code in this file was not fully tested (as it
+        may still be).  For example at the time of this writing, the
+        function <function>LZOpenFileA</function> had the following
+        lines in it:
+        <screen>
+if ((mode&~0x70)!=OF_READ)
+        return fd;
+if (fd==HFILE_ERROR)
+        return HFILE_ERROR;
+cfd=LZInit(fd);
+if ((INT)cfd <= 0) return fd;
+return cfd;
+        </screen>
+        Currently there are a few tests written to test this function;
+        however, these tests don't check that everything is correct.
+        For instance, <constant>HFILE_ERROR</constant> may be the wrong
+        error code to return.  Using gcov and directed tests, we can
+        validate the correctness of this line of code.  First, we see
+        what has been tested already by running gcov on the file.
+        To do this, do the following:
+        <screen>
+cvs checkout wine
+mkdir build
+cd build
+../wine/configure --enable-gcov
+make depend && make
+cd dlls/lxexpand/tests
+make test
+cd ..
+gcov ../../../wine/dlls/lzexpand/lzexpand_main.c
+  0.00% of 3 lines executed in file ../../../wine/include/wine/unicode.h
+  Creating unicode.h.gcov.
+  0.00% of 4 lines executed in file /usr/include/ctype.h
+  Creating ctype.h.gcov.
+  0.00% of 6 lines executed in file /usr/include/bits/string2.h
+  Creating string2.h.gcov.
+  100.00% of 3 lines executed in file ../../../wine/include/winbase.h
+  Creating winbase.h.gcov.
+  50.83% of 240 lines executed in file ../../../wine/dlls/lzexpand/lzexpand_main.c
+  Creating lzexpand_main.c.gcov.
+less lzexpand_main.c.gcov
+        </screen>
+        Note that there is more output, but only output of gcov is
+        shown.  The output file
+        <filename>lzexpand_main.c.gcov</filename> looks like this.
+        <screen>
+        9:  545:        if ((mode&~0x70)!=OF_READ)
+        6:  546:                return fd;
+        3:  547:        if (fd==HFILE_ERROR)
+    #####:  548:                return HFILE_ERROR;
+        3:  549:        cfd=LZInit(fd);
+        3:  550:        if ((INT)cfd <= 0) return fd;
+        3:  551:        return cfd;
+        </screen>
+        <command>gcov</command> output consists of three components:
+        the number of times a line was run, the line number, and the
+        actual text of the line.  Note: If a line is optimized out by
+        the compiler, it will appear as if it was never run.  The line
+        of code which returns <constant>HFILE_ERROR</constant> is
+        never executed (and it is highly unlikely that it is optimized
+        out), so we don't know if it is correct.  In order to validate
+        this line, there are two parts of this process.  First we must
+        write the test.  Please see <xref linkend="testing"> to
+        learn more about writing tests.  We insert the following lines
+        into a test case:
+        <screen>
+INT file;
+
+/* Check for non-existent file. */
+file = LZOpenFile("badfilename_", &amp;test, OF_READ);
+ok(file == LZERROR_BADINHANDLE,
+   "LZOpenFile succeeded on nonexistent file\n");
+LZClose(file);
+       </screen>
+       Once we add in this test case, we now want to know if the line
+       in question is run by this test and works as expected.  You
+       should be in the same directory as you left off in the previous
+       command example.  The only difference is that we have to remove
+       the <filename>*.da</filename> files in order to start the
+       count over (if we leave the files than the number of times the
+       line is run is just added, e.g. line 545 below would be run 19 times)
+       and we remove the <filename>*.gcov</filename> files because
+       they are out of date and need to be recreated.
+      </para>
+      <screen>
+rm *.da *.gcov
+cd tests
+make
+make test
+cd ..
+gcov ../../../wine/dlls/lzexpand/lzexpand_main.c
+  0.00% of 3 lines executed in file ../../../wine/include/wine/unicode.h
+  Creating unicode.h.gcov.
+  0.00% of 4 lines executed in file /usr/include/ctype.h
+  Creating ctype.h.gcov.
+  0.00% of 6 lines executed in file /usr/include/bits/string2.h
+  Creating string2.h.gcov.
+  100.00% of 3 lines executed in file ../../../wine/include/winbase.h
+  Creating winbase.h.gcov.
+  51.67% of 240 lines executed in file ../../../wine/dlls/lzexpand/lzexpand_main.c
+  Creating lzexpand_main.c.gcov.
+less lzexpand_main.c.gcov
+      </screen>
+      <para>
+        Note that there is more output, but only output of gcov is
+        shown.  The output file
+        <filename>lzexpand_main.c.gcov</filename> looks like this.
+      </para>
+      <screen>
+       10:  545:        if ((mode&~0x70)!=OF_READ)
+        6:  546:                return fd;
+        4:  547:        if (fd==HFILE_ERROR)
+        1:  548:                return HFILE_ERROR;
+        3:  549:        cfd=LZInit(fd);
+        3:  550:        if ((INT)cfd <= 0) return fd;
+        3:  551:        return cfd;
+      </screen>
+      <para>
+        Based on gcov, we now know that
+        <constant>HFILE_ERROR</constant> is returned once.  And since
+        all of our other tests have remain unchanged, we can assume
+        that the one time it is returned is to satisfy the one case we
+        added where we check for it.  Thus we have validated a line of
+        code.  While this is a cursory example, it demostrates the
+        potential usefulness of this tool.
+      </para>
+      <para>
+        For a further in depth description of gcov, the official gcc
+        compiler suite page for gcov is <ulink
+        url="http://gcc.gnu.org/onlinedocs/gcc-3.2.3/gcc/Gcov.html">
+        http://gcc.gnu.org/onlinedocs/gcc-3.2.3/gcc/Gcov.html</ulink>.
+        There is also an excellent article written by Steve Best for
+        Linux Magazine which describes and illustrates this process
+        very well at
+        <ulink url="http://www.linux-mag.com/2003-07/compile_01.html">
+        http://www.linux-mag.com/2003-07/compile_01.html</ulink>.
+      </para>
+    </sect1>

   </chapter>

Index: loader/Makefile.in
===================================================================
RCS file: /home/wine/wine/loader/Makefile.in,v
retrieving revision 1.17
diff -u -u -r1.17 Makefile.in
--- loader/Makefile.in	11 Aug 2004 20:59:09 -0000	1.17
+++ loader/Makefile.in	9 Nov 2004 05:18:32 -0000
@@ -23,6 +23,10 @@

 LIBPTHREAD  = @LIBPTHREAD@
 LDEXECFLAGS = @LDEXECFLAGS@
+LOADEREXTRACFLAGS = @LOADEREXTRACFLAGS@
+
+.c.o:
+	$(CC) -c $(INCLUDES) $(DEFS) $(DLLFLAGS) $(LOADEREXTRACFLAGS) $(CPPFLAGS) $(CFLAGS) -o $@ $<

 wine-glibc: glibc.o Makefile.in
 	$(CC) -o $@ glibc.o $(LIBWINE) $(LIBPORT) $(LIBPTHREAD) $(EXTRALIBS) $(LDFLAGS)



More information about the wine-patches mailing list