Re: problems with gdb

[Christopher Blizzard <blizzard at redhat dot com>]

Mark Kettenis wrote:
> 
>    Date: Fri, 11 Feb 2000 16:26:33 -0500
>    From: Chris Blizzard <blizzard@mozilla.org>
> 
>    Hi, folks.  I've been talking about some problems that I've been
>    suffering through with gdb and mozilla which people on this mailing
>    list may or may not be aware of.  Jason Molenda suggested that I
>    start flushing these out in the open to get some feedback on them.
> 
>    I'm interested in getting my hands dirty and try to get these
>    problems fixed.  I'm not a debugger hacker though so I might end up
>    asking some silly questions.  :)
> 
> No problem!
> 
> May I ask some questions first?  What version of GDB are you using?
> What version of GCC are you using?
> 

This is the gdb that is distributed with the Red Hat 6.2 Beta.  I'm
using this version of gcc:

Reading specs from /usr/lib/gcc-lib/i386-redhat-linux/egcs-2.91.66/specs
gcc version egcs-2.91.66 19990314/Linux (egcs-1.1.2 release)


>    Here's the blurb, slightly edited for content.
> 
>    ...My problems are mostly related to how well gdb scales to handle
>    large shared libraries and large numbers of shared libraries.  At
>    last count, there were 111 .so files in mozilla, the largest of
>    which is about 27 meg with debugging symbols.  If you don't use
>    "set auto-solib-add 0" in your .gdbinit file, gdb will easily grow
>    to over 200 meg in size when starting the debugger.  Someone once
>    did some estimates and it seems to use 5 times the size of a .so
>    after loading a shared library to debug.  A lot of times, gdb won't
>    be able to load some of the larger .so files.  It just hangs.
> 
> Let me first say that Mozilla seems to stretch things to the limit.
> The huge number of shared libraries that you guys are using have
> already uncovered several bugs in the dynamic linker and the Mozilla
> developers have uncovered more than a few bugs in the LinuxThreads
> library.  That's mostly because you have gone where no one's gone
> before :-)
> 

Hey, we're trying.  :)  Someone pointed out to me that the problem
didn't start happening until after I upgraded to 6.1 -> 6.2 beta.  So,
it could be any of the components on the system.

> >From a quick glance at the output of `ps' on my system when loading a
> program that uses about 10 shared libraries it seems that
> the GDB memory usage is aproximately equal to the size of the shraed
> libraries on disk.  I guess the "factor 5" estimate, is referring to
> the space used for debugging symbols as compared to the actual
> code-size of the shared library.  So it seems that your biggest
> problem is the size of your shared libraries and the amount of
> debugging information that's generated (which is basically
> proportional to the amount of code in the libraries).  I think that
> using C++ is in a large way responsible for the `code bloat'.  Maybe
> an intelligent use of C++ features (check for compiler switches like
> -fno-rtti and use them if appropriate) can reduce the size of the
> resulting code.  Also playing around with the options that control the
> way debugging information is generated might help.
> 

Here's how I'm compiling now:

c++ -o nsLocalFileUnix.o -c -fno-rtti -fno-exceptions -Wall -Wconversion
-Wpointer-arith -Wbad-function-cast -Wcast-align -Woverloaded-virtual
-Wsynth -Wshadow -pedantic -Wno-long-long -pthread -g -DMOZILLA_CLIENT
-include ../../config-defs.h -Wp,-MD,.deps/nsLocalFileUnix.pp
-DOSTYPE=\"Linux2.2.15-2.5\" -DOJI -D_IMPL_NS_COM -D_IMPL_NS_BASE
-D_BSD_SOURCE  -I../../dist/include -I../../include    
-I/usr/X11R6/include   -fPIC nsLocalFileUnix.cpp

We're not compiling with optimization and we're reducing size as much as
possible.

> In principle the large amounts of debugging info shouldn't be a
> problem.  GDB can simply mmap the relevant sections, such that only
> the debugging info that's really needed is actually pages in.  I don't
> know how the BFD library (the part of GDB that is responsible for
> reading the sections containing debugging info) and the code that that
> actually interprets this information implements these things.  There
> might be room for improvement there.  Of course if all pages
> containing debugging info are touched, you lose :-(.
> 
>    A lot of times, trying to use "step" to step into a c++ method that
>    happens to be part of the same class just skips as if you had used
>    "next."  That means that any time you want to step into a method
>    you have to set a temporary breakpoint by name on the method and
>    then allow the breakpoint to get you into that method.  Doing that
>    to step into a dozen or so classes gets a little tedious.  This is
>    hard to reproduce and I'm trying to build a test case.
> 
> It is a known problem that GDB has problems with the debugging output
> generated by recent GCC compilers.  Help in resolving those problems
> would certainly be appreciated, and a (small) test case is really
> essential if you want to get somebody else to look into it.
> 
> Compiling without optimization might circumvent these problems.
> 
>    There are other much needed features, like not being able to
>    preload a .so and setting a breakpoint in the library before it
>    loads.  Mozilla is entirely component based and this makes
>    debugging very, very difficult.  I usually break on _dl_open in
>    glibc and wait until my library gets loaded before trying to set
>    the breakpoint that I need.  That gets pretty bad after 27
>    libraries are loaded.
> 
> I think that the way GDB looks up symbols is differs from the way
> the dynamic linker does that.  That means that overriding symbols in
> shared libraries probably doesn't work properly.  Since the primary
> use of preloaded shared libraries is overriding symbols you're likely
> to experience problems.  I don't think this problem is easy to solve.
> 
> Setting breakpoints in not-yet loaded shared libraries should not be
> difficult to implement.  Just make sure they start out as
> `shlib_disabled' (see breakpoint.h) if the symbol cannot be found.  It
> is necessary to introduce a new command to do this (suggested name
> `shlib-break' or `solib-break').  Reusing the guts of the ordinary
> breakpoint setting command should be possible.
> 
>    There are also various problems with threads.  A lot of times gdb
>    won't exit after the last thread exits because it keeps trying to
>    kill a process which doesn't exist any more.
> 
> Probably caused by the strange way threads interact with signals on
> Linux.  It's very likely that the real bug is in the LinuxThreads
> library and not in GDB.  A lot of LinuxThreads problems have recently
> been solved.  You might want to try the latest glibc 2.1.3
> pre-release.  Or have a little patience, glibc 2.1.3 is supposed to be
> released real soon now.

I think the Red Hat beta comes with a 2.1.3 pre release.

--Chris

-- 
------------
Christopher Blizzard
http://people.redhat.com/blizzard/
------------