Re: gdb: detection and/or fork+gethostbyname crash workaround?

[Kevin Buettner <kevinb at cygnus dot com>]

On Jul 23,  9:01pm, Glenn F. Maynard wrote:

> Is there any way to tell if a (C) program is being debugged by gdb?

You didn't say what platform you want this for.

If it's one which uses ptrace(), the kernel usually prohibits two
processes from invoking ptrace() on the same inferior.  So one
strategy might be to have the program in question cause ptrace to be
invoked on itself.  I don't think the process will be able to do this
itself; I think it's likely that it would have to fork and let the
child attempt this.  The return status from wait() or waitpid() could
indicate whether the attempt to invoke ptrace() was successful or not.

If it's a system which uses /proc as the debug interface, I think
/proc/PID/ctl may only be opened with write access by only one process.
So you could test to see if this file can be opened with write access.
If it can, it means that it's not being debugged; if it can't, it's
being debugged.  (If you're successful in opening it, make sure you
close it...)

More generally, the debug interface provided by the OS (of which
ptrace() and /proc are but two examples) usually has a mechanism
which prevents a process from being debugged twice.  You just have
to figure out what that mechanism is...

Of course, you should keep in mind that a race condition is possible.
Just because you've determined at one point in your program that you
aren't being debugged doesn't mean that this condition will hold
later on.

> I've had the "dynamic linking in a forked process being debugged"
> bite me twice now.  The first time around, I moved the process
> to another binary (a resolver binary; exec()ing it fixed this).  I'm
> trying to work around this bug in another program, now, where making
> a separate binary for resolving is undesirable; I want to force it
> to not fork resolves when being debugged, and do so automatically.

Could you describe this problem in a bit more detail?  It sounds to
me like something ought to be fixed on the GDB side of things, but
I don't really understand what the problem is...

> Portability isn't much of a concern: the workaround can be autoconf'd
> to only happen on the architectures it works (and/or is necessary) for.
> 
> Alternatively, a workaround for this particular bug would be equally
> sufficient.  Of course, having it fixed would be nice ... it's been around
> for at least a year and prevents all "fork for DNS" code from working under
> gdb unless it happens to exec to do so, and it's an odd enough bug
> that it's probably cost a great deal of time for those hit by it. (It's
> been reported multiple times in both the "forked gethostbyname crashes"
> and "forked dynamic linking crashes" guises.)

I searched gdb@sources.redhat.com w/ both of these search phrases and
only came up with the message to which I'm responding.  Can you provide
some URLs for the archived messages?

Kevin