This is the mail archive of the mailing list for the GDB project.

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: [RFC] Tighten and tweak ptrace argument checks

> Date: Fri, 9 Sep 2005 17:28:26 -0400
> From: Daniel Jacobowitz <>
> Hi Mark,
> I discovered recently that your rewrite of the ptrace argument type
> checks had busted some (not-yet-contributed, sorry) changes to use long
> long for ptrace calls on MIPS n32.
> This made me take a closer look at the tests.  It turns out that
> neither the return type checks nor argument type checks work on
> GNU/Linux.  The return type check will work with oldish versions of
> GCC, but not with 4.0 (or 3.4, I think).  The argument type checks
> won't work at all.  Here's the problem:
>   extern long int ptrace (enum __ptrace_request, ...);
> () won't match ...; this is important because of different calling
> conventions on some platforms.

It's always been my understanding that the the varargs stuff was used
because the number of arguments is actually dependent on the request.
It allows you to leave off any unused arguments instead of providing a
summy value like what's been done traditionally.  Bear in mind that
varargs functions only really work with word-sized arguments.
Otherwise you'll find yourself on a slippery slope very soon,
especially with system calls.

Anyway, the varargs stuff is a major nightmare, because different
versions of GCC treat the prototypes very differently.  That's why I
included a sane default.

> On i386-pc-linux-gnu the defaults work so this is merely an annoyance.
> When the real type is long long instead of long, well, it wasn't
> pretty...

But 'long long' as a return type or argument type really doesn't make
sense for ptrace(2).  But then of course MIPS doesn't make any sense
either, so that's perfectly fine ;-).

> So here's a proposed patch.  It handles the GNU/Linux case.  It handles
> long long.  It also is violently fatal if it fails, instead of making
> up something sensible - this is because I wasted several days trying to
> figure out what was wrong with GDB when it was casting all the ptrace
> arguments to the wrong type.  I'm sure it'll break the build in a lot
> of places but I think it's worth fixing if you want to use autoconf for
> this at all!

Sorry 'bout that, but you've created a ptrace variant that's
incompatible with everything else on the planet.  All other Linux
ports use long.  I presume you invented the 'long long' to be able to
return 64-bit register values with PTRACE_PEEKUSER.  That interface
should really die, and be replaced with PTRACE_GETREGS and friends.

> Any comments on the patch?

Sorry, yes, I'm almost certain this will break on OSF/1 or AIX or
HP-UX.  It'll probably break even for Linux if you're using an older

I think it'd be better to explicitly check for the Linux
varargs-with-enum type prototype.  Give me a few days, and I'll try to
come up with something better.

> I also noticed considerable PTRACE_ARG3_TYPE vs PTRACE_TYPE_ARG3
> confusion in the sources.  I think that the last time I looked at this,
> I convinced myself that it was possible to get them out of sync, and
> the results would be pretty gruesome.  I didn't touch that for now.

This is because there are several ports that are basically
unmaintained; nobody feels responsible enough to keep them up to date.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]