Re: frame.c backtrace limit issue and fix

[Daniel Jacobowitz <drow at false dot org>]

On Mon, May 23, 2005 at 07:19:04PM +0100, Jonathan Larmour wrote:
> On my embedded m68k target, where debugging appears problematic, sometimes 
> frames called in frame.c:get_prev_frame() have their frame level set to 
> -1. If you have a backtrace limit set (with "set backtrace limit") then 
> this would mean the following test is done:
> 
> 
>   if (this_frame->level > backtrace_limit)
>     {
>       error ("Backtrace limit of %d exceeded", backtrace_limit);
>     }
> 
> This looks like it would be right even if the level is -1. However 
> backtrace_limit is unsigned and this results in this_frame->level being 
> promoted to an unsigned 0xffffffff value.
> 
> Therefore I'm attaching a patch to correct this, simply by making 
> backtrace_limit now be signed. Anyone who needs more than 2 billion stack 
> levels has bigger problems :-).
> 
> I do have write access so can check this in if someone approves.

You seem to be submitting patches against an old version; there already
is a function named add_setshow_integer_cmd.  Andrew added it in
February.  Also, someone wrote a testcase for this problem, but I think
it's still in my queue somewhere... ah, no, it's on Paul's plate at the
moment.

So only about half this patch is needed.  I took that, stuck a properly
formatted ChangeLog and PR number on it, and tested it.
Unfortunately... it didn't work.  The errors went away, but backtraces
didn't stop in the right place.

(gdb) bt
#0  factorial (value=4) at /big/fsf/commit/src/gdb/testsuite/gdb.base/break.c:111
#1  0x080484e3 in factorial (value=5) at /big/fsf/commit/src/gdb/testsuite/gdb.base/break.c:112
#2  0x080484e3 in factorial (value=6) at /big/fsf/commit/src/gdb/testsuite/gdb.base/break.c:112
Backtrace limit of 1 exceeded

Oops, that's not a backtrace limit of 1, that's a backtrace limit of
3... the documentation suggests that "bt 3" and "set backtrace limit 3"
should behave similarly in this regard.

I also don't think that the error() is appropriate.  "set backtrace
past-main" doesn't give an error at main.

With all those fixed, I get the attached patch, which I've tested,
regression tested, and checked in.  Thanks.

-- 
Daniel Jacobowitz
CodeSourcery, LLC

2005-05-28  Daniel Jacobowitz  <dan@codesourcery.com>
	    Jonathan Larmour  <jifl@eCosCentric.com>

	PR backtrace/1760
	* frame.c (backtrace_limit): Change type to int.
	(get_prev_frame): Update backtrace limit support.
	(_initialize_frame): Use add_setshow_integer_cmd for backtrace_limit.

Index: frame.c
===================================================================
RCS file: /cvs/src/src/gdb/frame.c,v
retrieving revision 1.209
diff -u -p -r1.209 frame.c
--- frame.c	22 May 2005 14:53:33 -0000	1.209
+++ frame.c	28 May 2005 22:41:17 -0000
@@ -141,7 +141,7 @@ Whether backtraces should continue past 
 		    value);
 }
 
-static unsigned int backtrace_limit = UINT_MAX;
+static int backtrace_limit = INT_MAX;
 static void
 show_backtrace_limit (struct ui_file *file, int from_tty,
 		      struct cmd_list_element *c, const char *value)
@@ -1258,9 +1258,16 @@ get_prev_frame (struct frame_info *this_
       return NULL;
     }
 
-  if (this_frame->level > backtrace_limit)
+  /* If the user's backtrace limit has been exceeded, stop.  We must
+     add two to the current level; one of those accounts for backtrace_limit
+     being 1-based and the level being 0-based, and the other accounts for
+     the level of the new frame instead of the level of the current
+     frame.  */
+  if (this_frame->level + 2 > backtrace_limit)
     {
-      error (_("Backtrace limit of %d exceeded"), backtrace_limit);
+      frame_debug_got_null_frame (gdb_stdlog, this_frame,
+				  "backtrace limit exceeded");
+      return NULL;
     }
 
   /* If we're already inside the entry function for the main objfile,
@@ -1610,16 +1617,16 @@ the rest of the stack trace."),
 			   &set_backtrace_cmdlist,
 			   &show_backtrace_cmdlist);
 
-  add_setshow_uinteger_cmd ("limit", class_obscure,
-			    &backtrace_limit, _("\
+  add_setshow_integer_cmd ("limit", class_obscure,
+			   &backtrace_limit, _("\
 Set an upper bound on the number of backtrace levels."), _("\
 Show the upper bound on the number of backtrace levels."), _("\
 No more than the specified number of frames can be displayed or examined.\n\
 Zero is unlimited."),
-			    NULL,
-			    show_backtrace_limit,
-			    &set_backtrace_cmdlist,
-			    &show_backtrace_cmdlist);
+			   NULL,
+			   show_backtrace_limit,
+			   &set_backtrace_cmdlist,
+			   &show_backtrace_cmdlist);
 
   /* Debug this files internals. */
   add_setshow_zinteger_cmd ("frame", class_maintenance, &frame_debug,  _("\