64-bit emacs crashes a lot

Ryan Johnson ryan.johnson@cs.utoronto.ca
Thu Aug 15 12:32:00 GMT 2013


On 14/08/2013 10:04 AM, Ryan Johnson wrote:
> On 10/08/2013 2:01 PM, Ken Brown wrote:
>> I'm not sure that garbage collection is the underlying problem here.
> Agree it's probably not GC... GC would just tend to trip over any bad 
> pointers that were lurking around...
>
> After a rash of crashes where I either forgot to attach gdb or forgot 
> to set appropriate breakpoints, I finally managed to catch the stack 
> trace below. It occurred during M-x compile, while emacs parsed the 
> compilation's rather copious output, which is by far the most common 
> type of crash I've been getting lately.
>
This time I really caught one, again during a compile (aside: the crash 
is usually right at the beginning of a compile, not partway through as 
my previous comment suggested). The short version is that I seem to have 
tickled a latent bug in emacs' character handling at character.c:189, 
where processing an 8-bit char could result in a return value larger 
than CHAR_MAX (which later triggers an abort). New stack trace and 
analysis follow.

A quick peek at the backtrace and some source diving suggests something 
went wrong in bidi_fetch_char (bidi.c:928)... which unfortunately has 
about a dozen different ways of coming up with a character. There are 
only two major flavors that look like they can possibly fail, though: 
via STRING_CHAR_AND_LENGTH or UNIBYTE_TO_CHAR. Both are macros in 
character.h. The latter calls BYTE8_TO_CHAR, which simply returns 
byte+0x3fff00 (and can only take values between 0x3ffe80 and 0x3fff7f 
regardless of whether "byte" is signed or unsigned, though I think it's 
unsigned). The former is a pretty large macro (lots of ?:) that usually 
ends up fetching a char from p[], but has a test for sign bit to prevent 
unwanted sign extension (but again, I'm pretty sure they use unsigned 
bytes, so that shouldn't be an issue). However, the macro bottoms out by 
calling string_char (character.c:170), which includes the following 
expression:

((p)[1] & 0x3F) << 18

That expression can return values significantly larger than CHAR_MAX 
(0x3FFFFF), the largest such being 0xFC0000.

I suspect this is a bug in emacs (typo), and that the mask should really 
be 0xF: that would keep the value in bounds, and would also mirror a 
different branch of the if/else that differs only in the index passed to 
p[] and in using 0xF for the leading byte instead of 0x3F.

On the other hand, the offending code seems to be trying to decode a 
5-byte unicode sequence. If so, then the bug is not the mask 0x3F, but 
(a) forgetting to actually fetch the most significant two bits of the 
code point from p[0], and (b) having CHAR_MAX smaller than a valid 
5-byte code point. Interestingly, CHAR_MAX is 0x3FFFFF---five F---while 
the largest valid 5-byte code point is 0x3FFFFFF---six F---and the 
largest 4-byte is 0x1FFFFF, not 0x3FFFFF (though perhaps the latter is 
intentional, to give emacs some out-of-band values to work with?). 
Meanwhile, 5- and 6-byte utf-8 sequences were banished in 2003 
(according to the Wikipedia entry on UTF-8), so it seems like this code 
path shouldn't even exist.

I don't quite grok what sequence of input bytes it would take to tickle 
the above code path. I suspect that the input byte is invalid (= illegal 
unicode), due to the same memory corruption that made GC seg fault in a 
past crash, and just happens to tickle this latent bug. However, this 
charset bug should probably be sorted out and fixed, regardless of 
whether it's the cause of the troubles or merely the symptom of 
something deeper.

Thoughts?
Ryan

Breakpoint 2, 0x000000010055d190 in kill ()
(gdb) bt
#0  0x000000010055d190 in kill ()
#1  0x000000010053702e in process_send_signal (process=<optimized out>, 
signo=signo@entry=1, current_group=<optimized out>, nomsg=nomsg@entry=1) 
at /usr/src/debug/emacs-24.3-4/src/process.c:5948
#2  0x0000000100537ae0 in kill_buffer_processes (buffer=4304412722) at 
/usr/src/debug/emacs-24.3-4/src/process.c:7157
#3  0x0000000100485cb0 in shut_down_emacs (sig=sig@entry=6, 
stuff=4304412722) at /usr/src/debug/emacs-24.3-4/src/emacs.c:1915
#4  0x0000000100485ea7 in terminate_due_to_signal (sig=6, 
backtrace_limit=10) at /usr/src/debug/emacs-24.3-4/src/emacs.c:329
#5  0x00000001004a0363 in emacs_abort () at 
/usr/src/debug/emacs-24.3-4/src/sysdep.c:2152
#6  0x000000010046d365 in bidi_get_type (ch=<optimized out>, 
override=override@entry=NEUTRAL_DIR) at 
/usr/src/debug/emacs-24.3-4/src/bidi.c:107
#7  0x000000010046d527 in bidi_get_type (override=NEUTRAL_DIR, 
ch=<optimized out>) at /usr/src/debug/emacs-24.3-4/src/bidi.c:104
#8  bidi_resolve_explicit_1 (bidi_it=bidi_it@entry=0x225d18) at 
/usr/src/debug/emacs-24.3-4/src/bidi.c:1411
#9  0x000000010046d8b5 in bidi_resolve_explicit 
(bidi_it=bidi_it@entry=0x225d18) at 
/usr/src/debug/emacs-24.3-4/src/bidi.c:1540
#10 0x000000010046db0a in bidi_resolve_weak 
(bidi_it=bidi_it@entry=0x225d18) at 
/usr/src/debug/emacs-24.3-4/src/bidi.c:1625
#11 0x000000010046e3c1 in bidi_resolve_neutral 
(bidi_it=bidi_it@entry=0x225d18) at 
/usr/src/debug/emacs-24.3-4/src/bidi.c:1861
#12 0x000000010046e7a0 in bidi_type_of_next_char (bidi_it=0x225d18) at 
/usr/src/debug/emacs-24.3-4/src/bidi.c:2032
#13 bidi_level_of_next_char (bidi_it=bidi_it@entry=0x225d18) at 
/usr/src/debug/emacs-24.3-4/src/bidi.c:2145
#14 0x000000010046faba in bidi_move_to_visually_next 
(bidi_it=bidi_it@entry=0x225d18) at 
/usr/src/debug/emacs-24.3-4/src/bidi.c:2355
#15 0x000000010041c44f in set_iterator_to_next (it=it@entry=0x225370, 
reseat_p=reseat_p@entry=1) at /usr/src/debug/emacs-24.3-4/src/xdisp.c:7109
#16 0x000000010042477a in display_line (it=it@entry=0x225370) at 
/usr/src/debug/emacs-24.3-4/src/xdisp.c:19864
#17 0x0000000100426cd8 in try_window (window=window@entry=25782575869, 
pos=..., flags=flags@entry=1) at 
/usr/src/debug/emacs-24.3-4/src/xdisp.c:16353
#18 0x000000010042c45a in redisplay_window 
(window=window@entry=25782575869, 
just_this_one_p=just_this_one_p@entry=0) at 
/usr/src/debug/emacs-24.3-4/src/xdisp.c:15879
#19 0x000000010042e2b6 in redisplay_window_0 
(window=window@entry=25782575869) at 
/usr/src/debug/emacs-24.3-4/src/xdisp.c:13934
#20 0x00000001004f5d9c in internal_condition_case_1 
(bfun=bfun@entry=0x10042e290 <redisplay_window_0>, arg=25782575869, 
handlers=4304384246, hfun=hfun@entry=0x10040fe90 <redisplay_window_error>)
     at /usr/src/debug/emacs-24.3-4/src/eval.c:1327
#21 0x000000010041458a in redisplay_windows (window=25782575869) at 
/usr/src/debug/emacs-24.3-4/src/xdisp.c:13914
#22 0x00000001004145a8 in redisplay_windows (window=25780199573) at 
/usr/src/debug/emacs-24.3-4/src/xdisp.c:13908
#23 0x000000010042f449 in redisplay_internal () at 
/usr/src/debug/emacs-24.3-4/src/xdisp.c:13493
#24 0x0000000100430f95 in redisplay_preserve_echo_area 
(from_where=<optimized out>) at 
/usr/src/debug/emacs-24.3-4/src/xdisp.c:13754
#25 0x0000000100535faa in wait_reading_process_output 
(time_limit=<optimized out>, nsecs=0, read_kbd=read_kbd@entry=-1, 
do_display=do_display@entry=true, wait_for_cell=4304412722, 
wait_proc=wait_proc@entry=0x0,
     just_wait_proc=just_wait_proc@entry=0) at 
/usr/src/debug/emacs-24.3-4/src/process.c:4862
#26 0x00000001004098da in sit_for (timeout=<optimized out>, 
reading=reading@entry=true, display_option=display_option@entry=1) at 
/usr/src/debug/emacs-24.3-4/src/dispnew.c:5978
#27 0x0000000100491920 in read_char (commandflag=1, nmaps=2, 
maps=0x22a200, prev_event=4304412722, used_mouse_menu=0x22a327, 
end_time=end_time@entry=0x0) at 
/usr/src/debug/emacs-24.3-4/src/keyboard.c:2669
#28 0x00000001004926a3 in read_key_sequence 
(keybuf=keybuf@entry=0x22a470, prompt=<optimized out>, 
dont_downcase_last=dont_downcase_last@entry=false, 
can_return_switch_frame=can_return_switch_frame@entry=true,
     fix_current_buffer=fix_current_buffer@entry=true, bufsize=30) at 
/usr/src/debug/emacs-24.3-4/src/keyboard.c:9231
#29 0x000000010049495e in command_loop_1 () at 
/usr/src/debug/emacs-24.3-4/src/keyboard.c:1459
#30 0x00000001004f5c2e in internal_condition_case 
(bfun=bfun@entry=0x100494740 <command_loop_1>, handlers=4304470642, 
hfun=hfun@entry=0x10048ae40 <cmd_error>) at 
/usr/src/debug/emacs-24.3-4/src/eval.c:1289
#31 0x000000010048630a in command_loop_2 
(ignore=ignore@entry=4304412722) at 
/usr/src/debug/emacs-24.3-4/src/keyboard.c:1168
#32 0x00000001004f5aef in internal_catch (tag=<optimized out>, 
func=func@entry=0x1004862e0 <command_loop_2>, arg=4304412722) at 
/usr/src/debug/emacs-24.3-4/src/eval.c:1060
#33 0x000000010048a914 in command_loop () at 
/usr/src/debug/emacs-24.3-4/src/keyboard.c:1147
#34 recursive_edit_1 () at /usr/src/debug/emacs-24.3-4/src/keyboard.c:779
#35 0x000000010048ac47 in Frecursive_edit () at 
/usr/src/debug/emacs-24.3-4/src/keyboard.c:843
#36 0x000000010055e8ef in main (argc=<optimized out>, argv=<optimized 
out>) at /usr/src/debug/emacs-24.3-4/src/emacs.c:1537

>
> Thanks,
> Ryan
>
> Breakpoint 2, 0x000000010055d190 in kill ()
> (gdb) bt
> #0  0x000000010055d190 in kill ()
> #1  0x000000010053702e in process_send_signal 
> (process=process@entry=25781889629, signo=signo@entry=2, 
> current_group=<optimized out>, nomsg=nomsg@entry=0) at 
> /usr/src/debug/emacs-24.3-4/src/process.c:5948
> #2  0x0000000100537198 in Finterrupt_process (process=25781889629, 
> current_group=<optimized out>) at 
> /usr/src/debug/emacs-24.3-4/src/process.c:5966
> #3  0x00000001004f7761 in Ffuncall (nargs=<optimized out>, 
> args=<optimized out>) at /usr/src/debug/emacs-24.3-4/src/eval.c:2781
> #4  0x000000010052b5ed in exec_byte_code (bytestr=4294962344, 
> vector=2268896, maxdepth=2, args_template=4303595040, 
> nargs=4304157760, args=0x100902032 <bss_sbrk_buffer+250194>)
>     at /usr/src/debug/emacs-24.3-4/src/bytecode.c:900
> #5  0x00000001004f7293 in funcall_lambda (fun=25778101277, 
> nargs=nargs@entry=0, arg_vector=arg_vector@entry=0x22a188) at 
> /usr/src/debug/emacs-24.3-4/src/eval.c:3010
> #6  0x00000001004f75cb in Ffuncall (nargs=nargs@entry=1, 
> args=args@entry=0x22a180) at /usr/src/debug/emacs-24.3-4/src/eval.c:2839
> #7  0x00000001004f8bef in apply1 (fn=25778613730, fn@entry=4304161216, 
> arg=arg@entry=4304412722) at /usr/src/debug/emacs-24.3-4/src/eval.c:2539
> #8  0x00000001004f3567 in Fcall_interactively (function=4304161216, 
> record_flag=4304412722, keys=4299711881) at 
> /usr/src/debug/emacs-24.3-4/src/callint.c:377
> #9  0x00000001004f7752 in Ffuncall (nargs=nargs@entry=4, 
> args=args@entry=0x22a3b0) at /usr/src/debug/emacs-24.3-4/src/eval.c:2785
> #10 0x00000001004f91b7 in call3 (fn=<optimized out>, arg1=<optimized 
> out>, arg2=<optimized out>, arg3=<optimized out>) at 
> /usr/src/debug/emacs-24.3-4/src/eval.c:2603
> #11 0x00000001004883cd in Fcommand_execute (cmd=<optimized out>, 
> record_flag=<optimized out>, keys=<optimized out>, special=<optimized 
> out>) at /usr/src/debug/emacs-24.3-4/src/keyboard.c:10241
> #12 0x0000000100494ae8 in command_loop_1 () at 
> /usr/src/debug/emacs-24.3-4/src/keyboard.c:1587
> #13 0x00000001004f5c2e in internal_condition_case 
> (bfun=bfun@entry=0x100494740 <command_loop_1>, handlers=4304470642, 
> hfun=hfun@entry=0x10048ae40 <cmd_error>) at 
> /usr/src/debug/emacs-24.3-4/src/eval.c:1289
> #14 0x000000010048630a in command_loop_2 
> (ignore=ignore@entry=4304412722) at 
> /usr/src/debug/emacs-24.3-4/src/keyboard.c:1168
> #15 0x00000001004f5aef in internal_catch (tag=<optimized out>, 
> func=func@entry=0x1004862e0 <command_loop_2>, arg=4304412722) at 
> /usr/src/debug/emacs-24.3-4/src/eval.c:1060
> #16 0x000000010048a914 in command_loop () at 
> /usr/src/debug/emacs-24.3-4/src/keyboard.c:1147
> #17 recursive_edit_1 () at /usr/src/debug/emacs-24.3-4/src/keyboard.c:779
> #18 0x000000010048ac47 in Frecursive_edit () at 
> /usr/src/debug/emacs-24.3-4/src/keyboard.c:843
> #19 0x000000010055e8ef in main (argc=<optimized out>, argv=<optimized 
> out>) at /usr/src/debug/emacs-24.3-4/src/emacs.c:1537
>
>
> -- 
> Problem reports:       http://cygwin.com/problems.html
> FAQ:                   http://cygwin.com/faq/
> Documentation:         http://cygwin.com/docs.html
> Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
>


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple



More information about the Cygwin mailing list