64-bit emacs crashes a lot

Ken Brown kbrown@cornell.edu
Thu Aug 15 16:58:00 GMT 2013 

On 8/15/2013 8:32 AM, Ryan Johnson wrote:
> On 14/08/2013 10:04 AM, Ryan Johnson wrote:
>> On 10/08/2013 2:01 PM, Ken Brown wrote:
>>> I'm not sure that garbage collection is the underlying problem here.
>> Agree it's probably not GC... GC would just tend to trip over any bad
>> pointers that were lurking around...
>>
>> After a rash of crashes where I either forgot to attach gdb or forgot
>> to set appropriate breakpoints, I finally managed to catch the stack
>> trace below. It occurred during M-x compile, while emacs parsed the
>> compilation's rather copious output, which is by far the most common
>> type of crash I've been getting lately.
>>
> This time I really caught one, again during a compile (aside: the crash
> is usually right at the beginning of a compile, not partway through as
> my previous comment suggested). The short version is that I seem to have
> tickled a latent bug in emacs' character handling at character.c:189,
> where processing an 8-bit char could result in a return value larger
> than CHAR_MAX (which later triggers an abort). New stack trace and
> analysis follow.
>
> A quick peek at the backtrace and some source diving suggests something
> went wrong in bidi_fetch_char (bidi.c:928)... which unfortunately has
> about a dozen different ways of coming up with a character. There are
> only two major flavors that look like they can possibly fail, though:
> via STRING_CHAR_AND_LENGTH or UNIBYTE_TO_CHAR. Both are macros in
> character.h. The latter calls BYTE8_TO_CHAR, which simply returns
> byte+0x3fff00 (and can only take values between 0x3ffe80 and 0x3fff7f
> regardless of whether "byte" is signed or unsigned, though I think it's
> unsigned). The former is a pretty large macro (lots of ?:) that usually
> ends up fetching a char from p[], but has a test for sign bit to prevent
> unwanted sign extension (but again, I'm pretty sure they use unsigned
> bytes, so that shouldn't be an issue). However, the macro bottoms out by
> calling string_char (character.c:170), which includes the following
> expression:
>
> ((p)[1] & 0x3F) << 18
>
> That expression can return values significantly larger than CHAR_MAX
> (0x3FFFFF), the largest such being 0xFC0000.
>
> I suspect this is a bug in emacs (typo), and that the mask should really
> be 0xF: that would keep the value in bounds, and would also mirror a
> different branch of the if/else that differs only in the index passed to
> p[] and in using 0xF for the leading byte instead of 0x3F.
>
> On the other hand, the offending code seems to be trying to decode a
> 5-byte unicode sequence. If so, then the bug is not the mask 0x3F, but
> (a) forgetting to actually fetch the most significant two bits of the
> code point from p[0], and (b) having CHAR_MAX smaller than a valid
> 5-byte code point. Interestingly, CHAR_MAX is 0x3FFFFF---five F---while
> the largest valid 5-byte code point is 0x3FFFFFF---six F---and the
> largest 4-byte is 0x1FFFFF, not 0x3FFFFF (though perhaps the latter is
> intentional, to give emacs some out-of-band values to work with?).
> Meanwhile, 5- and 6-byte utf-8 sequences were banished in 2003
> (according to the Wikipedia entry on UTF-8), so it seems like this code
> path shouldn't even exist.
>
> I don't quite grok what sequence of input bytes it would take to tickle
> the above code path. I suspect that the input byte is invalid (= illegal
> unicode), due to the same memory corruption that made GC seg fault in a
> past crash, and just happens to tickle this latent bug. However, this
> charset bug should probably be sorted out and fixed, regardless of
> whether it's the cause of the troubles or merely the symptom of
> something deeper.
>
> Thoughts?
> Ryan

Eli is the expert on bidi.c (he wrote it).  He can probably tell you 
whether you've really bumped into an emacs bug here.

> Breakpoint 2, 0x000000010055d190 in kill ()
> (gdb) bt
> #0  0x000000010055d190 in kill ()
> #1  0x000000010053702e in process_send_signal (process=<optimized out>,
> signo=signo@entry=1, current_group=<optimized out>, nomsg=nomsg@entry=1)
> at /usr/src/debug/emacs-24.3-4/src/process.c:5948
> #2  0x0000000100537ae0 in kill_buffer_processes (buffer=4304412722) at
> /usr/src/debug/emacs-24.3-4/src/process.c:7157
> #3  0x0000000100485cb0 in shut_down_emacs (sig=sig@entry=6,
> stuff=4304412722) at /usr/src/debug/emacs-24.3-4/src/emacs.c:1915
> #4  0x0000000100485ea7 in terminate_due_to_signal (sig=6,
> backtrace_limit=10) at /usr/src/debug/emacs-24.3-4/src/emacs.c:329
> #5  0x00000001004a0363 in emacs_abort () at
> /usr/src/debug/emacs-24.3-4/src/sysdep.c:2152
> #6  0x000000010046d365 in bidi_get_type (ch=<optimized out>,
> override=override@entry=NEUTRAL_DIR) at
> /usr/src/debug/emacs-24.3-4/src/bidi.c:107
> #7  0x000000010046d527 in bidi_get_type (override=NEUTRAL_DIR,
> ch=<optimized out>) at /usr/src/debug/emacs-24.3-4/src/bidi.c:104
> #8  bidi_resolve_explicit_1 (bidi_it=bidi_it@entry=0x225d18) at
> /usr/src/debug/emacs-24.3-4/src/bidi.c:1411
> #9  0x000000010046d8b5 in bidi_resolve_explicit
> (bidi_it=bidi_it@entry=0x225d18) at
> /usr/src/debug/emacs-24.3-4/src/bidi.c:1540
> #10 0x000000010046db0a in bidi_resolve_weak
> (bidi_it=bidi_it@entry=0x225d18) at
> /usr/src/debug/emacs-24.3-4/src/bidi.c:1625
> #11 0x000000010046e3c1 in bidi_resolve_neutral
> (bidi_it=bidi_it@entry=0x225d18) at
> /usr/src/debug/emacs-24.3-4/src/bidi.c:1861
> #12 0x000000010046e7a0 in bidi_type_of_next_char (bidi_it=0x225d18) at
> /usr/src/debug/emacs-24.3-4/src/bidi.c:2032
> #13 bidi_level_of_next_char (bidi_it=bidi_it@entry=0x225d18) at
> /usr/src/debug/emacs-24.3-4/src/bidi.c:2145
> #14 0x000000010046faba in bidi_move_to_visually_next
> (bidi_it=bidi_it@entry=0x225d18) at
> /usr/src/debug/emacs-24.3-4/src/bidi.c:2355
> #15 0x000000010041c44f in set_iterator_to_next (it=it@entry=0x225370,
> reseat_p=reseat_p@entry=1) at /usr/src/debug/emacs-24.3-4/src/xdisp.c:7109
> #16 0x000000010042477a in display_line (it=it@entry=0x225370) at
> /usr/src/debug/emacs-24.3-4/src/xdisp.c:19864
> #17 0x0000000100426cd8 in try_window (window=window@entry=25782575869,
> pos=..., flags=flags@entry=1) at
> /usr/src/debug/emacs-24.3-4/src/xdisp.c:16353
> #18 0x000000010042c45a in redisplay_window
> (window=window@entry=25782575869,
> just_this_one_p=just_this_one_p@entry=0) at
> /usr/src/debug/emacs-24.3-4/src/xdisp.c:15879
> #19 0x000000010042e2b6 in redisplay_window_0
> (window=window@entry=25782575869) at
> /usr/src/debug/emacs-24.3-4/src/xdisp.c:13934
> #20 0x00000001004f5d9c in internal_condition_case_1
> (bfun=bfun@entry=0x10042e290 <redisplay_window_0>, arg=25782575869,
> handlers=4304384246, hfun=hfun@entry=0x10040fe90 <redisplay_window_error>)
>      at /usr/src/debug/emacs-24.3-4/src/eval.c:1327
> #21 0x000000010041458a in redisplay_windows (window=25782575869) at
> /usr/src/debug/emacs-24.3-4/src/xdisp.c:13914
> #22 0x00000001004145a8 in redisplay_windows (window=25780199573) at
> /usr/src/debug/emacs-24.3-4/src/xdisp.c:13908
> #23 0x000000010042f449 in redisplay_internal () at
> /usr/src/debug/emacs-24.3-4/src/xdisp.c:13493
> #24 0x0000000100430f95 in redisplay_preserve_echo_area
> (from_where=<optimized out>) at
> /usr/src/debug/emacs-24.3-4/src/xdisp.c:13754
> #25 0x0000000100535faa in wait_reading_process_output
> (time_limit=<optimized out>, nsecs=0, read_kbd=read_kbd@entry=-1,
> do_display=do_display@entry=true, wait_for_cell=4304412722,
> wait_proc=wait_proc@entry=0x0,
>      just_wait_proc=just_wait_proc@entry=0) at
> /usr/src/debug/emacs-24.3-4/src/process.c:4862
> #26 0x00000001004098da in sit_for (timeout=<optimized out>,
> reading=reading@entry=true, display_option=display_option@entry=1) at
> /usr/src/debug/emacs-24.3-4/src/dispnew.c:5978
> #27 0x0000000100491920 in read_char (commandflag=1, nmaps=2,
> maps=0x22a200, prev_event=4304412722, used_mouse_menu=0x22a327,
> end_time=end_time@entry=0x0) at
> /usr/src/debug/emacs-24.3-4/src/keyboard.c:2669
> #28 0x00000001004926a3 in read_key_sequence
> (keybuf=keybuf@entry=0x22a470, prompt=<optimized out>,
> dont_downcase_last=dont_downcase_last@entry=false,
> can_return_switch_frame=can_return_switch_frame@entry=true,
>      fix_current_buffer=fix_current_buffer@entry=true, bufsize=30) at
> /usr/src/debug/emacs-24.3-4/src/keyboard.c:9231
> #29 0x000000010049495e in command_loop_1 () at
> /usr/src/debug/emacs-24.3-4/src/keyboard.c:1459
> #30 0x00000001004f5c2e in internal_condition_case
> (bfun=bfun@entry=0x100494740 <command_loop_1>, handlers=4304470642,
> hfun=hfun@entry=0x10048ae40 <cmd_error>) at
> /usr/src/debug/emacs-24.3-4/src/eval.c:1289
> #31 0x000000010048630a in command_loop_2
> (ignore=ignore@entry=4304412722) at
> /usr/src/debug/emacs-24.3-4/src/keyboard.c:1168
> #32 0x00000001004f5aef in internal_catch (tag=<optimized out>,
> func=func@entry=0x1004862e0 <command_loop_2>, arg=4304412722) at
> /usr/src/debug/emacs-24.3-4/src/eval.c:1060
> #33 0x000000010048a914 in command_loop () at
> /usr/src/debug/emacs-24.3-4/src/keyboard.c:1147
> #34 recursive_edit_1 () at /usr/src/debug/emacs-24.3-4/src/keyboard.c:779
> #35 0x000000010048ac47 in Frecursive_edit () at
> /usr/src/debug/emacs-24.3-4/src/keyboard.c:843
> #36 0x000000010055e8ef in main (argc=<optimized out>, argv=<optimized
> out>) at /usr/src/debug/emacs-24.3-4/src/emacs.c:1537
>
>>
>> Thanks,
>> Ryan
>>
>> Breakpoint 2, 0x000000010055d190 in kill ()
>> (gdb) bt
>> #0  0x000000010055d190 in kill ()
>> #1  0x000000010053702e in process_send_signal
>> (process=process@entry=25781889629, signo=signo@entry=2,
>> current_group=<optimized out>, nomsg=nomsg@entry=0) at
>> /usr/src/debug/emacs-24.3-4/src/process.c:5948
>> #2  0x0000000100537198 in Finterrupt_process (process=25781889629,
>> current_group=<optimized out>) at
>> /usr/src/debug/emacs-24.3-4/src/process.c:5966
>> #3  0x00000001004f7761 in Ffuncall (nargs=<optimized out>,
>> args=<optimized out>) at /usr/src/debug/emacs-24.3-4/src/eval.c:2781
>> #4  0x000000010052b5ed in exec_byte_code (bytestr=4294962344,
>> vector=2268896, maxdepth=2, args_template=4303595040,
>> nargs=4304157760, args=0x100902032 <bss_sbrk_buffer+250194>)
>>     at /usr/src/debug/emacs-24.3-4/src/bytecode.c:900
>> #5  0x00000001004f7293 in funcall_lambda (fun=25778101277,
>> nargs=nargs@entry=0, arg_vector=arg_vector@entry=0x22a188) at
>> /usr/src/debug/emacs-24.3-4/src/eval.c:3010
>> #6  0x00000001004f75cb in Ffuncall (nargs=nargs@entry=1,
>> args=args@entry=0x22a180) at /usr/src/debug/emacs-24.3-4/src/eval.c:2839
>> #7  0x00000001004f8bef in apply1 (fn=25778613730, fn@entry=4304161216,
>> arg=arg@entry=4304412722) at /usr/src/debug/emacs-24.3-4/src/eval.c:2539
>> #8  0x00000001004f3567 in Fcall_interactively (function=4304161216,
>> record_flag=4304412722, keys=4299711881) at
>> /usr/src/debug/emacs-24.3-4/src/callint.c:377
>> #9  0x00000001004f7752 in Ffuncall (nargs=nargs@entry=4,
>> args=args@entry=0x22a3b0) at /usr/src/debug/emacs-24.3-4/src/eval.c:2785
>> #10 0x00000001004f91b7 in call3 (fn=<optimized out>, arg1=<optimized
>> out>, arg2=<optimized out>, arg3=<optimized out>) at
>> /usr/src/debug/emacs-24.3-4/src/eval.c:2603
>> #11 0x00000001004883cd in Fcommand_execute (cmd=<optimized out>,
>> record_flag=<optimized out>, keys=<optimized out>, special=<optimized
>> out>) at /usr/src/debug/emacs-24.3-4/src/keyboard.c:10241
>> #12 0x0000000100494ae8 in command_loop_1 () at
>> /usr/src/debug/emacs-24.3-4/src/keyboard.c:1587
>> #13 0x00000001004f5c2e in internal_condition_case
>> (bfun=bfun@entry=0x100494740 <command_loop_1>, handlers=4304470642,
>> hfun=hfun@entry=0x10048ae40 <cmd_error>) at
>> /usr/src/debug/emacs-24.3-4/src/eval.c:1289
>> #14 0x000000010048630a in command_loop_2
>> (ignore=ignore@entry=4304412722) at
>> /usr/src/debug/emacs-24.3-4/src/keyboard.c:1168
>> #15 0x00000001004f5aef in internal_catch (tag=<optimized out>,
>> func=func@entry=0x1004862e0 <command_loop_2>, arg=4304412722) at
>> /usr/src/debug/emacs-24.3-4/src/eval.c:1060
>> #16 0x000000010048a914 in command_loop () at
>> /usr/src/debug/emacs-24.3-4/src/keyboard.c:1147
>> #17 recursive_edit_1 () at /usr/src/debug/emacs-24.3-4/src/keyboard.c:779
>> #18 0x000000010048ac47 in Frecursive_edit () at
>> /usr/src/debug/emacs-24.3-4/src/keyboard.c:843
>> #19 0x000000010055e8ef in main (argc=<optimized out>, argv=<optimized
>> out>) at /usr/src/debug/emacs-24.3-4/src/emacs.c:1537



--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

More information about the Cygwin mailing list