Re: Spurious testsuite failures due to multibyte locale

[Michael Elizabeth Chastain <mec at shout dot net>]

Okay, I wrote a standalone test program, it really wasn't painful.  :)

I sent a bug report to bug-readline@gnu.org.  I can't find a Web link
to it so I am including a copy here for gdb history.

Now the readline maintainers can fix the bug, spin a release, and
Elena Z can pick it up.  Meanwhile, we can use the workaround from
Andreas S.

Michael C

===

Synopsis

  Readline 4.3 does perverse refresh in UTF-8 mode.
  This breaks the gdb automated test suite.

Environment

  target      = i686-pc-linux-gnu
  host        = i686-pc-linux-gnu
  osversion   = red-hat-8.0
  gcc         = 3.2.1
  binutils    = 2.13.2.1
  libreadline = 4.3
  libtermcap  = 2.0.8-31-rh
  libc        = 2.2.93-5-rh

Description

  I see perverse screen output.  Instead of "(gdb) foo bar", I see
  "(gdb) ^M(gdb) f^M(gdb) fo^M(Gdb) foo^M ..." with a carriage return
  for each character.

  This output style comes as a surprise to all the dejagnu patterns
  in the gdb test suite, breaking the test suite.  The symptoms are
  massive timeouts starting in gdb.base/annota1.exp and continuing
  throughout the test suite.

How to Reproduce

  Script started on Sun Jan  5 19:56:44 2003

  bash-2.05b$ cat z1.c
  #include <stdio.h>
  #include <readline/readline.h>
  
  int main ()
  {
    char * line;
  
    for (;;)
    {
      line = readline ("(prompt) ");
      if (line == NULL)
	break;
  
      printf ("%s\n", line);
    }
  
    printf ("\n");
    return 0;
  }

  bash-2.05b$ gcc -v -g -c z1.c
  Reading specs from /berman/migchain/install/target/native/gcc-3.2.1-binutils-2.13.2.1/lib/gcc-lib/i686-pc-linux-gnu/3.2.1/specs
  Configured with: /berman/migchain/source/gcc-3.2.1/configure --prefix=/berman/migchain/install/target/native/gcc-3.2.1-binutils-2.13.2.1 --disable-shared --with-gnu-as --with-as=/berman/migchain/install/target/native/binutils-2.13.2.1/bin/as --with-gnu-ld --with-ld=/berman/migchain/install/target/native/binutils-2.13.2.1/bin/ld
  Thread model: posix
  gcc version 3.2.1
   /berman/migchain/install/target/native/gcc-3.2.1-binutils-2.13.2.1/lib/gcc-lib/i686-pc-linux-gnu/3.2.1/cc1 -lang-c -v -D__GNUC__=3 -D__GNUC_MINOR__=2 -D__GNUC_PATCHLEVEL__=1 -D__GXX_ABI_VERSION=102 -D__ELF__ -Dunix -D__gnu_linux__ -Dlinux -D__ELF__ -D__unix__ -D__gnu_linux__ -D__linux__ -D__unix -D__linux -Asystem=posix -D__NO_INLINE__ -D__STDC_HOSTED__=1 -Acpu=i386 -Amachine=i386 -Di386 -D__i386 -D__i386__ -D__tune_i686__ -D__tune_pentiumpro__ z1.c -quiet -dumpbase z1.c -g -version -o /tmp/cck1itw9.s
  GNU CPP version 3.2.1 (cpplib) (i386 Linux/ELF)
  GNU C version 3.2.1 (i686-pc-linux-gnu)
	  compiled by GNU C version 3.2.1.
  ignoring nonexistent directory "/berman/migchain/install/target/native/gcc-3.2.1-binutils-2.13.2.1/i686-pc-linux-gnu/include"
  #include "..." search starts here:
  #include <...> search starts here:
   /usr/local/include
   /berman/migchain/install/target/native/gcc-3.2.1-binutils-2.13.2.1/include
   /berman/migchain/install/target/native/gcc-3.2.1-binutils-2.13.2.1/lib/gcc-lib/i686-pc-linux-gnu/3.2.1/include
   /usr/include
  End of search list.
   /berman/migchain/install/target/native/binutils-2.13.2.1/bin/as -V -Qy -o z1.o /tmp/cck1itw9.s
  GNU assembler version 2.13.2.1 (i686-pc-linux-gnu) using BFD version 2.13.2.1

  bash-2.05b$ gcc -v -g -o z1 z1.o ./install/lib/libreadline.a -ltermcap
  Reading specs from /berman/migchain/install/target/native/gcc-3.2.1-binutils-2.13.2.1/lib/gcc-lib/i686-pc-linux-gnu/3.2.1/specs
  Configured with: /berman/migchain/source/gcc-3.2.1/configure --prefix=/berman/migchain/install/target/native/gcc-3.2.1-binutils-2.13.2.1 --disable-shared --with-gnu-as --with-as=/berman/migchain/install/target/native/binutils-2.13.2.1/bin/as --with-gnu-ld --with-ld=/berman/migchain/install/target/native/binutils-2.13.2.1/bin/ld
  Thread model: posix
  gcc version 3.2.1
   /berman/migchain/install/target/native/gcc-3.2.1-binutils-2.13.2.1/lib/gcc-lib/i686-pc-linux-gnu/3.2.1/collect2 --eh-frame-hdr -m elf_i386 -dynamic-linker /lib/ld-linux.so.2 -o z1 /usr/lib/crt1.o /usr/lib/crti.o /berman/migchain/install/target/native/gcc-3.2.1-binutils-2.13.2.1/lib/gcc-lib/i686-pc-linux-gnu/3.2.1/crtbegin.o -L/berman/migchain/install/target/native/gcc-3.2.1-binutils-2.13.2.1/lib/gcc-lib/i686-pc-linux-gnu/3.2.1 -L/berman/migchain/install/target/native/gcc-3.2.1-binutils-2.13.2.1/lib/gcc-lib/i686-pc-linux-gnu/3.2.1/../../.. z1.o ./install/lib/libreadline.a -ltermcap -lgcc -lc -lgcc /berman/migchain/install/target/native/gcc-3.2.1-binutils-2.13.2.1/lib/gcc-lib/i686-pc-linux-gnu/3.2.1/crtend.o /usr/lib/crtn.o

  bash-2.05b$ INPUTRC=/dev/null LANG=en_US ./z1
  (prompt) foo
  foo
  (prompt) 

  bash-2.05b$ INPUTRC=/dev/null LANG=en_US.UTF-8 ./z1
  

(prompt) 
(prompt) 
(prompt) f
(prompt) f
(prompt) fo
(prompt) fo
(prompt) foo
(prompt) foo
  foo
  

(prompt) 
(prompt) 
  bash-2.05b$ exit

  Script done on Sun Jan  5 19:58:24 2003

Analysis

  It looks like readline/display.c:_rl_move_cursor_relative
  has a multibyte bug (MB_CUR_MAX != 1 &&d rl_byte_oriented == 0).
  The first optimization "_rl_last_c_pos == new" is disabled
  because "_rl_last_c_pos" is a display position but "new"
  is an index in a a multibyte string, so they are not comparable.
  That is okay.  But later on, this block appears:
 
   #if defined (HANDLE_MULTIBYTE)
   /* NEW points to the buffer point, but _rl_last_c_pos is the display point.
        The byte length of the string is probably bigger than the column width
        of the string, which means that if NEW == _rl_last_c_pos, then NEW's
        display point is less than _rl_last_c_pos. */
     else if (_rl_last_c_pos >= new)
   #else
     else if (_rl_last_c_pos > new)
   #endif
 
  Not so fast, cowboy!  _rl_last_c_pos == new here, but all the
  characters are plain, so they might match just fine.  But readline
  pessimizes out and issues a CR and reprints the line every time.
  There should be logic here to compare the strings rather than just
  assume that a left-move is needed.

Fix

  I don't have one.

Workaround

  Avoid UTF-8 encoding: "export LANG=en_US" (for en_US people).