Bug in libiconv?

[Charles Wilson]

On 1/24/2011 10:41 AM, Corinna Vinschen wrote:
> Here's what happens on Cygwin:
> 
>   $ gcc -g -o ic ic.c -liconv
>   $ ./ic
>   iconv: 138 <Invalid or incomplete multibyte or wide character>
>   in = <Liian pitkä sana>, inbuf = <ä sana>, inbytesleft = 7, outbytesleft = 492
>   iconv: 138 <Invalid or incomplete multibyte or wide character>
>   in = <Liian pitkä sana>, inbuf = <ä sana>, inbytesleft = 7, outbytesleft = 492
>   iconv: 138 <Invalid or incomplete multibyte or wide character>
>   in = <Liian pitkä sana>, inbuf = <ä sana>, inbytesleft = 7, outbytesleft = 492
>   in = <Liian pitkä sana>, inbuf = <>, inbytesleft = 0, outbytesleft = 480

Confirmed.

> So, AFAICS, there are two problems:
> 
>   - Even though iconv_open has been opened explicitely with "UTF-8" as
>     input string, the conversion still depends on the current application
>     codeset.  That dsoesn't make sense.
> 
>   - Even though the last parameter to iconv is defined in bytes, the
>     value of outbytesleft after the conversion is the number of remaining
>     wchar"t's, not the number of remaining bytes.  That's contrary to what
>     POSIX defines, see
>     http://pubs.opengroup.org/onlinepubs/9699919799/functions/iconv.html
> 
> Is this analyzes correct?  Is there by any chance a newer version of
> libiconv2 which does not have these problems?

Well, iconv's behavior is very dependent on detailed characteristics of
the system on which it was compiled -- e.g. it's very finicky about the
platform's behavior vis character sets.

Now, cygwin's libiconv-1.13.1 was built a LONG time ago (2009 Dec 23),
and many things have changed in cygwin itself since then (e.g.
cygwin-1.7.1-1 was current at that time).

Now, since there has not yet been an updated upstream release of
libiconv, my first step would be to simply rebuild our existing
libiconv-1.13.1 on a platform with current cygwin (1.7.7-1), and try the
test case again.

If that doesn't correct the issue...then I'd try to run your test case
on linux, but *explicitly* using libiconv on that system, rather than
(as is typically the case on linux) relying on the underlying glibc
implementation of iconv functionality.  If the test case fails there,
then we've got a presumption that the problem is in the (generic,
cross-platform bits of) libiconv library itself.  Then, it's debugging
time... :-(

--
Chuck



--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple