Cygwin fails to utilize Unicode replacement character

[Steven Penny]

On Tue, 4 Sep 2018 23:43:16, Thomas Wolff wrote:
> Traditionally, many terminals used to display the DEL character as a 
> checkered block, which is more or less the MEDIUM SHADE.
> This makes the glyph appear somewhat "erroneous" by convention.

I see - now that Unicode has some dedicated characters for this, it would make
sense to use them, especially since linux is already using them:

1. U+FFFD: http://unicode.org/charts/nameslist/n_FFF0.html
2. U+25A1: http://unicode.org/charts/nameslist/n_25A0.html

> valid code point with no glyph in font -> .notdef glyph -> WHITE SQUARE

this is not true. "WHITE SQUARE" refers to U+25A1, which is an actual character
and different from the ".notdef" glyph. as has been discussed as length in this
thread, the ".notdef glyph" is not an actual character, but a glyph that exists
at position 0 in the font, and while its appearance is not strictly defined,
some recommendations exist:

- empty rectangle
- rectangle with a question mark
- rectangle with an X

> Now if you switch to FFFD REPLACEMENT CHARACTER for invalid code point, 
> and considering that it does not exist in most actual fonts and that the 
> console does not apply font fallback, it will resolve to WHITE SQUARE, thus:
> folding the two different use cases into the same appearance,
> which is bad.

no again, it will resolve to ".notdef glyph", as I put above. otherwise yes, you
do have a point. in the case of a font without U+FFFD, you have ultimately:

invalid code point: .notdef glyph
missing character: .notdef glyph

several ideas have been proposed:

1. keep U+FFFD
2. go back to U+2592
3. use U+25A1 instead
4. use U+FFFD if possible else fallback to U+2592 or U+25A1

if we choose option 1, people not happy with the ambiguity can simply install
"dejavu-fonts" or similar, which Cygwin provides.


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple