This is the mail archive of the cygwin mailing list for the Cygwin project.

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: File output Question

On 3 January 2011 03:04, ERIC HO wrote:
> Looking at this closer, I've found the file contains hex '00' between characters. In vim, it displays ^@ for hex '00'. So it looks unreadable. But when I use cat or tail commands, it strips off hex '00' and looks readable. For example. in vim, it displays:
> P^@r^@o^@d^@u^@c^@t^@:
> With cat and tail, the output displays Product:

Apparently the file is encoded in little-endian UTF-16, which is
Windows' favoured Unicode encoding. That effectively inserts a NUL
byte after each ASCII byte. The ^@ is caret notation for NUL.

When sending the file directly to the terminal with cat or tail, the
terminal will simply ignore the NUL bytes, whereas vim makes an
attempt to show you the whole content of the file. UTF-16 files ought
to have the so-called Byte Order Mark (BOM) for indicating
little-endian or big-endian at the start, and vim automatically
recognises a file as UTF-16 when that's there. Your file doesn't
appear to have that though, so vim interprets it as ASCII instead.


Problem reports:
Unsubscribe info:

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]