3.1.x: Mangled input/output when calling non-cygwin programs

[Takashi Yano]

On Mon, 3 Feb 2020 04:00:33 +0900
Takashi Yano wrote:
> On Sun, 2 Feb 2020 13:18:03 +0100
> m0viefreak wrote:
> > It seems that the cygwin1.dll does not care about TERM, but if a
> > non-native application *uses* TERM for whatever reason things start to
> > break. If indeed 'cygwin' is the right terminfo to be used, should the
> > PTY code then somehow set TERM=cygwin in the non-native process? But
> > what if the none-native program doesn't even know the 'cygwin' terminfo?
> 
> This make sens only if apps use TERM *right way*. In other words,
> if apps uses TERM in the manner of termcap or terminfo. I don't
> think Maven does that.
> 
> Now I am looking into jansi library which Maven uses, and found
> it sets IS_CYGWIN flag according to TERM environmen. It seems
> that IS_CYGWIN is set to false if TERM=cygwin. In this case,
> jansi uses SetConsoleTextAttribute() call to set text color
> rather than escape sequences such as ESC[31m. Ohterwise, escape
> sequence is used.
> 
> See:
> https://github.com/fusesource/jansi/blob/master/jansi/src/main/java/org/fusesource/jansi/AnsiConsole.java
> 
> New cygwin provide pure windows console for native apps, so
> escape sequence is not handled correctly. This is the cause
> of the problem 1) and 2). It may be a problem that TERM is
> set even though pure windows console does not have the
> capability which TERM environment declares. For the one
> thought, it can make sense to unset TERM before execute
> native-console apps. However, this does not solve the issue
> of Maven. If Maven (jansi) handles TERM *correctly*, this
> can be a solution.

As for the problem 1) and 2), I think Maven (jansi) should be
fixed rather than cygwin. I have applied the following patch
to jansi, and confirmed this solves the problem 1) and 2).

diff --git a/jansi/src/main/java/org/fusesource/jansi/AnsiConsole.java b/jansi/src/main/java/org/fusesource/jansi/AnsiConsole.java
index 628cad3..a5400c5 100644
--- a/jansi/src/main/java/org/fusesource/jansi/AnsiConsole.java
+++ b/jansi/src/main/java/org/fusesource/jansi/AnsiConsole.java
@@ -19,6 +19,10 @@ import static org.fusesource.jansi.internal.CLibrary.STDERR_FILENO;
 import static org.fusesource.jansi.internal.CLibrary.STDOUT_FILENO;
 import static org.fusesource.jansi.internal.CLibrary.isatty;
 
+import static org.fusesource.jansi.internal.Kernel32.GetConsoleMode;
+import static org.fusesource.jansi.internal.Kernel32.GetStdHandle;
+import static org.fusesource.jansi.internal.Kernel32.STD_OUTPUT_HANDLE;
+
 import java.io.FilterOutputStream;
 import java.io.IOException;
 import java.io.OutputStream;
@@ -54,10 +58,11 @@ public class AnsiConsole {
      */
     static final boolean IS_CON_EMU_ANSI = "ON".equals(System.getenv("ConEmuANSI"));
 
+    static private int[] mode = {0};
     static final boolean IS_CYGWIN = IS_WINDOWS
             && System.getenv("PWD") != null
             && System.getenv("PWD").startsWith("/")
-            && !"cygwin".equals(System.getenv("TERM"));
+            && GetConsoleMode(GetStdHandle(STD_OUTPUT_HANDLE), mode) == 0;
 
     static final boolean IS_MINGW_XTERM = IS_WINDOWS
             && System.getenv("MSYSTEM") != null


-- 
Takashi Yano <takashi.yano@nifty.ne.jp>

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple