[newlib-cygwin] Cygwin: pty: Avoid client deadlock when pty master stops to read.

[Takashi Yano]

https://sourceware.org/git/gitweb.cgi?p=newlib-cygwin.git;h=c4fb5da2787693ea3bf20b2ac847f2c7d50da67c

commit c4fb5da2787693ea3bf20b2ac847f2c7d50da67c
Author: Takashi Yano <takashi.yano@nifty.ne.jp>
Date:   Mon Jul 1 17:44:53 2024 +0900

    Cygwin: pty: Avoid client deadlock when pty master stops to read.
    
    Previsouly, the following commands hangs:
      mintty -e timeout 1 dash -c 'yes aaaaaaaaaaaaaaaaaaaaaaaaa | cat'
    
    The mechanism is as follows.
    
    When the child process (timeout) is terminated, mintty seems to stop
    reading pty master even if yes or cat still alive.
    
    If the the pipe used to transfer output from pty slave to pty master
    is full due to lack of master reader, WriteFile() to the pipe is
    blocked. WriteFile() cannot be canceled by cygwin signal, therefore,
    pty slave hangs.
    
    This patch avoids hanging by checking pipe space before calling
    WriteFile() and prevents writing data more than space.
    
    Addresses: https://cygwin.com/pipermail/cygwin/2024-June/256178.html
    Reported-by: jojelino <jojelino@gmail.com>
    Signed-off-by: Takashi Yano <takashi.yano@nifty.ne.jp>

Diff:
---
 winsup/cygwin/fhandler/pty.cc | 25 ++++++++++++++++++++++---
 winsup/cygwin/release/3.5.4   |  4 ++++
 2 files changed, 26 insertions(+), 3 deletions(-)

diff --git a/winsup/cygwin/fhandler/pty.cc b/winsup/cygwin/fhandler/pty.cc
index 9d7ef3c9d..fa6bf1096 100644
--- a/winsup/cygwin/fhandler/pty.cc
+++ b/winsup/cygwin/fhandler/pty.cc
@@ -3118,8 +3118,22 @@ fhandler_pty_common::process_opost_output (HANDLE h, const void *ptr,
     return res; /* Discard write data */
   while (towrite)
     {
+      ssize_t space = towrite;
       if (!is_echo)
 	{
+	  IO_STATUS_BLOCK iosb = {{0}, 0};
+	  FILE_PIPE_LOCAL_INFORMATION fpli = {0};
+	  NTSTATUS status;
+
+	  status = NtQueryInformationFile (h, &iosb, &fpli, sizeof (fpli),
+					   FilePipeLocalInformation);
+	  if (!NT_SUCCESS (status))
+	    {
+	      if (towrite < len)
+		break;
+	      len = -1;
+	      return FALSE;
+	    }
 	  if (ttyp->output_stopped && is_nonblocking)
 	    {
 	      if (towrite < len)
@@ -3131,13 +3145,18 @@ fhandler_pty_common::process_opost_output (HANDLE h, const void *ptr,
 		  return TRUE;
 		}
 	    }
-	  while (ttyp->output_stopped)
-	    cygwait (10);
+	  if (ttyp->output_stopped || fpli.WriteQuotaAvailable == 0)
+	    {
+	      cygwait (1);
+	      continue;
+	    }
+	  space = fpli.WriteQuotaAvailable;
 	}
 
       if (!(ttyp->ti.c_oflag & OPOST))	// raw output mode
 	{
 	  DWORD n = MIN (OUT_BUFFER_SIZE, towrite);
+	  n = MIN (n, space);
 	  res = WriteFile (h, ptr, n, &n, NULL);
 	  if (!res)
 	    break;
@@ -3150,7 +3169,7 @@ fhandler_pty_common::process_opost_output (HANDLE h, const void *ptr,
 	  char *buf = (char *)ptr;
 	  DWORD n = 0;
 	  ssize_t rc = 0;
-	  while (n < OUT_BUFFER_SIZE && rc < towrite)
+	  while (n < OUT_BUFFER_SIZE && n < space && rc < towrite)
 	    {
 	      switch (buf[rc])
 		{
diff --git a/winsup/cygwin/release/3.5.4 b/winsup/cygwin/release/3.5.4
index 2a5f2b1bd..17db61146 100644
--- a/winsup/cygwin/release/3.5.4
+++ b/winsup/cygwin/release/3.5.4
@@ -15,3 +15,7 @@ Fixes:
 
 - Fix a problem that ldd command against cygwin DLLs sometimes hangs.
   Addresses: https://cygwin.com/pipermail/cygwin/2024-May/255991.html
+
+- Fix a problem that pty slave hangs on writing when pty master stops
+  to read.
+  Addresses: https://cygwin.com/pipermail/cygwin/2024-June/256178.html