Sv: Sv: Sv: Sv: Named pipes and multiple writers

[Ken Brown]

On 3/28/2020 10:19 PM, Ken Brown via Cygwin wrote:
> On 3/28/2020 11:43 AM, Ken Brown via Cygwin wrote:
>> On 3/28/2020 8:10 AM, sten.kristian.ivarsson@gmail.com wrote:
>>>> On 3/27/2020 10:53 AM, sten.kristian.ivarsson@gmail.com wrote:
>>>>>> On 3/26/2020 7:19 PM, Ken Brown via Cygwin wrote:
>>>>>>> On 3/26/2020 6:39 PM, Ken Brown via Cygwin wrote:
>>>>>>>> On 3/26/2020 6:01 PM, sten.kristian.ivarsson@gmail.com wrote:
>>>>>>>>> The ENIXIO occurs when parallel child-processes simultaneously
>>>>>>>>> using O_NONBLOCK opening the descriptor.
>>>>>>>>
>>>>>>>> This is consistent with my guess that the error is generated by
>>>>>>>> fhandler_fifo::wait.  I have a feeling that read_ready should have
>>>>>>>> been created as a manual-reset event, and that more care is needed
>>>>>>>> to make sure it's set when it should be.
>>>>>>>>
>>>>>>>>> I could provide a code-snippet
>>>>>>>>> to reproduce it if wanted ?
>>>>>>>>
>>>>>>>> Yes, please!
>>>>>>>
>>>>>>> That might not be necessary.  If you're able to build the git repo
>>>>>>> master branch, please try the attached patch.
>>>>>
>>>>>> Here's a better patch.
>>>>>
>>>>>
>>>>> I finally succeeded to build latest master (make is not my favourite
>>>>> tool) and added the patch, but still no success in my little
>>>>> test-program (see
>>>>> attachment) when creating a write-file-descriptor with O_NONBLOCK
>>>
>>>> Your test program fails for me on Linux too.  Here's the output from one
>>> run:
>>>
>>> You're right. That was extremely careless of me to not test this in Linux
>>> first :-)
>>
>> No problem.
>>
>>> I can assure that we have a use case that works on Linux but not in Cygwin,
>>> but it seems like I failed to narrow it down in the wrong way
>>>
>>> I'll try to rearrange my code (that works in Linux) to mimic our application
>>> but in a simple way (I'll be back)
>>
>> OK, I'll be waiting for you.  BTW, if it's not too hard to write your test 
>> case in plain C, or at least less modern C++, that would simplify things for 
>> me.  For example, your pipe.cpp failed to compile on one Linux machine I 
>> wanted to test it on, presumably because that machine had an older C++ compiler.
> 
> Never mind.  I was able to reproduce the problem and find the cause.  What 
> happens is that when the first subprocess exits, fhandler_fifo::close resets 
> read_ready.  That causes the second and subsequent subprocesses to think that 
> there's no reader open, so their attempts to open a writer with O_NONBLOCK fail 
> with ENXIO.
> 
> I should be able to fix this tomorrow.

I've pushed what I think is a fix to the topic/fifo branch.  I tested it with 
the attached program, which is a variant of the test case you sent last week. 
Please test it in your use case.

Note: If you've previously pulled the topic/fifo branch, then you will probably 
get a lot of conflicts when you pull again, because I did a forced push a few 
days ago.  If that happens, just do

   git reset --hard origin/topic/fifo

It turned out that the fix required some of the ideas that I've been working on 
in connection with allowing multiple readers.  Even though the code allows a 
FIFO to be *explicitly* opened for reading only once, there can still be several 
open file descriptors for readers because of dup and fork.  The existing code on 
git master doesn't handle those situations properly.

The code on topic/fifo doesn't completely fix that yet, but I think it should 
work under the following assumptions:

1. The FIFO is opened only once for reading.

2. The file descriptor obtained from this is the only one on which a read is 
attempted.

I'm working on removing both of these restrictions.

Ken
-------------- next part --------------
/* Adapted from
   https://sourceware.org/pipermail/cygwin/2020-March/244219.html */

#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <errno.h>
#include <sys/wait.h>
#include <string.h>

#define FIFO "/tmp/myfifo"
#define nsubproc  5
#define nmessages 4
#define pid_len   4

int
error (const int n, const char *name)
{
  fprintf (stderr, "\n%d\t%s:\t%d\t%s\n", getpid (), name, n, strerror (n));
  return n;
}

int
main ()
{
  if (mkfifo (FIFO, S_IRUSR | S_IWUSR | S_IWGRP) < 0
      && errno != EEXIST)
    return error (errno, "mkfifo");

  int rfd = open (FIFO, O_RDWR);
  if (rfd < 0)
    return error (errno, "open reader");

  for (int i = 0; i < nsubproc; i++)
    {
      pid_t pid = fork ();

      if (pid < 0)
	return error (errno, "fork");

      if (pid == 0)
	{
	  printf ("child %d\n", getpid ());
	  for (int j = 0; j < nmessages; j++)
	    {
	      char buf[pid_len + 2]; /* +1 for newline, +1 for nul */

	      int wfd = open (FIFO, O_WRONLY | O_NONBLOCK);
	      if (wfd < 0)
		_exit (error (errno, "open writer"));
	      sprintf (buf, "%d\n", getpid ());
	      ssize_t nwritten = write (wfd, buf, strlen (buf));
	      if (nwritten < 0)
		error (errno, "write");
	      /* printf ("i = %d, j = %d, nwritten = %zd\n", i, j, nwritten); */
	      close (wfd);
	    }
	  _exit (0);
	}
    }

  printf ("parent\n");
  char buf[pid_len + 2];
  for (int i = 0; i < nsubproc; i++)
    for (int j = 0; j < nmessages; j++)
      {
	if (read (rfd, buf, nsubproc) < 0)
	  error (errno, "read");
	buf[pid_len + 1] = '\0';
	printf ("%s", buf);
      }
  close (rfd);
}