Re: 1.7.8: write fails with EAGAIN

[Christopher Faylor <cgf-use-the-mailinglist-please at cygwin dot com>]

On Mon, Mar 07, 2011 at 11:38:49AM -0500, Christopher Faylor wrote:
>On Mon, Mar 07, 2011 at 10:37:08AM -0500, Christopher Faylor wrote:
>>On Mon, Mar 07, 2011 at 11:39:51AM +0100, Corinna Vinschen wrote:
>>>On Mar  5 21:12, Robert Wruck wrote:
>>>> Hi,
>>>> 
>>>> recently, I found that cygwin-git was not able to 'cat-file' files
>>>> that exceeded some size (in my case about 80MB).
>>>> I tracked this down to the cygwin implementation of write() that
>>>> behaves quite odd in some cases.
>>>> 
>>>> I wrote a small program (source attached) that mmaps a given file
>>>> and tries to write it to another file or stdout.
>>>> 
>>>> The results vary:
>>>> 
>>>> If the destination is a file (`writetest infile outfile` or
>>>> `writetest infile > outfile`), the write succeeds in a single call.
>>>> 
>>>> If the destination is a pipe (`writetest infile | cat > outfile`),
>>>> the write succeeds in most cases. BUT:
>>>> 
>>>> Under WinXP (XP Service Pack 2, 32bit), the call returns -1 and
>>>> errno=EAGAIN. Nevertheless, SOME data is written to the pipe (in my
>>>> case 4096 byte for each call).
>>>> This breaks git since it does an infinite loop while errno=EAGAIN.
>>>
>>>Hang on, you are saying that a *blocking* write(2) to a pipe returns
>>>with EAGAIN?  Are you sure?  It would be quite a surprise if git would
>>>actually do that.  EAGAIN is only an expected error for non-blocking
>>>I/O, so applications which use blocking I/O usually only test for EINTR.
>>
>>I can barely convince myself that there's a pathological case where an
>>EAGAIN could leak out.  I'm investigating now.
>
>Actually, in this case, it looks like the problem is that Windows
>doesn't like sending a huge buffer to a pipe.  The errno in this case
>should probably be something like EFBIG rather than EAGAIN.
>
>Does git deal with this type of errno gracefully or does it just abort
>if the write() fails for any reason?  I'd rather just fail and let the
>caller deal with it than complicate Cygwin's code by trying to loop
>writing smaller amount of data to the pipe so I'd prefer just changing
>the errno if that solves the problem.

Answering my own question: No, git doesn't make any useful decisions based
on the errno.  So, just returning a different errno is not going to make
this work.

So, my options are to:

1) Limit the buffer size to some value like 64 mib and expect the caller to
deal with that.

2) Limit the buffer size to some value like 64 mib and loop in
fhandler_overlapped::write_overlapped until everything has been written.

3) Attempt the WriteFile(), notice the error condition and then do 1) or 2).

2) or 3) would be the most like Linux.  1) is by far the easiest and would
probably solve the current git error case.

Just thinking out loud here.  I think I've convinced myself that I should
go with 3 + 2.

cgf