[PATCH] Cygwin: ptsname_r: always return an error number on failure

[Corinna Vinschen]

On Jan 21 17:48, Ken Brown via Cygwin-patches wrote:
> On 1/20/2021 1:00 PM, Ken Brown via Cygwin-patches wrote:
> > Following Linux, return ENOTTY on a bad file descriptor and also set
> > errno to ENOTTY.
> > 
> > Previously 0 was returned and errno was set to EBADF.  Returning 0
> > violates the requirement in
> > https://man7.org/linux/man-pages/man3/ptsname_r.3.html that an error
> > number should be returned on failure.  (That man page doesn't specify
> > setting errno.)
> > 
> > Addresses: https://lists.gnu.org/archive/html/bug-gnulib/2021-01/msg00245.html
> > ---
> >   winsup/cygwin/release/3.2.0 | 3 +++
> >   winsup/cygwin/syscalls.cc   | 5 ++++-
> >   2 files changed, 7 insertions(+), 1 deletion(-)
> > 
> > diff --git a/winsup/cygwin/release/3.2.0 b/winsup/cygwin/release/3.2.0
> > index 43725cec2..f748a9bc8 100644
> > --- a/winsup/cygwin/release/3.2.0
> > +++ b/winsup/cygwin/release/3.2.0
> > @@ -52,3 +52,6 @@ Bug Fixes
> >   - Fix the errno when a path contains .. and the prefix exists but is
> >     not a directory.
> >     Addresses: https://lists.gnu.org/archive/html/bug-gnulib/2021-01/msg00214.html
> > +
> > +- Fix the return value when ptsname_r(3) is called with a bad file descriptor
> > +  Addresses: https://lists.gnu.org/archive/html/bug-gnulib/2021-01/msg00245.html
> > diff --git a/winsup/cygwin/syscalls.cc b/winsup/cygwin/syscalls.cc
> > index 4742c6653..18d9e3f88 100644
> > --- a/winsup/cygwin/syscalls.cc
> > +++ b/winsup/cygwin/syscalls.cc
> > @@ -3364,7 +3364,10 @@ ptsname_r (int fd, char *buf, size_t buflen)
> >     cygheap_fdget cfd (fd);
> >     if (cfd < 0)
> > -    return 0;
> > +    {
> > +      set_errno (ENOTTY);
> > +      return ENOTTY;
> > +    }
> >     return cfd->ptsname_r (buf, buflen);
> >   }
> > 
> 
> I'm not really convinced we should blindly follow Linux here, when EBADF
> would seem to make more sense.  See
> 
>   https://lists.gnu.org/archive/html/bug-gnulib/2021-01/msg00264.html
> 
> Corinna, what's your preference?

EBADF actually makes more sense, as Bruno points out.

Please push, whatever you prefer.


Thanks,
Corinna