[Fwd: 1.5.11-1: sftp performance problem]

Bob Byrnes byrnes@curl.com
Fri Sep 10 15:55:00 GMT 2004

On Sep 10, 11:01am, vinschen@redhat.com (Corinna Vinschen) wrote:
-- Subject: [Fwd: 1.5.11-1: sftp performance problem]
> this problem is easily reproducible with sftp.  It seems to be in some
> way related to your pipe patch.
> Due to speed considerations, Cygwin's implementation of OpenSSH uses
> pipes for local IPC instead of socketpairs.  I switched back the whole
> OpenSSH suite to use socketpairs and the problem disappears.  Getting
> a 1.4Megs file takes about a minute when using pipes, a split second
> when using socketpairs.  For some reason only receiving files is affected,
> not sending.
> Would you mind to have a look into that?

Sure, I'll look.  I don't use sftp much, but we pump *enormous* amounts
of data through sshd otherwise, so it's odd that I haven't noticed any
performance impact.

> Or is that perhaps a problem solved by one of your upcoming patches?
-- End of excerpt from Corinna Vinschen

No, all of my patches are to avoid various deadlocks.  They don't
directly address performance, although of course I am trying not to make
things slower.

I guess it's possible that my patch interacts badly with some other
recent submissions; I did most of my testing with an earlier snapshot.

On Sep 10, 11:41am, vinschen@redhat.com (Corinna Vinschen) wrote:
-- Subject: Re: [Fwd: 1.5.11-1: sftp performance problem]
> Hmm, I just found that this can't be quite valid anymore.  Using
> socketpairs is way faster, even when using Cygwin 1.5.10.  I digged
> in the openssh-unix-dev ML archives and the decision to switch to
> USE_PIPES has been made 3 1/2 years ago.  I'm wondering if it's
> time to switch back to socketpairs...
-- End of excerpt from Corinna Vinschen

One really annoying consequence of socketpairs for sshd is that
Windows-native (i.e. non-Cygwin) programs don't know how to write
directly to sockets on stdout (or stderr0, so if you try to use
them via ssh, their output silently disappears.  You can see this
by doing ...

    # no output
    cl -help

    # works normally
    cl -help | cat

... with an sshd that uses socketpairs instead of pipes.

I think OpenSSH uses pipes on most platforms by default, so I'm also
somewhat concerned that the socketpair-specific code in OpenSSH might be
a bit crufty.

It's not obvious to me why socketpairs would be inherently faster
than pipes.  Maybe my latest patch exacerbated a longstanding but
not-fully-appreciated performance problem ... I'd certainly be
interested in improving that.


More information about the Cygwin-patches mailing list