3.0.7: Possible sshd bug when running as a service and public key offered

[Christopher Mapes]

I sent a two-part emails about this, but somehow only the second one went through.  Apologies; The second part doesn't make much sense by itself.  Following are both parts merged into one along with some edits for clarification.

~

I've found an interesting issue with sshd when running as a service and a any key is offered by the ssh client.  Right after the key is offered, sshd immediately closes the connection.  This happens whether the corresponding key is installed on the server-side in authorized_keys or not, which is strange.  In other words, even if the offered key is not used, sshd still immediately closes the connection, and the user is not prompted for a password to spite the fact that authentications that can continue are password and keyboard-interactive.

If there is no key offered (I tested by renaming .ssh/id_rsa to .ssh/_id_rsa temporarily on the client), or if the sshd process on the server is started from the console rather than a service, it continues on to prompt for a password which is the correct behavior.

I'm pretty sure this issue is new to cygwin 3.x.  We've been using cygwin 2.x in our environment for years without issue.  This is the first time we're using 3.x, and the first time we've had this issue.  3.x does run the service as LocalSystem (SYSTEM) by default, so I went with that.  Previous 2.x installs we had run it under cyg_server.

Of course, it could be something in our environment, for example these servers are in one domain, but the users logging in are in a different trusted domain.  They are running 2016 whereas the previous servers were running 2003, 2008 or 2012.  There may be some group policy rule causing an issue.  Windows Defender could be causing problems, but I'm pretty sure it isn't because I have added the cygwin directory to the exclusion list.  It does happen on every server I installed cygwin 3.x on (3 of them so far).  

A full log of the failure from the client side is attached as sshd-client-fail-example.log.  Here is a snippet:
...
debug1: Trying private key: /root/.ssh/id_rsa
debug1: read PEM private key done: type RSA
debug3: sign_and_send_pubkey
debug2: we sent a publickey packet, wait for reply
Connection closed by 1.2.3.4

I tried to get a debug log of the failure from the server side by running sshd with the -D -ddd option interactively, but doing this makes the issue go away.  I was able to get the debug log by modifying the service parameters from just "-D" to "-D -ddd" in the registry then restarting the service.  When I do that, the issue still occurs.  Here is a snippet of the server-side log:
...
debug2: input_userauth_request: try method publickey [preauth]
debug2: userauth_pubkey: valid user user-xx attempting public key ssh-rsa rsa-data-xx [preauth]
debug3: userauth_pubkey: have ssh-rsa signature for RSA SHA256:sha-data-xx [preauth]
debug3: mm_key_allowed entering [preauth]
debug3: mm_request_send entering: type 22 [preauth]
debug3: mm_key_allowed: waiting for MONITOR_ANS_KEYALLOWED [preauth]
debug3: mm_request_receive_expect entering: type 23 [preauth]
debug3: mm_request_receive entering [preauth]
debug3: mm_request_receive entering
debug3: monitor_read: checking request 22
debug3: mm_answer_keyallowed entering
debug3: mm_answer_keyallowed: key_from_blob: 0x60008fcf0
debug1: temporarily_use_uid: 1074753138/1073742337 (e=18/18)
seteuid 1074753138: No such device or address
debug1: do_cleanup
debug1: Killing privsep child 302

Based on some info I read on the cygwin site, I figured from the "seteuid: * No such device or address" error that it was probably trying to do an LSA lookup for my account and somehow bombing out when it could not find it.  So, stored my password in LSA with passwd -R, and voila -- it works: The offered certificate is gracefully rejected, and ssh continues on to prompt me for my password.  A full log from the client side of this success is attached as sshd-client-success-example.log.  Here is a snippet:
...
debug1: Trying private key: /root/.ssh/id_rsa
debug1: read PEM private key done: type RSA
debug3: sign_and_send_pubkey
debug2: we sent a publickey packet, wait for reply
debug1: Authentications that can continue: publickey,password,keyboard-interactive
...
debug3: remaining preferred:
debug3: authmethod_is_enabled password
debug1: Next authentication method: password
user-xx@fqdn-xx's password:

Storing the obfuscated password in the LSA presents security issues as this server is shared by a lot of users and accessible from a fairly large network, so it is not a viable work-around.

cygcheck.out also attached.  I've and changed all the ips and host/user/domain names in the files for security purposes.

If you need any further details, let me know.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cygcheck.out
Type: application/octet-stream
Size: 26275 bytes
Desc: cygcheck.out
URL: <http://cygwin.com/pipermail/cygwin/attachments/20191205/677944f3/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sshd-client-fail-example.log
Type: application/octet-stream
Size: 6363 bytes
Desc: sshd-client-fail-example.log
URL: <http://cygwin.com/pipermail/cygwin/attachments/20191205/677944f3/attachment-0001.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sshd-client-success-example.log
Type: application/octet-stream
Size: 7211 bytes
Desc: sshd-client-success-example.log
URL: <http://cygwin.com/pipermail/cygwin/attachments/20191205/677944f3/attachment-0002.obj>
-------------- next part --------------

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple