cvs over ssh with tcsh (testcase and workaround)
Jay Abel
jabel@flex.com
Sun Apr 9 12:01:00 GMT 2006
>> Sent: Wednesday, April 05, 2006 3:18 AM
>> Subject: Re: cvs over ssh with tcsh
>>
>> > On Tue, 4 Apr 2006, Jay Abel wrote:
>> >
>> > > This is just a ping to see if anyone has noticed problems running
>> > > cvs in command line server mode over ssh with tcsh installed as the
>> > > login shell (in /etc/passwd). After a lot of testing (thanks Rene
>> > > Berber) is seems that the problem only occurs when tcsh is my login
>> > > shell. If I change my login shell to /bin/bash, the problem goes
>> > > away.
>> > >
>> > > versions are as follows:
>> > > [snip]
>> > > I googled 'ssh tcsh' and found that some programs (sftp) don't like
>> > > noisy rc scripts, but
>> >
>> > cvs also doesn't like noisy scripts.
>> >
>> > > tcsh -C /bin/true
>> > >
>> > > produces no output.
>> >
>> > I take it you meant "tcsh -c /bin/true" (lowercase "c"). Your test
>> > above only tests tcsh in non-login mode. Try 'tcsh -l' or '(exec -l
>> > tcsh)' from bash instead.
>> >
>> > > The symptoms of failure that I obsererve fall into two categories:
>> > >
>> > > 1. cvs client reports 'unrecognized command' along with a piece of
>> > > one of the files uploaded displayed as the offending command, or
>> > >
>> > > 2. cvs trace stops dead, with both client and server processes in an
>> > > O state by ps.
>> >
>> > Both of these seem to indicate extra output from somewhere.
>> >
>> > > [snip]
>> > > At this point I'm mostly interested in whether anyone else has ever
>> > > seen this behavior. If not, I'll continue to try to acertain what
>> > > it is about my tcsh configuration which is causing the problem.
>> >
>> > I'd say you have a noisy .login (or /etc/csh.login)... If you rule
>> > that out, we can look for other causes.
>> >
>> > FWIW, simply "ssh user@host cvs server" should show you all of the
>> > output your cvs client sees (and complains about)...
>>
>> Test Number 1: Is tcsh login noisy?
>>
>> [jabel@jabelxp][home/jabel] % tcsh -l < /dev/null | od -ab
>> tcsh -l < /dev/null | od -ab
>> 0000000
>> [jabel@jabelxp][home/jabel] %
>
> Ok, fair enough. FWIW, I was actually wrong -- "ssh u@h command" doesn't
> use a login shell. In fact, even though cvs doesn't *like* the noise, it
> doesn't complain about it in the same way (i.e., putting an "echo Hi" into
> .tcshrc, I get back "cvs update: warning: unrecognized response `Hi' from
> cvs server"). Also, I get this as essentially the first thing after
> "Starting server:", whereas you get this in the middle of updating...
>
>> Test Number 2: does problem exist with CVS?
>>
>> [jabel@jabelxp][jabel/cvstemp] % cvs -t up
>> cvs -t up
>> -> main loop with CVSROOT=:ext:jayabel.com:/home/spring2006
>> -> Starting server: /bin/ssh jayabel.com cvs server
>> -> Sending file `report.pdf' to server
>> -> Sending file `report.tex' to server
>> unrecognized request `OF'
>> -> Lock_Cleanup()
>> -> Lock_Cleanup()
>> [jabel@jabelxp][jabel/cvstemp] %
>>
>> Am I missing something here? If cvs uses a plain old login shell, isn't
>> it subject to the vagaries of things like CTRL-z suspending the job,
>> CTRL-s suspending output, and certain other control and escape sequences
>> sending back all kinds of chatter?
>
> It doesn't -- that was a red herring. However, you didn't try the last
> test I proposed, i.e., "ssh jayabel.com cvs server".
>
>> Is there something obvious I'm supposed to have in my startup scripts
>> that turns all that stuff off if the connection is supposed to be
>> binary, and how does CVS tell ssh it wants a binary shell?
>>
>> I'm currently using /bin/ssh as my CVS_RSH variable, but at least
>> according to the trace output cvs isn't sending any special options.
>> Do I need a wrapper script to create a silent connection?
>
> Nah, I think we're barking up the wrong tree here.
>
> One other observation is that, when I put something in the .rc script, my
> cvs complained about "unrecognized response", whereas yours complains
> about "unrecognized request". This makes me think that it's not the
> server sending something bad, it's the client. You could write a cvs
> wrapper script that does something like
>
> tee CVS_OUT | /bin/cvs "$@"
>
> stick it in the PATH before /bin, and examine ~/CVS_OUT for commands sent
> from the client.
> HTH,
> Igor
THE PROBLEM
Okay, I have made some progress. Highlights are, there is a way to
reproduce the error, and I've attached a possible workaround.
When bash is selected as the login shell, the cvs server process is passed
stdin open in binary mode. When tcsh is the login shell, the cvs server
process is passed stdin open in some kind of text mode where CRLF gets
mapped to LF. For protocol commands, the cvs server doesn't care. When
receiving files, as when performing import, add, update and commit
operations, the server does indeed care. When the client wants to send a
file, it first sends an 'Entry' command followed by the number of bytes to
be sent, then it sends those bytes. The server receives exactly that many
bytes then starts parsing the next command.
If CRLF's are present during the file upload, the number of bytes actually
received is reduced by one for each CRLF in the stream. If file compression
is enabled, on the command line or in or in the client .cvsrc file, then the
occurrence of CRLF's is quasi-random, but repeatable. If compression is
turned off, the contents of the file can be manipulated to cause the error.
Steps to reproduce the problem:
A few necessary preliminaries:
0A. ssh must be installed and properly configured.
0B. tcsh must be installed
0C. all directories used must be binmode, particularly (as used in this
example):
/var/repo/
~/testcase/
/tmp/
THE TESTCASE
1. Turn off file compression (if present in .cvsrc look for a line starting
with cvs, if a -z switch is present change it to -z0.
2. Change user's shell to tcsh by modifying /etc/passwd:
from:
user::......:/bin/bash
to:
user::.....:/bin/tcsh
3. Create an empty repository, e.g.
mkdir /var/repo/
cvs -d ":local:/var/repo" init
4. Create an import directory, copy testcase files, and import them:
% cd
% mkdir testcase
% cd testcase
% cp ??/abnormal.txt .
% cp ??/normal.txt .
% setenv CVS_CLIENT_LOG /home/usr/cvsclient
% cvs -td ":ext:user@localhost:/var/repo" import -m 'broken import' broke
LOCAL IMPORT
unrecognized request `ified normal.txt'
if you don't see the error, verify that your mailer hasn't 'fixed' these
files:
abnormal.txt has a few lines, three of which end in CRLF instead of newline.
normal.txt has a few lines of ordinary unix-like text
and recheck the preconditions at the start of this procedure.
If you do get the error message, take a look at ~/cvsclient.in to see the
commands the client `thinks' it is sending to the server. While there is no
way I know of to intercept the stream to the server, you should be able to
see that the number of characters missing from the command following the bad
file is equal to the number of CRLF line endings.
THE WORKAROUND:
Fortunately, the server code is pretty straightforward. A simple way to get
a clean input stream is just to have cvs reopen its input in binary mode
when it is started as a server. The attached patch modifies server.c to
reopen stdin as "rb". There might be a gentler way to do this, and it may
be that tcsh should do whatever bash is doing, but when cvs is recompiled
with the attached patch, the problem described is fixed. What, if anything
else, is broken, is left as an exercise for the reader. Considering,
though, that pserver and kserver both use sockets (which are binary) and
that they both run the same server code as the command line server once
authentication is complete, I doubt there will be any issues.
To apply the patch:
1. Download unzip, untar cvs-1.11.21
2. ./configure
3. make
(don't try to make check, it's broken)
4. cd /src/
5. patch <??/patch (wherever you put the attached patch file)
6. make
7. cp /bin/cvs /bin/cvs-original
8. cp cvs /bin/cvs
Verify that cvs now behaves as expected. You should now be able to complete
the import, check out the added repository `broke', fiddle with the files
therein, update, and commit.
My point here isn't to say how cvs / cygwin / tcsh should work, only to
point out that they currently aren't playing by the same rulebook. The
workaround is only provided to show that the problem is correctly diagnosed,
I don't have any stake in how this is resolved.
THANKS TO
Rene' Berber for allowing me to test against his server, for help in
figuring out what was wrong, and for guidance in the (hopefully) proper way
to post a problem report.
Igor Peshansky for insight in how to diagnose the problem.
Jay Abel
-------------- next part --------------
A non-text attachment was scrubbed...
Name: patch
Type: application/octet-stream
Size: 1056 bytes
Desc: not available
URL: <http://cygwin.com/pipermail/cygwin/attachments/20060409/b7a1f187/attachment.obj>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: abnormal.txt
URL: <http://cygwin.com/pipermail/cygwin/attachments/20060409/b7a1f187/attachment.txt>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: normal.txt
URL: <http://cygwin.com/pipermail/cygwin/attachments/20060409/b7a1f187/attachment-0001.txt>
-------------- next part --------------
--
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Problem reports: http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ: http://cygwin.com/faq/
More information about the Cygwin
mailing list