Re: cygwin and rsync

["Brett Serkez" <techie at serkez dot net>]

I'm not sure, other than what you've already pointed out, the number of
bytes being close to a magic number, almost like a counter or index is
overflowing.

I'd be inclined to build my own rsync for debugging, start it under gdb,
continue and when it hangs, use control-c  and trace to see where it is
in the program.

As to the environment, which version of Windows are you running?   I'd
like to better understand why on the surface it sounds like we're
installing Cygwin the same way, yet yours runs Windows to Linux and mine
consistently hangs without recompiling without socketpair.

Brett

On Thu, 05 Jan 2006 20:53:23 -0500, "Ken Senior"
<seniork@gmail.com> said:
> Let me say again, THANKS for helping!
>
> Well, the screen has stayed frozen for some 10 minutes.  Some
> observations:
>
> 1.  Killing the rsync process (CTRL-C) on [LOCAL/Linux] does not kill
>     the process on [REMOTE/Windows].  I have to kill the process
>     manually on [REMOTE].
>
> 2.  The process seems to stop at one or two of the same files each
>     time: run1 : hung at file A        bytes transferred: 2,392,064
>     run2 : hung at file B        bytes transferred: 2,408,448 run3 :
>     hung at file B        bytes transferred: 2,408,448 run4 : hung
>     back at file A   bytes transferred: 2,408,448 run5 : hung at file
>     A        bytes transferred: 2,408,448 Deleted all previously-
>     transferred files and then: run6 : hung at file A        bytes
>     transferred: 2,359,296 run7 : hung at file A        bytes
>     transferred: 2,359,296 run8 : hung at file A        bytes
>     transferred: 2,359,296 Deleted all previously-transferred files
>     and then: run9 : hung at file A        bytes transferred:
>     2,359,296 Deleted all previously-transferred files and then:
>     run10: hung at file A        bytes transferred: 2,359,296
>
> Notice that some of these have interesting differences: 2,359,296 -
> 2,392,064=32,768=2^15 2,408,448 - 2,392,064 = 16,384 = 2^14.  Files A
> and B are each small files as are files adjacent to them.
>
> 3.  To answer your earlier question, I installed cygwin and added the
>     rsync and openssh which were part of the distribution but were not
>     installed by default.  I did not compile my own.
>
> Brett Serkez wrote:
>
> >Ken,
> >
> >The rsync protocol actually does check-suming of blocks to
> >efficiently detect and transfer files.  While it may look like it is
> >hung, it may actually be just transfering check sums on each file.
> >When I perform long transfers it looks hung from time to time, but my
> >use of the extra v switches helped me better understand the protocol.
> >
> >When it stops at a different directory and file, is it always
> >further along?
> >
> >It may well be hung, but be sure.  Interesting that it works the
> >other way without having to build a custom rsync.  I've found this
> >behavior consistent, but then again, I always install Cygwin with the
> >same set of packages.  Do you do a full install of Cygwin?  I usually
> >perform a default install and just add vi, tcsh, openssh and rsync.
> >
> >Brett
> >
> >On Thu, 05 Jan 2006 19:37:22 -0500, "Ken Senior"
> ><seniork@gmail.com> said:
> >
> >
> >>Thanks Brett for the quick reply.  The multiple -v is a handy thing
> >>to remember.
> >>
> >>In fact this is not my problem though.  It looks like the listing of
> >>files just hangs midstream.  Some local directories are created and
> >>sometimes a few files make it too, but it just hangs.  For example:
> >>
> >>  . . . recv_generator(MATLAB/wavelab/Papers/SpinCycle/cspinfo03.m,-
> >>    8425) recv_generator(MATLAB/wavelab/Papers/SpinCycle/cspinfo04.-
> >>    m,8425) recv_generator(MATLAB/wavelab/Papers/SpinCycle/cspinfo0-
> >>    5.m,8425)
> >>
> >>None of these files were actually transferred and there are plenty
> >>more of these similarly-named files well beyond cspinfo05.m, leaving
> >>me no additional info.  Moreover, there are no special characters or
> >>spaces in these files to suggest a problem in the file or directory
> >>name. Also, each launch seems to stop at a different directory and
> >>file.  Bizarre. I don't know how to use strace or I'd try that.  -K
> >>
> >>
> >>Brett Serkez wrote:
> >>
> >>
> >>
> >>>Ken,
> >>>
> >>>I run rsync both Windows and Linux to Linux, Linux always being the
> >>>'server'.  In my case I found the hang to be up front, before
> >>>secure shell even attempted to access the network and also in my
> >>>case changing from a local socket to a local pipe resolved the
> >>>issue:
> >>>
> >>>http://cygwin.com/ml/cygwin/2005-12/msg01096.html
> >>>
> >>>Have you tried adding more v switches to your command line, like:
> >>>
> >>>rsync -avvz user@[REMOTE/Windows]:/path/to/stuff/ dest/on/local/
> >>>rsync -avvvz user@[REMOTE/Windows]:/path/to/stuff/ dest/on/local/
> >>>rsync -avvvvvz user@[REMOTE/Windows]:/path/to/stuff/ dest/on/local/
> >>>
> >>>Each time you a a v switch, it increases the debug output, I think
> >>>up to 4 or 5.  This would help to narrow the issue as you'll see
> >>>how far your getting in the protocol which might help narrow the
> >>>problem.
> >>>
> >>>Is it at all possible to go the other way?  I know if you are
> >>>willing to build your own rsync with the socketpair() called
> >>>disabled it will work.
> >>>
> >>>Brett
> >>>
> >>>
> >>>
> >>>On Thu, 05 Jan 2006 18:46:04 -0500, "Ken Senior"
> >>><seniork@gmail.com> said:
> >>>
> >>>
> >>>
> >>>
> >>>>Hi.
> >>>>
> >>>>I posted the message listed below to the rsync list, but one of
> >>>>the readers there suggested I post it here indicating that there's
> >>>>a known/common problem in cygwin of data loss in local pipes
> >>>>(whatever that means I am not sure).  I find my rsync command
> >>>>hanging, whether or not I run it over SSH, that is with our
> >>>>without the --rsh='ssh -l username'.  I searched and read the
> >>>>archived mail on the cygwin list archives and though there are six
> >>>>pages of "rsync hanging" issues, I didn't find much help.
> >>>>Unfortunately, the logs are saying nothing and I couldn't get the
> >>>>strace business to work---no doubt because I've never used it. Can
> >>>>anyone here suggest anything?  -Thanks
> >>>>
> >>>>
> >>>>
> >>>>----
> >>>>
> >>>>I just installed the latest version of cygwin (1.5.18-1) from
> >>>>www.cygwin.com on my Windows XP machine.  I have had a lot of
> >>>>success with rsync between Linux boxes but after many months of
> >>>>mixed results I have had much less than perfect luck with rsync
> >>>>from a Linux box [LOCAL] to a windows box [REMOTE]---that is,
> >>>>constant hanging.
> >>>>
> >>>>I have tried to follow the rsync FAQ on using strace to figure out
> >>>>why things are hanging, but I'm not understanding what gets
> >>>>launched where and in what order.  I created the rsync-debug
> >>>>script (below) as suggested, but it's unclear how to use it. Would
> >>>>you guys mind giving a step-by-step on how to get this strace
> >>>>info?
> >>>>
> >>>>For example, let's say on [LOCAL/Linux] I want to issue the
> >>>>command:
> >>>>
> >>>>rsync -avz user@[REMOTE/Windows]:/path/to/stuff/ dest/on/local/
> >>>>
> >>>>Do I first launch the rsync-debug on [REMOTE/Windows]?  Do I
> >>>>modify the above command in order to get things rolling?
> >>>>
> >>>>Cheers and thanks in advance.
> >>>>
> >>>>-Ken
> >>>>
> >>>>----
> >>>>
> >>>>rsync-debug script:
> >>>>
> >>>>ulimit -c unlimited strace -f rsync --daemon --no-detach 2>/tmp/rsync-
> >>>>$$.out
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>--
> >>>>Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
> >>>>Problem reports:       http://cygwin.com/problems.html
> >>>>Documentation:         http://cygwin.com/docs.html FAQ:
> >>>>http://cygwin.com/faq/
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>----------------------------------------------------------------
> >>>Brett C. Serkez, Techie
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >----------------------------------------------------------------
> >Brett C. Serkez, Techie
> >
> >
> >
> >
>
----------------------------------------------------------------
Brett C. Serkez, Techie



--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/