Wget ignores robot.txt entry

[Max Bowsher]

Randall R Schulz wrote:
> Wget is orphaned? That's bad news, since it seems to have it all over
> cURL. (Sure. Go ahead and prove me wrong. I might as well get it over
> with... for now.)

cURL doesn't do recursive web-suck (does it?)

Yes, wget is orphaned. There's no one on the wget mailing list who has CVS
write access. Which is a great shame, as there are a surprising amount of
patches being sent in.


Max.


--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/