This is the mail archive of the
cygwin
mailing list for the Cygwin project.
RE: Cygwin Time Machine
- From: "Richard Campbell" <richard dot campbell at air2web dot com>
- To: <cygwin at cygwin dot com>
- Date: Thu, 10 Feb 2005 14:35:14 -0500
- Subject: RE: Cygwin Time Machine
Peter Castro wrote:
>
> (if there's a way to make wget ignore robots.txt, I'd love to know
it!)
http://www.gnu.org/software/wget/faq.html#3.0
"3.0 How can I make GNU wget ignore a robots.txt file?
Try using:
wget -erobots=off http://your.site.here "
-Richard Campbell
--
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Problem reports: http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ: http://cygwin.com/faq/