FreshPorts -- The Place For Ports If you buy from Amazon USA, please support us by using this link.
Follow us
Blog
Twitter

I am looking for an LTO tape library. Do you have one to spare?
Port details
webcrawl 1.10_1 www on this many watch lists=6 search for ports that depend on this port
Download web sites without user interaction by following links
Maintained by: demon@FreeBSD.org search for ports maintained by this maintainer
Port Added: unknown
License: not specified in port
WebCrawl is a program designed to download an entire web-site without
user interaction (although an interactive mode is available). It works simply
by starting with a single web page, and following all links from that page
to attempt to recreate the directory structure on the remote server.
SVNWeb : Distfiles Availability : PortsMonThere are no ports dependent upon this port

To install the port: cd /usr/ports/www/webcrawl/ && make install clean
To add the package: pkg install www/webcrawl


Configuration Options
     No options to configure

Master Sites:
  1. ftp://ftp.chg.ru/pub/Linux/sunsite/apps/www/mirroring/
  2. ftp://ftp.cs.tu-berlin.de/pub/linux/Mirrors/sunsite.unc.edu/apps/www/mirroring/
  3. ftp://ftp.cse.cuhk.edu.hk/pub4/Linux/apps/www/mirroring/
  4. ftp://ftp.etsimo.uniovi.es/pub/linux/apps/www/mirroring/
  5. ftp://ftp.icm.edu.pl/pub/Linux/sunsite/apps/www/mirroring/
  6. ftp://ftp.kddlabs.co.jp/Linux/metalab.unc.edu/apps/www/mirroring/
  7. ftp://ftp.lip6.fr/pub/linux/sunsite/apps/www/mirroring/
  8. ftp://ftp.nluug.nl/pub/metalab/apps/www/mirroring/
  9. ftp://ftp.nvg.ntnu.no/pub/mirrors/metalab.unc.edu/apps/www/mirroring/
  10. ftp://ftp.sun.ac.za/pub/mirrors/sunsite.unc.edu/pub/Linux/apps/www/mirroring/
  11. http://distcache.FreeBSD.org/ports-distfiles/
  12. http://www.gtlib.gatech.edu/pub/Linux/apps/www/mirroring/
  13. http://www.ibiblio.org/pub/Linux/apps/www/mirroring/

Number of commits found: 13

Commit History - (may be incomplete: see SVNWeb link above for full details)
DateByDescription
29 Jul 2014 15:49:47
Original commit files touched by this commit  1.10_1
Revision:363346
adamw search for other commits by this committer
Rename www/ patch-xy patches to reflect the files they modify.
13 Mar 2014 14:57:17
Original commit files touched by this commit  1.10_1
Revision:348079
ehaupt search for other commits by this committer
Support staging
20 Sep 2013 23:36:54
Original commit files touched by this commit  1.10_1
Revision:327776
bapt search for other commits by this committer
Add NO_STAGE all over the place in preparation for the staging support (cat:
www)
24 Oct 2011 09:11:38
Original commit files touched by this commit  1.10_1
dougb search for other commits by this committer
The vast majority of pkg-descr files had the following format when they
had both lines:

Author: ...
WWW: ....

So standardize on that, and move them to the end of the file when necessary.

Also fix some more whitespace, and remove more "signature tags" of varying
forms, like -- name, etc.

s/AUTHOR/Author/

A few other various formatting issues
19 Mar 2011 12:38:54
Original commit files touched by this commit  1.10_1
miwi search for other commits by this committer
- Get Rid MD5 support
24 Jan 2006 03:14:23
Original commit files touched by this commit  1.10_1
edwin search for other commits by this committer
SHA256ify

Approved by: krion@
18 Mar 2004 18:01:13
Original commit files touched by this commit  1.10_1
trevor search for other commits by this committer
Add size data, approved by maintainers.
06 Feb 2004 13:12:53
Original commit files touched by this commit  1.10_1
trevor search for other commits by this committer
Use PLIST_FILES (bento-tested, marcus-reviewed).
07 Mar 2003 06:12:57
Original commit files touched by this commit  1.10_1
ade search for other commits by this committer
Clear moonlight beckons.
Requiem mors pacem pkg-comment,
And be calm ports tree.

E Nomini Patri, E Fili, E Spiritu Sancti.
08 Nov 2001 06:48:32
files touched by this commit
demon search for other commits by this committer
Add support for virtual hosts.    
05 Feb 2001 15:33:58
files touched by this commit
olgeni search for other commits by this committer
Some spaces -> tabs for ports/www.    
13 Nov 2000 12:45:41
files touched by this commit
demon search for other commits by this committer
Change MAINTAINER field.   I am now at FreeBSD dot org    
08 Oct 2000 08:11:25
files touched by this commit
asami search for other commits by this committer
Convert category www to new layout.  

Number of commits found: 13

Login
User Login
Create account

Servers and bandwidth provided by
New York Internet, SuperNews, and RootBSD

This site
What is FreshPorts?
About the authors
FAQ
How big is it?
The latest upgrade!
Privacy
Forums
Blog
Contact

Search
Enter Keywords:
 
more...

Latest Vulnerabilities
ntpDec 20
ntp-develDec 20
gitDec 19
otrsDec 16
mod_dav_svnDec 15
subversionDec 15
subversion16Dec 15
subversion17Dec 15
nvidia-driverDec 14
nvidia-driver-173Dec 14
nvidia-driver-304Dec 14
asterisk11Dec 11
bind99Dec 11
xorg-serverDec 10
unboundDec 09

10 vulnerabilities affecting 27 ports have been reported in the past 14 days

* - modified, not new

All vulnerabilities


Ports
Home
Categories
Deleted ports
Sanity Test Failures
Newsfeeds

Statistics
Graphs
NEW Graphs (Javascript)
Traffic

Calculated hourly:
Port count 24435
Broken 144
Deprecated 70
Ignore 397
Forbidden 2
Restricted 207
No CDROM 93
Vulnerable 19
Expired 0
Set to expire 65
Interactive 0
new 24 hours 4
new 48 hours4
new 7 days25
new fortnight41
new month131

Servers and bandwidth provided by
New York Internet, SuperNews, and RootBSD
Valid HTML, CSS, and RSS.
Copyright © 2000-2014 Dan Langille. All rights reserved.