notbugAs an Amazon Associate I earn from qualifying purchases.
Want a good read? Try FreeBSD Mastery: Jails (IT Mastery Book 15)
Port details
p5-Scrappy All Powerful Web Harvester, Spider, Scraper fully automated
0.94112090_1 www on this many watch lists=0 search for ports that depend on this port Find issues related to this port Report an issue related to this port 0.94112090Version of this port present on the latest quarterly branch.
Maintainer: perl@FreeBSD.org search for ports maintained by this maintainer
Port Added: 2011-01-27 12:43:55
Last Update: 2018-10-06 14:06:45
SVN Revision: 481365
Also Listed In: perl5
License: not specified in port
Description:
SVNWeb : Homepage
pkg-plist: as obtained via: make generate-plist
Expand this list (29 items)
  1. bin/scrappy
  2. lib/perl5/site_perl/Scrappy.pm
  3. lib/perl5/site_perl/Scrappy/Action.pm
  4. lib/perl5/site_perl/Scrappy/Action/Download.pm
  5. lib/perl5/site_perl/Scrappy/Action/Generate.pm
  6. lib/perl5/site_perl/Scrappy/Action/Help.pm
  7. lib/perl5/site_perl/Scrappy/Logger.pm
  8. lib/perl5/site_perl/Scrappy/Plugin.pm
  9. lib/perl5/site_perl/Scrappy/Plugin/RandomProxy.pm
  10. lib/perl5/site_perl/Scrappy/Project.pm
  11. lib/perl5/site_perl/Scrappy/Project/Document.pm
  12. lib/perl5/site_perl/Scrappy/Queue.pm
  13. lib/perl5/site_perl/Scrappy/Scraper.pm
  14. lib/perl5/site_perl/Scrappy/Scraper/Control.pm
  15. lib/perl5/site_perl/Scrappy/Scraper/Parser.pm
  16. lib/perl5/site_perl/Scrappy/Scraper/UserAgent.pm
  17. lib/perl5/site_perl/Scrappy/Session.pm
  18. lib/perl5/site_perl/auto/share/dist/Scrappy/support/chrome.txt
  19. lib/perl5/site_perl/auto/share/dist/Scrappy/support/explorer.txt
  20. lib/perl5/site_perl/auto/share/dist/Scrappy/support/firefox.txt
  21. lib/perl5/site_perl/auto/share/dist/Scrappy/support/opera.txt
  22. lib/perl5/site_perl/auto/share/dist/Scrappy/support/safari.txt
  23. lib/perl5/site_perl/man/man3/Scrappy.3.gz
  24. lib/perl5/site_perl/man/man3/Scrappy::Logger.3.gz
  25. lib/perl5/site_perl/man/man3/Scrappy::Queue.3.gz
  26. lib/perl5/site_perl/man/man3/Scrappy::Scraper::Control.3.gz
  27. lib/perl5/site_perl/man/man3/Scrappy::Scraper::Parser.3.gz
  28. lib/perl5/site_perl/man/man3/Scrappy::Session.3.gz
  29. lib/perl5/site_perl/man/man1/scrappy.1.gz
Collapse this list.
Dependency lines:
  • p5-Scrappy>0:www/p5-Scrappy
To install the port: cd /usr/ports/www/p5-Scrappy/ && make install clean
To add the package: pkg install p5-Scrappy
PKGNAME: p5-Scrappy
Flavors: there is no flavor information for this port.
distinfo:

Dependencies
NOTE: FreshPorts displays only information on required and default dependencies. Optional dependencies are not covered.
Build dependencies:
  1. p5-File-ShareDir-Install>=0.03 : devel/p5-File-ShareDir-Install
  2. p5-Array-Unique>=0 : devel/p5-Array-Unique
  3. p5-DateTime>=0 : devel/p5-DateTime
  4. p5-DateTime-Format-SQLite>=0 : devel/p5-DateTime-Format-SQLite
  5. p5-File-Find-Rule>=0 : devel/p5-File-Find-Rule
  6. p5-File-ShareDir>=0 : devel/p5-File-ShareDir
  7. p5-File-Slurp>=0 : devel/p5-File-Slurp
  8. p5-File-Util>=0 : devel/p5-File-Util
  9. p5-HTML-Tree>=0 : www/p5-HTML-Tree
  10. p5-Moose>=0 : devel/p5-Moose
  11. p5-Parallel-ForkManager>=0 : devel/p5-Parallel-ForkManager
  12. p5-Template-Toolkit>=0 : www/p5-Template-Toolkit
  13. p5-Try-Tiny>=0 : lang/p5-Try-Tiny
  14. p5-String-TT>=0 : devel/p5-String-TT
  15. p5-URI>=0 : net/p5-URI
  16. p5-WWW-Mechanize>=0 : www/p5-WWW-Mechanize
  17. p5-Web-Scraper>=0 : www/p5-Web-Scraper
  18. p5-YAML-Syck>=0 : textproc/p5-YAML-Syck
  19. perl5>=5.26<5.27 : lang/perl5.26
  20. perl5>=5.26<5.27 : lang/perl5.26
Runtime dependencies:
  1. p5-File-ShareDir-Install>=0.03 : devel/p5-File-ShareDir-Install
  2. p5-Array-Unique>=0 : devel/p5-Array-Unique
  3. p5-DateTime>=0 : devel/p5-DateTime
  4. p5-DateTime-Format-SQLite>=0 : devel/p5-DateTime-Format-SQLite
  5. p5-File-Find-Rule>=0 : devel/p5-File-Find-Rule
  6. p5-File-ShareDir>=0 : devel/p5-File-ShareDir
  7. p5-File-Slurp>=0 : devel/p5-File-Slurp
  8. p5-File-Util>=0 : devel/p5-File-Util
  9. p5-HTML-Tree>=0 : www/p5-HTML-Tree
  10. p5-Moose>=0 : devel/p5-Moose
  11. p5-Parallel-ForkManager>=0 : devel/p5-Parallel-ForkManager
  12. p5-Template-Toolkit>=0 : www/p5-Template-Toolkit
  13. p5-Try-Tiny>=0 : lang/p5-Try-Tiny
  14. p5-String-TT>=0 : devel/p5-String-TT
  15. p5-URI>=0 : net/p5-URI
  16. p5-WWW-Mechanize>=0 : www/p5-WWW-Mechanize
  17. p5-Web-Scraper>=0 : www/p5-Web-Scraper
  18. p5-YAML-Syck>=0 : textproc/p5-YAML-Syck
  19. perl5>=5.26<5.27 : lang/perl5.26
There are no ports dependent upon this port

Configuration Options

USES:

Master Sites:
  1. ftp://ftp.auckland.ac.nz/pub/perl/CPAN/authors/id/A/AW/AWNCORP/
  2. ftp://ftp.cpan.org/pub/CPAN/authors/id/A/AW/AWNCORP/
  3. ftp://ftp.cpan.org/pub/CPAN/modules/by-module/Scrappy/
  4. ftp://ftp.funet.fi/pub/languages/perl/CPAN/authors/id/A/AW/AWNCORP/
  5. ftp://ftp.kddlabs.co.jp/lang/perl/CPAN/authors/id/A/AW/AWNCORP/
  6. ftp://ftp.mirrorservice.org/sites/cpan.perl.org/CPAN/authors/id/A/AW/AWNCORP/
  7. http://backpan.perl.org/authors/id/A/AW/AWNCORP/
  8. http://ftp.jaist.ac.jp/pub/CPAN/authors/id/A/AW/AWNCORP/
  9. http://ftp.twaren.net/Unix/Lang/CPAN/authors/id/A/AW/AWNCORP/
  10. https://cpan.metacpan.org/authors/id/A/AW/AWNCORP/
  11. https://cpan.metacpan.org/modules/by-module/Scrappy/
  12. https://www.cpan.org/authors/id/A/AW/AWNCORP/

Number of commits found: 16

Commit History - (may be incomplete: see SVNWeb link above for full details)
DateByDescription
06 Oct 2018 13:06:45
Original commit files touched by this commit  0.94112090_1
Revision:481365
des search for other commits by this committer
Fix every instance of RUN_DEPENDS:=${BUILD_DEPENDS} in p5 ports, except
for where it resulted in a change in output from build-depends-list or
run-depends-list.

Approved by:	portmgr (adamw)
11 Sep 2018 19:18:54
Original commit files touched by this commit  0.94112090_1
Revision:479566
des search for other commits by this committer
There is no need to delete these files; they will not be included in
the package anyway.
10 Sep 2018 20:27:56
Original commit files touched by this commit  0.94112090_1
Revision:479469
sunpoet search for other commits by this committer
Add PORTSCOUT
10 Sep 2018 13:21:38
Original commit files touched by this commit  0.94112090_1
Revision:479413
des search for other commits by this committer
Use ${RM}.
10 Sep 2018 11:17:00
Original commit files touched by this commit  0.94112090_1
Revision:479390
des search for other commits by this committer
Reassign all of tobez@'s p5 ports to perl@.

Approved by:	tobez
27 May 2018 20:15:20
Original commit files touched by this commit  0.94112090_1
Revision:470993
sunpoet search for other commits by this committer
Update WWW

search.cpan.org is shutting down.
It will redirect to metacpan.org after June 25, 2018.

With hat:	perl
01 Apr 2016 14:33:58
Original commit files touched by this commit  0.94112090_1
Revision:412351
mat search for other commits by this committer
Remove ${PORTSDIR}/ from dependencies, categories v, w, x, y, and z.

With hat:	portmgr
Sponsored by:	Absolight
14 Sep 2015 12:19:50
Original commit files touched by this commit  0.94112090_1
Revision:396892
mat search for other commits by this committer
Make it so that the default Perl is always called perl5.

- Move Perl's man1 files along with its man3 files.
- Move where Perl installs its modules man1 pages.
- Convert the ports installing man1 pages.
- Make different Perl versions installable at the same time.
  Though you should note that only the default version can be used to
  install Perl modules, and the non default Perl versions cannot use the
  modules installed via ports if they contain .so as they are installed
  in a version specific directory.

Reviewed by:	bapt (the Mk bits)
Exp-run by:	antoine
Sponsored by:	Absolight
Differential Revision:	https://reviews.freebsd.org/D3542
26 Nov 2014 13:08:38
Original commit files touched by this commit  0.94112090_1
Revision:373448
mat search for other commits by this committer
Change the way Perl modules are installed, update the default Perl to 5.18.

Before, we had:

  site_perl :           lib/perl5/site_perl/5.18
  site_perl/perl_arch : lib/perl5/site_perl/5.18/mach
  perl_man3 :           lib/perl5/5.18/man/man3

Now we have:

  site_perl : lib/perl5/site_perl
  site_arch : lib/perl5/site_perl/mach/5.18
  perl_man3 : lib/perl5/site_perl/man/man3

Modules without any .so will be installed at the same place regardless of the
(Only the first 15 lines of the commit message are shown above View all of this commit message)
29 Jan 2014 12:23:20
Original commit files touched by this commit  0.94112090
Revision:341724
miwi search for other commits by this committer
- Stage support
20 Sep 2013 23:36:54
Original commit files touched by this commit  0.94112090
Revision:327776
bapt search for other commits by this committer
Add NO_STAGE all over the place in preparation for the staging support (cat:
www)
05 Aug 2013 15:05:30
Original commit files touched by this commit  0.94112090
Revision:324276
az search for other commits by this committer
- Convert to new perl framework
- Trim Makefile header

Approved by:	tobez@ (maintainer)
15 Nov 2012 09:20:14
Original commit files touched by this commit  0.94112090
Revision:307443
tobez search for other commits by this committer
Update to 0.94112090.

Changes: http://cpansearch.perl.org/src/AWNCORP/Scrappy-0.94112090/Changes

Feature safe:	yes
21 Jan 2012 17:40:15
Original commit files touched by this commit  0.62

eadler search for other commits by this committer
At the moment 1385 ports use BUILD_DEPENDS= ${RUN_DEPENDS} and 450
ports use BUILD_DEPENDS:= ${RUN_DEPENDS}. This patch fixes ports that are
currently broken. This is a temporary measure until we organically stop using
:= or someone(s) spend a lot of time changing all the ports over.

Explicit duplication > := > = and this just moves ports one step to the left

Approved by:    portmgr
21 Jun 2011 06:58:49
Original commit files touched by this commit  0.62

az search for other commits by this committer
- Replace ../../authors in MASTER_SITE_SUBDIR with CPAN:CPANID macro.
  See http://wiki.freebsd.org/Perl for details.
- Change maintainership from ports@ to perl@ for ports in this changeset.
- Remove MD5 checksum
- Utilize CPAN macro
27 Jan 2011 12:43:45
Original commit files touched by this commit  0.62

tobez search for other commits by this committer
Add www/p5-Scrappy 0.62, a web crawler Perl module.

Feature safe:   yes

Number of commits found: 16

Login
User Login
Create account

Servers and bandwidth provided by
New York Internet, iXsystems, and RootBSD

This site
What is FreshPorts?
About the authors
Issues
FAQ
How big is it?
The latest upgrade!
Privacy
Forums
Blog
Contact

Search
Enter Keywords:
 
more...

Latest Vulnerabilities
python37Oct 19
py-pillowOct 15
mod_perl2Oct 09
xpdfOct 06
xpdf3Oct 06
xpdf4Oct 06
unboundOct 03
cactiOct 02
gitlab-ceOct 02
gitlab-ceOct 02
ruby24Oct 02
ruby25Oct 02
mongodb34Sep 30
mongodb34Sep 30
mongodb36Sep 30

4 vulnerabilities affecting 12 ports have been reported in the past 14 days

* - modified, not new

All vulnerabilities

Last updated:
2019-10-19 10:52:46


Ports
Home
Categories
Deleted ports
Sanity Test Failures
Newsfeeds

Statistics
Graphs
NEW Graphs (Javascript)
Traffic

Calculated hourly:
Port count 37702
Broken 75
Deprecated 102
Ignore 340
Forbidden 3
Restricted 156
No CDROM 73
Vulnerable 26
Expired 15
Set to expire 73
Interactive 0
new 24 hours 3
new 48 hours6
new 7 days18
new fortnight41
new month143

Servers and bandwidth provided by
New York Internet, iXsystems, and RootBSD
Valid HTML, CSS, and RSS.
Copyright © 2000-2019 Dan Langille. All rights reserved.