FreshPorts -- The Place For Ports If you buy from Amazon USA, please support us by using this link.
Follow us
Blog
Twitter

Please give me your LTO-4 or better tape library and I'll put it to good use.
Port details
py-scrapy High level scraping and web crawling framework
0.22.2 www on this many watch lists=1 search for ports that depend on this port Find issues related to this port Report an issue related to this port
Maintainer: skreuzer@FreeBSD.org search for ports maintained by this maintainer
Port Added: 14 Dec 2009 02:58:59
Also Listed In: python
License: BSD3CLAUSE
Scrapy is a high level scraping and web crawling framework for writing
spiders to crawl and parse web pages for all kinds of purposes, from
information retrieval to monitoring or testing web sites.

WWW: http://scrapy.org/
SVNWeb : Homepage : Distfiles Availability : PortsMon

NOTE: FreshPorts displays only information on required and default dependencies. Optional dependencies are not covered.

Build dependencies:
  1. py27-setuptools27>0 : devel/py-setuptools27
  2. python2.7 : lang/python27
  3. python2 : lang/python2
Runtime dependencies:
  1. py27-lxml>=2.0 : devel/py-lxml
  2. py27-sqlite3>0 : databases/py-sqlite3
  3. py27-w3lib>=1.2 : www/py-w3lib
  4. py27-cssselect>=0.9 : www/py-cssselect
  5. py27-queuelib>=1.2.2 : sysutils/py-queuelib
  6. py27-openssl>0 : security/py-openssl
  7. py27-setuptools27>0 : devel/py-setuptools27
  8. python2.7 : lang/python27
  9. python2 : lang/python2
  10. __init__.py : devel/py-twistedCore
  11. __init__.py : www/py-twistedWeb
There are no ports dependent upon this port

To install the port: cd /usr/ports/www/py-scrapy/ && make install clean
To add the package: pkg install www/py-scrapy


Configuration Options
===> The following configuration options are available for py27-Scrapy-0.22.2:
     SSL=on: SSL protocol support
===> Use 'make config' to modify these settings

USES:
python:2 twisted:run,web

Master Sites:
  1. http://distcache.FreeBSD.org/ports-distfiles/
  2. http://pypi.crate.io/packages/source/S/Scrapy/
  3. http://pypi.python.jp/Scrapy/
  4. http://pypi.python.org/packages/source/S/Scrapy/

Number of commits found: 17

Commit History - (may be incomplete: see SVNWeb link above for full details)
DateByDescription
02 Feb 2015 18:24:48
Original commit files touched by this commit  0.22.2
Revision:378312
skreuzer search for other commits by this committer
Adopt port
03 Jan 2015 08:41:21
Original commit files touched by this commit  0.22.2
Revision:376109
mva search for other commits by this committer
- Convert USE_PYTHON to new USES in www/

With hat:	python@
02 Dec 2014 22:40:57
Original commit files touched by this commit  0.22.2
Revision:373790
bapt search for other commits by this committer
Reset maintainership:
- Lots of maintainers timeout
- unresponsive for almost a year

PR:		195059
25 Nov 2014 21:48:20
Original commit files touched by this commit  0.22.2
Revision:373434
skreuzer search for other commits by this committer
Update to version 0.22.2

PR:		189719
Submitted by:	amim.knabben@gmail.com
Approved by:	maintainer timeout (clsung, 28 weeks)
29 Jan 2014 13:21:32
Original commit files touched by this commit  0.16.5_1
Revision:341733
miwi search for other commits by this committer
- Stage support
- Convert to PYDISTUTILS_AUTOPLIST
- Bump PORTREVISION
13 Jan 2014 21:00:04
Original commit files touched by this commit  0.16.5
Revision:339634
rene search for other commits by this committer
Python cleanup:
- USE_PYTHON* = 2.X -> USE_PYTHON* = 2
- USE_PYTHON* = 2.X+ -> USE_PYTHON* = yes
Reviewed by:	python (mva, rm)
Approved by:	portmgr-lurkers (mat)
08 Dec 2013 12:04:08
Original commit files touched by this commit  0.16.5
Revision:335877
mva search for other commits by this committer
New USES=twisted, to replace the old USE_TWISTED knob.

twisted can be configured with the arguments run or build to replace
the previous USE_TWISTED_RUN and USE_TWISTED_BUILD knobs. The twisted
components can be added as comma-separated arguments. If you previously
wrote
    USE_TWISTED=        yes
    USE_TWISTED=        conch names
    USE_TWISTED_RUN=    yes
you now would write
    USES=       twisted
    USES=       twisted:conch,names
    USES=       twisted:run
20 Sep 2013 23:36:54
Original commit files touched by this commit  0.16.5
Revision:327776
bapt search for other commits by this committer
Add NO_STAGE all over the place in preparation for the staging support (cat:
www)
21 Jun 2013 06:17:24
Original commit files touched by this commit  0.16.5
Revision:321444
clsung search for other commits by this committer
- Update to 0.16.5
12 Feb 2013 12:46:23
Original commit files touched by this commit  0.16.4
Revision:312109
clsung search for other commits by this committer
- Update to 0.16.4
02 Dec 2012 10:58:49
Original commit files touched by this commit  0.16.2
Revision:308097
rm search for other commits by this committer
- update to 0.16.2
- trim Makefile header
- remove indefinite article from COMMENT
- add LICENSE
- remove not needed BUILD_DEPENDS
- depend on TwistedCore via USE_TWISTED_RUN instead of direct dependency on
meta-port
- convert to optionsng
- convert PYTHON_SITELIBDIR usage with PYTHON_PKGNAMEPREFIX

PR:		173661
Submitted by:	rm (myself)
Approved by:	maintainer timeout (2 weeks+)
Feature safe:	yes
18 Nov 2011 02:22:51
Original commit files touched by this commit  0.12.0.2542
clsung search for other commits by this committer
- Unbreak
- take maintainership

PR:             ports/162557
Submitted by:   clsung
Approved by:    maintainer (qingfeng_AT_douban.com)
Feature safe:   yes
11 Jul 2011 20:48:01
Original commit files touched by this commit  0.12.0.2542
pav search for other commits by this committer
- Mark BROKEN: does not compile

Reported by:    pointyhat
28 Jun 2011 04:27:35
Original commit files touched by this commit  0.12.0.2542
delphij search for other commits by this committer
Update to version 0.12.0.2542, change maintainer email address to a new
one.

PR:             ports/158370
Submitted by:   maintainer
19 Mar 2011 12:38:54
Original commit files touched by this commit  0.10.3
miwi search for other commits by this committer
- Get Rid MD5 support
02 Oct 2010 07:55:19
Original commit files touched by this commit  0.10.3
lwhsu search for other commits by this committer
- Update to 0.10.3
- Add sqlite to RUN_DEPENDS
- Add support for SSL and JSON

PR:             ports/149884 (based on)
Submitted by:   lwhsu
Approved by:    maintainer timeout
14 Dec 2009 02:58:44
Original commit files touched by this commit  0.8
wen search for other commits by this committer
Scrapy is a high level scraping and web crawling framework for writing
spiders to crawl and parse web pages for all kinds of purposes, from
information retrieval to monitoring or testing web sites.

WWW: http://scrapy.org/

PR:             ports/136811
Submitted by:   Qing Feng <qingfeng@me.com>

Number of commits found: 17

Login
User Login
Create account

Servers and bandwidth provided by
New York Internet, SuperNews, and RootBSD

This site
What is FreshPorts?
About the authors
FAQ
How big is it?
The latest upgrade!
Privacy
Forums
Blog
Contact

Search
Enter Keywords:
 
more...

Latest Vulnerabilities
php5Apr 25
php55Apr 25
php56Apr 25
wordpress*Apr 24
wordpress*Apr 24
wordpress*Apr 24
wordpress*Apr 24
wordpress-zh_CN*Apr 24
wordpress-zh_TW*Apr 24
libtasn1Apr 22
firefoxApr 21
linux-firefoxApr 21
chronyApr 18
sqlite3Apr 18
dulwichApr 17

10 vulnerabilities affecting 33 ports have been reported in the past 14 days

* - modified, not new

All vulnerabilities


Ports
Home
Categories
Deleted ports
Sanity Test Failures
Newsfeeds

Statistics
Graphs
NEW Graphs (Javascript)
Traffic

Calculated hourly:
Port count 24692
Broken 252
Deprecated 49
Ignore 522
Forbidden 1
Restricted 204
No CDROM 95
Vulnerable 21
Expired 0
Set to expire 42
Interactive 0
new 24 hours 1
new 48 hours2
new 7 days15
new fortnight29
new month109

Servers and bandwidth provided by
New York Internet, SuperNews, and RootBSD
Valid HTML, CSS, and RSS.
Copyright © 2000-2014 Dan Langille. All rights reserved.