notbugAs an Amazon Associate I earn from qualifying purchases.
Want a good read? Try FreeBSD Mastery: Jails (IT Mastery Book 15)
Port details
py-scrapy High level scraping and web crawling framework
1.6.0_2 www on this many watch lists=1 search for ports that depend on this port Find issues related to this port Report an issue related to this port
1.6.0Version of this port present on the latest quarterly branch.
Maintainer: skreuzer@FreeBSD.org search for ports maintained by this maintainer
Port Added: 2009-12-14 02:58:59
Last Update: 2019-04-10 07:18:32
SVN Revision: 498530
Also Listed In: python
License: BSD3CLAUSE
Scrapy is a high level scraping and web crawling framework for writing
spiders to crawl and parse web pages for all kinds of purposes, from
information retrieval to monitoring or testing web sites.

WWW: https://scrapy.org/
SVNWeb : Homepage

There is no configure plist information for this port.

Dependency lines:
  • py36-Scrapy>0:www/py-scrapy

To install the port: cd /usr/ports/www/py-scrapy/ && make install clean
To add the package: pkg install py36-Scrapy

PKGNAME: py36-Scrapy

Package flavors (<flavor>: <package>)
  • py36: py36-Scrapy
  • py27: py27-Scrapy

distinfo:

TIMESTAMP = 1549229032
SHA256 (Scrapy-1.6.0.tar.gz) = 558dfd10ac53cb324ecd7eefd3eac412161c7507c082b01b0bcd2c6e2e9f0766
SIZE (Scrapy-1.6.0.tar.gz) = 926576


NOTE: FreshPorts displays only information on required and default dependencies. Optional dependencies are not covered.

Build dependencies:
  1. py36-setuptools>0 : devel/py-setuptools@py36
  2. python3.6 : lang/python36
Runtime dependencies:
  1. py36-twisted>=13.1.0 : devel/py-twisted@py36
  2. py36-lxml>0 : devel/py-lxml@py36
  3. py36-sqlite3>0 : databases/py-sqlite3@py36
  4. py36-w3lib>=1.17.0 : www/py-w3lib@py36
  5. py36-cssselect>=0.9 : www/py-cssselect@py36
  6. py36-queuelib>0 : sysutils/py-queuelib@py36
  7. py36-pydispatcher>=2.0.5 : devel/py-pydispatcher@py36
  8. py36-service_identity>0 : security/py-service_identity@py36
  9. py36-six>=1.5.2 : devel/py-six@py36
  10. py36-parsel>=1.5 : textproc/py-parsel@py36
  11. py36-openssl>0 : security/py-openssl@py36
  12. py36-setuptools>0 : devel/py-setuptools@py36
  13. python3.6 : lang/python36
There are no ports dependent upon this port

Configuration Options
===> The following configuration options are available for py36-Scrapy-1.6.0_2:
     SSL=on: SSL protocol support
===> Use 'make config' to modify these settings

USES:
python

Master Sites:
  1. https://files.pythonhosted.org/packages/source/S/Scrapy/
  2. https://pypi.org/packages/source/S/Scrapy/
Notes from UPDATING
These upgrade notes are taken from /usr/ports/UPDATING
  • 2017-11-30
    Affects: */py*
    Author: mat@FreeBSD.org
    Reason: 
      Ports using Python via USES=python are now flavored.  All the py3-* ports
      have been removed and folded into their py-* master ports.
    
      People using Poudriere 3.2+ and binary packages do not have to do anything.
    
      For other people, to build the Python 3.6 version of, for example,
      databases/py-gdbm, you need to run:
    
        # make FLAVOR=py36 install
    
    

Number of commits found: 29

Commit History - (may be incomplete: see SVNWeb link above for full details)
DateByDescription
10 Apr 2019 06:18:32
Original commit files touched by this commit  1.6.0_2
Revision:498530
antoine search for other commits by this committer
Bump a few PORTREVISIONs after r498529

With hat:	portmgr
23 Mar 2019 01:50:31
Original commit files touched by this commit  1.6.0_1
Revision:496622
yuri search for other commits by this committer
textproc/py-scrapy: Fix the dependency version

PR:		235471
Submitted by:	wen
Approved by:	portmgr (port compliance)
16 Mar 2019 12:47:20
Original commit files touched by this commit  1.6.0
Revision:495872
skreuzer search for other commits by this committer
Fix typo in RUN_DEPENDS

Require py-parsel >= 1.2.0
09 Mar 2019 00:55:41
Original commit files touched by this commit  1.6.0
Revision:495093
skreuzer search for other commits by this committer
Update to 1.6.0

PR:		235471
Submitted by:	yuri@
20 Jun 2018 17:05:44
Original commit files touched by this commit  1.3.3_2
Revision:472884
mat search for other commits by this committer
Use PY_FLAVOR for dependencies.

FLAVOR is the current port's flavor, it should not be used outside of
this scope.

Sponsored by:	Absolight
19 Jun 2018 01:41:42
Original commit files touched by this commit  1.3.3_2
Revision:472739
yuri search for other commits by this committer
www/py-scrapy: Add USE_PYTHON=concurrent

Approved by:	portmgr
30 Nov 2017 15:50:34
Original commit files touched by this commit  1.3.3_1
Revision:455210  Sanity Test Failure
mat search for other commits by this committer
Convert Python ports to FLAVORS.

  Ports using USE_PYTHON=distutils are now flavored.  They will
  automatically get flavors (py27, py34, py35, py36) depending on what
  versions they support.

  There is also a USE_PYTHON=flavors for ports that do not use distutils
  but need FLAVORS to be set.  A USE_PYTHON=noflavors can be set if
  using distutils but flavors are not wanted.

  A new USE_PYTHON=optsuffix that will add PYTHON_PKGNAMESUFFIX has been
  added to cope with Python ports that did not have the Python
  PKGNAMEPREFIX but are flavored.

  USES=python now also exports a PY_FLAVOR variable that contains the
(Only the first 15 lines of the commit message are shown above View all of this commit message)
07 Jun 2017 00:25:33
Original commit files touched by this commit  1.3.3_1
Revision:442812
skreuzer search for other commits by this committer
Fix dependencies

PR:		219311
Submitted by:	Eric Camachat <eric@camachat.org>
12 May 2017 01:49:11
Original commit files touched by this commit  1.3.3
Revision:440672
skreuzer search for other commits by this committer
Update to 1.3.3

PR:		219212
Submitted by:	Daniel Ylitalo
19 Jul 2016 13:24:52
Original commit files touched by this commit  1.1.1
Revision:418782
skreuzer search for other commits by this committer
Update to version 1.1.1
01 Apr 2016 14:33:58
Original commit files touched by this commit  0.22.2
Revision:412351
mat search for other commits by this committer
Remove ${PORTSDIR}/ from dependencies, categories v, w, x, y, and z.

With hat:	portmgr
Sponsored by:	Absolight
17 Aug 2015 14:20:41
Original commit files touched by this commit  0.22.2
Revision:394508
mat search for other commits by this committer
Remove UNIQUENAME and LATEST_LINK.

UNIQUENAME was never unique, it was only used by USE_LDCONFIG and now,
we won't have conflicts there.

Use PKGBASE instead of LATEST_LINK in PKGLATESTFILE, the *only* consumer
is pkg-devel, and it works just fine without LATEST_LINK as pkg-devel
has the correct PKGNAME anyway.

Now that UNIQUENAME is gone, OPTIONSFILE is too. (it's been called
OPTIONS_FILE now.)

Reviewed by:	antoine, bapt
Exp-run by:	antoine
Sponsored by:	Absolight
Differential Revision:	https://reviews.freebsd.org/D3336
02 Feb 2015 18:24:48
Original commit files touched by this commit  0.22.2
Revision:378312
skreuzer search for other commits by this committer
Adopt port
03 Jan 2015 08:41:21
Original commit files touched by this commit  0.22.2
Revision:376109
mva search for other commits by this committer
- Convert USE_PYTHON to new USES in www/

With hat:	python@
02 Dec 2014 22:40:57
Original commit files touched by this commit  0.22.2
Revision:373790
bapt search for other commits by this committer
Reset maintainership:
- Lots of maintainers timeout
- unresponsive for almost a year

PR:		195059
25 Nov 2014 21:48:20
Original commit files touched by this commit  0.22.2
Revision:373434
skreuzer search for other commits by this committer
Update to version 0.22.2

PR:		189719
Submitted by:	amim.knabben@gmail.com
Approved by:	maintainer timeout (clsung, 28 weeks)
29 Jan 2014 13:21:32
Original commit files touched by this commit  0.16.5_1
Revision:341733
miwi search for other commits by this committer
- Stage support
- Convert to PYDISTUTILS_AUTOPLIST
- Bump PORTREVISION
13 Jan 2014 21:00:04
Original commit files touched by this commit  0.16.5
Revision:339634
rene search for other commits by this committer
Python cleanup:
- USE_PYTHON* = 2.X -> USE_PYTHON* = 2
- USE_PYTHON* = 2.X+ -> USE_PYTHON* = yes
Reviewed by:	python (mva, rm)
Approved by:	portmgr-lurkers (mat)
08 Dec 2013 12:04:08
Original commit files touched by this commit  0.16.5
Revision:335877
mva search for other commits by this committer
New USES=twisted, to replace the old USE_TWISTED knob.

twisted can be configured with the arguments run or build to replace
the previous USE_TWISTED_RUN and USE_TWISTED_BUILD knobs. The twisted
components can be added as comma-separated arguments. If you previously
wrote
    USE_TWISTED=        yes
    USE_TWISTED=        conch names
    USE_TWISTED_RUN=    yes
you now would write
    USES=       twisted
    USES=       twisted:conch,names
    USES=       twisted:run
20 Sep 2013 23:36:54
Original commit files touched by this commit  0.16.5
Revision:327776
bapt search for other commits by this committer
Add NO_STAGE all over the place in preparation for the staging support (cat:
www)
21 Jun 2013 06:17:24
Original commit files touched by this commit  0.16.5
Revision:321444
clsung search for other commits by this committer
- Update to 0.16.5
12 Feb 2013 12:46:23
Original commit files touched by this commit  0.16.4
Revision:312109
clsung search for other commits by this committer
- Update to 0.16.4
02 Dec 2012 10:58:49
Original commit files touched by this commit  0.16.2
Revision:308097
rm search for other commits by this committer
- update to 0.16.2
- trim Makefile header
- remove indefinite article from COMMENT
- add LICENSE
- remove not needed BUILD_DEPENDS
- depend on TwistedCore via USE_TWISTED_RUN instead of direct dependency on
meta-port
- convert to optionsng
- convert PYTHON_SITELIBDIR usage with PYTHON_PKGNAMEPREFIX

PR:		173661
Submitted by:	rm (myself)
Approved by:	maintainer timeout (2 weeks+)
Feature safe:	yes
18 Nov 2011 02:22:51
Original commit files touched by this commit  0.12.0.2542

clsung search for other commits by this committer
- Unbreak
- take maintainership

PR:             ports/162557
Submitted by:   clsung
Approved by:    maintainer (qingfeng_AT_douban.com)
Feature safe:   yes
11 Jul 2011 20:48:01
Original commit files touched by this commit  0.12.0.2542

pav search for other commits by this committer
- Mark BROKEN: does not compile

Reported by:    pointyhat
28 Jun 2011 04:27:35
Original commit files touched by this commit  0.12.0.2542

delphij search for other commits by this committer
Update to version 0.12.0.2542, change maintainer email address to a new
one.

PR:             ports/158370
Submitted by:   maintainer
19 Mar 2011 12:38:54
Original commit files touched by this commit  0.10.3

miwi search for other commits by this committer
- Get Rid MD5 support
02 Oct 2010 07:55:19
Original commit files touched by this commit  0.10.3

lwhsu search for other commits by this committer
- Update to 0.10.3
- Add sqlite to RUN_DEPENDS
- Add support for SSL and JSON

PR:             ports/149884 (based on)
Submitted by:   lwhsu
Approved by:    maintainer timeout
14 Dec 2009 02:58:44
Original commit files touched by this commit  0.8

wen search for other commits by this committer
Scrapy is a high level scraping and web crawling framework for writing
spiders to crawl and parse web pages for all kinds of purposes, from
information retrieval to monitoring or testing web sites.

WWW: http://scrapy.org/

PR:             ports/136811
Submitted by:   Qing Feng <qingfeng@me.com>

Number of commits found: 29

Login
User Login
Create account

Servers and bandwidth provided by
New York Internet, iXsystems, and RootBSD

This site
What is FreshPorts?
About the authors
Issues
FAQ
How big is it?
The latest upgrade!
Privacy
Forums
Blog
Contact

Search
Enter Keywords:
 
more...

Latest Vulnerabilities
chromiumJun 15
neovimJun 13
phpmyadminJun 13
vimJun 13
vim-consoleJun 13
vim-tinyJun 13
mybbJun 12
linux-flashplayerJun 11
drupal7Jun 08
eximJun 06
py-django111Jun 06
py-django21Jun 06
py-django22Jun 06
ImageMagick7*Jun 04
ImageMagick7-nox11*Jun 04

11 vulnerabilities affecting 77 ports have been reported in the past 14 days

* - modified, not new

All vulnerabilities

Last updated:
2019-06-15 15:07:33


Ports
Home
Categories
Deleted ports
Sanity Test Failures
Newsfeeds

Statistics
Graphs
NEW Graphs (Javascript)
Traffic

Calculated hourly:
Port count 36688
Broken 83
Deprecated 124
Ignore 304
Forbidden 3
Restricted 163
No CDROM 74
Vulnerable 33
Expired 19
Set to expire 95
Interactive 0
new 24 hours 8
new 48 hours14
new 7 days44
new fortnight58
new month202

Servers and bandwidth provided by
New York Internet, iXsystems, and RootBSD
Valid HTML, CSS, and RSS.
Copyright © 2000-2019 Dan Langille. All rights reserved.