notbugAs an Amazon Associate I earn from qualifying purchases.
Want a good read? Try FreeBSD Mastery: Jails (IT Mastery Book 15)
Want a good monitor light? See my photosAll times are UTC
Ukraine
This referral link gives you 10% off a Fastmail.com account and gives me a discount on my Fastmail account.

Get notified when packages are built

A new feature has been added. FreshPorts already tracks package built by the FreeBSD project. This information is displayed on each port page. You can now get an email when FreshPorts notices a new package is available for something on one of your watch lists. However, you must opt into that. Click on Report Subscriptions on the right, and New Package Notification box, and click on Update.

Finally, under Watch Lists, click on ABI Package Subscriptions to select your ABI (e.g. FreeBSD:14:amd64) & package set (latest/quarterly) combination for a given watch list. This is what FreshPorts will look for.

Port details
py-dns-crawler Crawler for getting info about DNS domains and services
1.6.3 dns on this many watch lists=0 search for ports that depend on this port Find issues related to this port Report an issue related to this port View this port on Repology. pkg-fallout 1.6.2Version of this port present on the latest quarterly branch.
Maintainer: sunpoet@FreeBSD.org search for ports maintained by this maintainer
Port Added: 2021-06-14 17:48:47
Last Update: 2024-03-23 14:49:27
Commit Hash: 8e0de39
Also Listed In: python
License: GPLv3
WWW:
https://gitlab.nic.cz/adam/dns-crawler
Description:
Despite the name, the crawler gets info for more services than just DNS: - DNS: - all A/AAAA records (for the 2nd level domain and www.subdomain), annotated with GeoIP - TXT records (with SPF and DMARC parsed for easier filtering) - TLSA (for the 2nd level domain and www.subdomain) - MX - DNSSEC validation - nameservers: - each server IP annotated with GeoIP - HOSTNAME.BIND, VERSION.BIND, AUTHORS.BIND and fortune (also for all IPs) - users can add custom additional RRs in the config file - E-mail (for every server from MX): - SMTP server banners (optional, ports are configurable) - TLSA records - Web: - HTTP status & headers (inc. parsed cookies) for ports 80 & 443 on each IP from A/AAAA records - certificate info for HTTPS (optionally with an entire cert chain) - webpage content (optional) - everything of the above is saved for each step in the redirect history -- the crawler follows redirects until it gets a non-redirecting status or hits a configurable limit
Homepage    cgit ¦ Codeberg ¦ GitHub ¦ GitLab ¦ SVNWeb - no subversion history for this port

Manual pages:
FreshPorts has no man page information for this port.
pkg-plist: as obtained via: make generate-plist
There is no configure plist information for this port.
Dependency lines:
  • ${PYTHON_PKGNAMEPREFIX}dns-crawler>0:dns/py-dns-crawler@${PY_FLAVOR}
To install the port:
cd /usr/ports/dns/py-dns-crawler/ && make install clean
To add the package, run one of these commands:
  • pkg install dns/py-dns-crawler
  • pkg install py39-dns-crawler
NOTE: If this package has multiple flavors (see below), then use one of them instead of the name specified above.
NOTE: This is a Python port. Instead of py39-dns-crawler listed in the above command, you can pick from the names under the Packages section.
PKGNAME: py39-dns-crawler
Package flavors (<flavor>: <package>)
  • py39: py39-dns-crawler
distinfo:
TIMESTAMP = 1710712500 SHA256 (dns-crawler-1.6.3.tar.gz) = 4e794a9e7a978bdfd0f61e62144d4677c72626559632071d79623e917e57de6c SIZE (dns-crawler-1.6.3.tar.gz) = 45095

Packages (timestamps in pop-ups are UTC):
py39-dns-crawler
ABIaarch64amd64armv6armv7i386powerpcpowerpc64powerpc64le
FreeBSD:13:latest1.6.21.6.2-1.6.21.6.2---
FreeBSD:13:quarterly1.6.21.6.21.4.9_11.6.21.6.2-1.6.21.6.2
FreeBSD:14:latest1.6.21.6.21.4.91.6.21.6.21.4.9_1-1.4.9_1
FreeBSD:14:quarterly1.6.21.6.2-1.6.21.6.2-1.6.21.6.2
FreeBSD:15:latest1.6.21.6.2n/a1.6.2n/a-1.6.2-
FreeBSD:15:quarterly--n/a-n/a---
Dependencies
NOTE: FreshPorts displays only information on required and default dependencies. Optional dependencies are not covered.
Build dependencies:
  1. py39-setuptools>=41 : devel/py-setuptools@py39
  2. py39-setuptools-git-versioning>=0<2 : devel/py-setuptools-git-versioning@py39
  3. py39-wheel>=0 : devel/py-wheel@py39
  4. python3.9 : lang/python39
  5. py39-build>=0 : devel/py-build@py39
  6. py39-installer>=0 : devel/py-installer@py39
Test dependencies:
  1. python3.9 : lang/python39
Runtime dependencies:
  1. py39-asn1crypto>=1.5.1 : devel/py-asn1crypto@py39
  2. py39-cert-human>=1.0.7 : security/py-cert-human@py39
  3. py39-checkdmarc>=4.3.1 : mail/py-checkdmarc@py39
  4. py39-dnspython>=2.3.0,1 : dns/py-dnspython@py39
  5. py39-ecdsa>=0.18.0 : security/py-ecdsa@py39
  6. py39-forcediphttpsadapter>=1.0.2 : www/py-forcediphttpsadapter@py39
  7. py39-GeoIP2>=3.0.0 : net/py-GeoIP2@py39
  8. py39-hstspreload>=0 : www/py-hstspreload@py39
  9. py39-idna>=2.10 : dns/py-idna@py39
  10. py39-pyaml>=21.10.1 : textproc/py-pyaml@py39
  11. py39-pyicu>=2.10.2 : devel/py-pyicu@py39
  12. py39-openssl>=23.1.1,1 : security/py-openssl@py39
  13. py39-python-magic>=0.4.27 : devel/py-python-magic@py39
  14. py39-redis>=4.5.4 : databases/py-redis@py39
  15. py39-requests>=2.23.0 : www/py-requests@py39
  16. py39-requests-toolbelt>=0.10.1 : www/py-requests-toolbelt@py39
  17. py39-rq>=1.13.0 : devel/py-rq@py39
  18. py39-cryptography>=41.0.7,1 : security/py-cryptography@py39
  19. python3.9 : lang/python39
There are no ports dependent upon this port

Configuration Options:
No options to configure
Options name:
dns_py-dns-crawler
USES:
python
FreshPorts was unable to extract/find any pkg message
Master Sites:
Expand this list (2 items)
Collapse this list.
  1. https://files.pythonhosted.org/packages/source/d/dns-crawler/
  2. https://pypi.org/packages/source/d/dns-crawler/
Collapse this list.

Number of commits found: 14

Commit History - (may be incomplete: for full details, see links to repositories near top of page)
CommitCreditsLog message
1.6.3
23 Mar 2024 14:49:27
commit hash: 8e0de3997ecd493343a3adfb425413fb679a42c7commit hash: 8e0de3997ecd493343a3adfb425413fb679a42c7commit hash: 8e0de3997ecd493343a3adfb425413fb679a42c7commit hash: 8e0de3997ecd493343a3adfb425413fb679a42c7 files touched by this commit
Po-Chuan Hsieh (sunpoet) search for other commits by this committer
dns/py-dns-crawler: Update to 1.6.3

Changes:	https://gitlab.nic.cz/adam/dns-crawler/-/blob/master/CHANGELOG.md
1.6.2
25 Dec 2023 09:00:24
commit hash: bab68edeb4b6d338c2810d030519dcb7fd674618commit hash: bab68edeb4b6d338c2810d030519dcb7fd674618commit hash: bab68edeb4b6d338c2810d030519dcb7fd674618commit hash: bab68edeb4b6d338c2810d030519dcb7fd674618 files touched by this commit
Po-Chuan Hsieh (sunpoet) search for other commits by this committer
dns/py-dns-crawler: Update to 1.6.2

- Convert to USE_PYTHON=pep517

Changes:	https://gitlab.nic.cz/adam/dns-crawler/-/blob/master/CHANGELOG.md
1.5.8
05 Oct 2023 02:00:22
commit hash: e047381479ea35ba16474ef43690a765b24744c0commit hash: e047381479ea35ba16474ef43690a765b24744c0commit hash: e047381479ea35ba16474ef43690a765b24744c0commit hash: e047381479ea35ba16474ef43690a765b24744c0 files touched by this commit
Po-Chuan Hsieh (sunpoet) search for other commits by this committer
dns/py-dns-crawler: Update to 1.5.8

Changes:	https://gitlab.nic.cz/adam/dns-crawler/-/blob/master/CHANGELOG.md
1.4.9_1
29 Sep 2023 14:19:56
commit hash: 3754fc5f190b9a31f617d662bb35fd0f96ee8088commit hash: 3754fc5f190b9a31f617d662bb35fd0f96ee8088commit hash: 3754fc5f190b9a31f617d662bb35fd0f96ee8088commit hash: 3754fc5f190b9a31f617d662bb35fd0f96ee8088 files touched by this commit
Po-Chuan Hsieh (sunpoet) search for other commits by this committer
*: Replace USES=pycryptography* with USE_PYTHON=cryptography{,_build,_test}

- Introduce USE_PYTHON=cryptography{,_build,_test}
- Switch all 96 ports from USES=pycryptography to with
USE_PYTHON=cryptography{,_build,_test}
- Remove Mk/Uses/pycryptography.mk

PR:		273727
Approved by:	tcberner (portmgr)
Exp-run by:	antoine
1.4.9_1
31 Jul 2023 16:40:09
commit hash: 8465fa106260644d72c9aad5b93c355f170f0f7dcommit hash: 8465fa106260644d72c9aad5b93c355f170f0f7dcommit hash: 8465fa106260644d72c9aad5b93c355f170f0f7dcommit hash: 8465fa106260644d72c9aad5b93c355f170f0f7d files touched by this commit
Po-Chuan Hsieh (sunpoet) search for other commits by this committer
dns/py-dns-crawler: Sort USES
1.4.9_1
22 Jul 2023 07:46:38
commit hash: b28a739884e37c25bb57ea5f157252adb63065d3commit hash: b28a739884e37c25bb57ea5f157252adb63065d3commit hash: b28a739884e37c25bb57ea5f157252adb63065d3commit hash: b28a739884e37c25bb57ea5f157252adb63065d3 files touched by this commit
Tobias C. Berner (tcberner) search for other commits by this committer
framework: convert tree to use USES=pycryptography

A future commit will update to security/py-cryptography will introduce a
rust dependency.

PR:		254853
1.4.9_1
27 Jun 2023 19:34:34
commit hash: 3d9a815d9c5acbb71f4bb07738bdeab4879feacbcommit hash: 3d9a815d9c5acbb71f4bb07738bdeab4879feacbcommit hash: 3d9a815d9c5acbb71f4bb07738bdeab4879feacbcommit hash: 3d9a815d9c5acbb71f4bb07738bdeab4879feacb files touched by this commit
Rene Ladan (rene) search for other commits by this committer
all: remove explicit versions in USES=python for "3.x+"

The logic in USES=python will automatically convert this to 3.8+ by
itself.

Adjust two ports that only had Python 3.7 mentioned but build fine
on Python 3.8 too.

finance/quickfix: mark BROKEN with PYTHON

libtool: compile:  c++ -DHAVE_CONFIG_H -I. -I../.. -I -I. -I.. -I../.. -I../C++
-DLIBICONV_PLUG -DPYTHON_MAJOR_VERSION=3 -Wno-unused-variable
-Wno-maybe-uninitialized -O2 -pipe -DLIBICONV_PLUG -fstack-protector-strong
-fno-strict-aliasing -DLIBICONV_PLUG -Wall -ansi
-Wno-unused-command-line-argument -Wpointer-arith -Wwrite-strings
-Wno-overloaded-virtual -Wno-deprecated-declarations -Wno-deprecated -std=c++0x
-MT _quickfix_la-QuickfixPython.lo -MD -MP -MF
.deps/_quickfix_la-QuickfixPython.Tpo -c QuickfixPython.cpp  -fPIC -DPIC -o
.libs/_quickfix_la-QuickfixPython.o
warning: unknown warning option '-Wno-maybe-uninitialized'; did you mean
'-Wno-uninitialized'? [-Wunknown-warning-option]
QuickfixPython.cpp:175:11: fatal error: 'Python.h' file not found
          ^~~~~~~~~~
1 warning and 1 error generated.

Reviewed by:	portmgr, vishwin, yuri
Differential Revision:	<https://reviews.freebsd.org/D40568>
1.4.9_1
23 Apr 2023 09:09:58
commit hash: 8d3e020ed032a8db00208994d0db646de7dc6f5bcommit hash: 8d3e020ed032a8db00208994d0db646de7dc6f5bcommit hash: 8d3e020ed032a8db00208994d0db646de7dc6f5bcommit hash: 8d3e020ed032a8db00208994d0db646de7dc6f5b files touched by this commit
Gerald Pfeifer (gerald) search for other commits by this committer
*: Bump PORTREVISIONs for math/mpc update to 1.3.1
1.4.9
30 Jan 2023 13:00:06
commit hash: 5d485f62c6dbdf09c8801c196f00f43a1d54cce4commit hash: 5d485f62c6dbdf09c8801c196f00f43a1d54cce4commit hash: 5d485f62c6dbdf09c8801c196f00f43a1d54cce4commit hash: 5d485f62c6dbdf09c8801c196f00f43a1d54cce4 files touched by this commit
Po-Chuan Hsieh (sunpoet) search for other commits by this committer
dns/py-dns-crawler: Update version requirement of RUN_DEPENDS
1.4.9
11 Jan 2023 15:58:34
commit hash: 77d68471aded6907dbbcb5ef50d557e4c69db656commit hash: 77d68471aded6907dbbcb5ef50d557e4c69db656commit hash: 77d68471aded6907dbbcb5ef50d557e4c69db656commit hash: 77d68471aded6907dbbcb5ef50d557e4c69db656 files touched by this commit
Dmitry Marakasov (amdmi3) search for other commits by this committer
*/*: rename CHEESESHOP to PYPI in MASTER_SITES

PR:			267994
Differential revision:	D37518
Approved by:		bapt
07 Sep 2022 21:58:51
commit hash: fb16dfecae4a6efac9f3a78e0b759fb7a3c53de4commit hash: fb16dfecae4a6efac9f3a78e0b759fb7a3c53de4commit hash: fb16dfecae4a6efac9f3a78e0b759fb7a3c53de4commit hash: fb16dfecae4a6efac9f3a78e0b759fb7a3c53de4 files touched by this commit
Stefan Eßer (se) search for other commits by this committer
Remove WWW entries moved into port Makefiles

Commit b7f05445c00f has added WWW entries to port Makefiles based on
WWW: lines in pkg-descr files.

This commit removes the WWW: lines of moved-over URLs from these
pkg-descr files.

Approved by:		portmgr (tcberner)
1.4.9
07 Sep 2022 21:10:59
commit hash: b7f05445c00f2625aa19b4154ebcbce5ed2daa52commit hash: b7f05445c00f2625aa19b4154ebcbce5ed2daa52commit hash: b7f05445c00f2625aa19b4154ebcbce5ed2daa52commit hash: b7f05445c00f2625aa19b4154ebcbce5ed2daa52 files touched by this commit
Stefan Eßer (se) search for other commits by this committer
Add WWW entries to port Makefiles

It has been common practice to have one or more URLs at the end of the
ports' pkg-descr files, one per line and prefixed with "WWW:". These
URLs should point at a project website or other relevant resources.

Access to these URLs required processing of the pkg-descr files, and
they have often become stale over time. If more than one such URL was
present in a pkg-descr file, only the first one was tarnsfered into
the port INDEX, but for many ports only the last line did contain the
port specific URL to further information.

There have been several proposals to make a project URL available as
a macro in the ports' Makefiles, over time.
(Only the first 15 lines of the commit message are shown above View all of this commit message)
1.4.9
20 Jul 2022 14:21:47
commit hash: 66ee090ccad0160bf913ee776daa9c3bfdab1868commit hash: 66ee090ccad0160bf913ee776daa9c3bfdab1868commit hash: 66ee090ccad0160bf913ee776daa9c3bfdab1868commit hash: 66ee090ccad0160bf913ee776daa9c3bfdab1868 files touched by this commit
Tobias C. Berner (tcberner) search for other commits by this committer
dns: remove 'Created by' lines

A big Thank You to the original contributors of these ports:

  *  Aaron Dalton <aaron@FreeBSD.org>
  *  Akinori MUSHA aka knu <knu@idaemons.org>
  *  Alex Samorukov <samm@freebsd.org>
  *  Alexey Dokuchaev <danfe@FreeBSD.org>
  *  Allan Jude <allanjude@freebsd.org>
  *  Amar Takhar <verm@drunkmonk.net>
  *  Anders Nordby <anders@fix.no>
  *  Andrew Greenwood <greenwood.andy@gmail.com>
  *  Anton Berezin <tobez@FreeBSD.org>
  *  Ashish SHUKLA <ashish@FreeBSD.org>
  *  Attila Nagy <bra@fsn.hu>
(Only the first 15 lines of the commit message are shown above View all of this commit message)
1.4.9
14 Jun 2021 17:32:32
commit hash: 9ff6a0ecc059e6a27b4a589795a77eb76469183dcommit hash: 9ff6a0ecc059e6a27b4a589795a77eb76469183dcommit hash: 9ff6a0ecc059e6a27b4a589795a77eb76469183dcommit hash: 9ff6a0ecc059e6a27b4a589795a77eb76469183d files touched by this commit
Po-Chuan Hsieh (sunpoet) search for other commits by this committer
dns/py-dns-crawler: Add py-dns-crawler 1.4.9

Despite the name, the crawler gets info for more services than just DNS:
- DNS:
  - all A/AAAA records (for the 2nd level domain and www.subdomain), annotated
    with GeoIP
  - TXT records (with SPF and DMARC parsed for easier filtering)
  - TLSA (for the 2nd level domain and www.subdomain)
  - MX
  - DNSSEC validation
  - nameservers:
    - each server IP annotated with GeoIP
    - HOSTNAME.BIND, VERSION.BIND, AUTHORS.BIND and fortune (also for all IPs)
  - users can add custom additional RRs in the config file
- E-mail (for every server from MX):
(Only the first 15 lines of the commit message are shown above View all of this commit message)

Number of commits found: 14