notbugAs an Amazon Associate I earn from qualifying purchases.
Want a good read? Try FreeBSD Mastery: Jails (IT Mastery Book 15)
Want a good monitor light? See my photosAll times are UTC
Ukraine
Port details
py-pytokens Fast, spec compliant Python 3.14+ tokenizer that runs on older Pythons
0.3.0 devel on this many watch lists=0 search for ports that depend on this port Find issues related to this port Report an issue related to this port View this port on Repology. pkg-fallout Package not present on quarterly.This port was created during this quarter. It will be in the next quarterly branch but not the current one.
Maintainer: sunpoet@FreeBSD.org search for ports maintained by this maintainer
Port Added: 2025-11-30 07:21:05
Last Update: 2025-11-30 07:16:57
Commit Hash: c543b0e
Also Listed In: python
License: MIT
WWW:
https://github.com/tusharsadhwani/pytokens
Description:
pytokens provides a fast, spec compliant Python 3.14+ tokenizer that runs on older Pythons.
Homepage    cgit ¦ Codeberg ¦ GitHub ¦ GitLab ¦ SVNWeb - no subversion history for this port

Manual pages:
FreshPorts has no man page information for this port.
pkg-plist: as obtained via: make generate-plist
There is no configure plist information for this port.
USE_RC_SUBR (Service Scripts)
  • no SUBR information found for this port
Dependency lines:
  • ${PYTHON_PKGNAMEPREFIX}pytokens>0:devel/py-pytokens@${PY_FLAVOR}
To install the port:
cd /usr/ports/devel/py-pytokens/ && make install clean
To add the package, run one of these commands:
  • pkg install devel/py-pytokens
  • pkg install py311-pytokens
NOTE: If this package has multiple flavors (see below), then use one of them instead of the name specified above.
NOTE: This is a Python port. Instead of py311-pytokens listed in the above command, you can pick from the names under the Packages section.
PKGNAME: py311-pytokens
Package flavors (<flavor>: <package>)
  • py311: py311-pytokens
distinfo:
TIMESTAMP = 1763853128 SHA256 (pytokens-0.3.0.tar.gz) = 2f932b14ed08de5fcf0b391ace2642f858f1394c0857202959000b68ed7a458a SIZE (pytokens-0.3.0.tar.gz) = 17644

Packages (timestamps in pop-ups are UTC):
py311-pytokens
ABIaarch64amd64armv6armv7i386powerpcpowerpc64powerpc64le
FreeBSD:13:latest0.3.00.3.0--0.3.0n/an/an/a
FreeBSD:13:quarterly-----n/an/an/a
FreeBSD:14:latest0.3.00.3.0--0.3.0---
FreeBSD:14:quarterly----0.3.0---
FreeBSD:15:latest0.3.00.3.0n/a-n/an/a--
FreeBSD:15:quarterly--n/a-n/an/a--
FreeBSD:16:latest0.3.00.3.0n/a-n/an/a--
Dependencies
NOTE: FreshPorts displays only information on required and default dependencies. Optional dependencies are not covered.
Build dependencies:
  1. py311-setuptools>=0 : devel/py-setuptools@py311
  2. py311-wheel>=0 : devel/py-wheel@py311
  3. python3.11 : lang/python311
  4. py311-build>=0 : devel/py-build@py311
  5. py311-installer>=0 : devel/py-installer@py311
Test dependencies:
  1. python3.11 : lang/python311
Runtime dependencies:
  1. python3.11 : lang/python311
This port is required by:
for Run
  1. devel/py-black

Configuration Options:
No options to configure
Options name:
devel_py-pytokens
USES:
python
FreshPorts was unable to extract/find any pkg message
Master Sites:
Expand this list (2 items)
Collapse this list.
  1. https://files.pythonhosted.org/packages/source/p/pytokens/
  2. https://pypi.org/packages/source/p/pytokens/
Collapse this list.

Number of commits found: 1

Commit History - (may be incomplete: for full details, see links to repositories near top of page)
CommitCreditsLog message
0.3.0
30 Nov 2025 07:16:57
commit hash: c543b0e18486c8d1f2b4f2f0e347715a9c2cf877commit hash: c543b0e18486c8d1f2b4f2f0e347715a9c2cf877commit hash: c543b0e18486c8d1f2b4f2f0e347715a9c2cf877commit hash: c543b0e18486c8d1f2b4f2f0e347715a9c2cf877 files touched by this commit
Po-Chuan Hsieh (sunpoet) search for other commits by this committer
devel/py-pytokens: Add py-pytokens 0.3.0

pytokens provides a fast, spec compliant Python 3.14+ tokenizer that runs on
older Pythons.

Number of commits found: 1