notbugAs an Amazon Associate I earn from qualifying purchases.
Want a good read? Try FreeBSD Mastery: Jails (IT Mastery Book 15)
Want a good monitor light? See my photosAll times are UTC
Ukraine
I started running short on disk space for the non-production FreshPorts hosts. This time, I have decided to ask for donations. See my recent blog post which points to my Patreon account.
Port details
py-tiktoken Fast BPE tokeniser for use with OpenAI's models
0.11.0_1 textproc on this many watch lists=2 search for ports that depend on this port Find issues related to this port Report an issue related to this port View this port on Repology. pkg-fallout 0.11.0Version of this port present on the latest quarterly branch.
Maintainer: sunpoet@FreeBSD.org search for ports maintained by this maintainer
Port Added: 2023-08-21 17:15:36
Last Update: 2025-10-03 08:16:50
Commit Hash: 97b5de4
People watching this port, also watch:: jdictionary, py311-Automat, py311-python-gdsii, py311-PyOpenGL, p5-Sane
Also Listed In: python
License: MIT
WWW:
https://github.com/openai/tiktoken
Description:
tiktoken is a fast BPE tokeniser for use with OpenAI's models.
Homepage    cgit ¦ Codeberg ¦ GitHub ¦ GitLab ¦ SVNWeb - no subversion history for this port

Manual pages:
FreshPorts has no man page information for this port.
pkg-plist: as obtained via: make generate-plist
There is no configure plist information for this port.
USE_RC_SUBR (Service Scripts)
  • no SUBR information found for this port
Dependency lines:
  • ${PYTHON_PKGNAMEPREFIX}tiktoken>0:textproc/py-tiktoken@${PY_FLAVOR}
To install the port:
cd /usr/ports/textproc/py-tiktoken/ && make install clean
To add the package, run one of these commands:
  • pkg install textproc/py-tiktoken
  • pkg install py311-tiktoken
NOTE: If this package has multiple flavors (see below), then use one of them instead of the name specified above.
NOTE: This is a Python port. Instead of py311-tiktoken listed in the above command, you can pick from the names under the Packages section.
PKGNAME: py311-tiktoken
Package flavors (<flavor>: <package>)
  • py311: py311-tiktoken
distinfo:
TIMESTAMP = 1757120848 SHA256 (tiktoken-0.11.0.tar.gz) = 3c518641aee1c52247c2b97e74d8d07d780092af79d5911a6ab5e79359d9b06a SIZE (tiktoken-0.11.0.tar.gz) = 37648

Expand this list (62 items)

Collapse this list.

SHA256 (rust/crates/aho-corasick-1.1.3.crate) = 8e60d3430d3a69478ad0993f19238d2df97c507009a52b3c10addcd7f6bcb916 SIZE (rust/crates/aho-corasick-1.1.3.crate) = 183311 SHA256 (rust/crates/autocfg-1.5.0.crate) = c08606f8c3cbf4ce6ec8e28fb0014a2c086708fe954eaa885384a6165172e7e8 SIZE (rust/crates/autocfg-1.5.0.crate) = 18729 SHA256 (rust/crates/bit-set-0.5.3.crate) = 0700ddab506f33b20a03b13996eccd309a48e5ff77d0d95926aa0210fb4e95f1 SIZE (rust/crates/bit-set-0.5.3.crate) = 14470 SHA256 (rust/crates/bit-vec-0.6.3.crate) = 349f9b6a179ed607305526ca489b34ad0a41aed5f7980fa90eb03160b69598fb SIZE (rust/crates/bit-vec-0.6.3.crate) = 19927 SHA256 (rust/crates/bstr-1.12.0.crate) = 234113d19d0d7d613b40e86fb654acf958910802bcceab913a4f9e7cda03b1a4 SIZE (rust/crates/bstr-1.12.0.crate) = 351557 SHA256 (rust/crates/cfg-if-1.0.3.crate) = 2fd1289c04a9ea8cb22300a459a72a385d7c73d3259e2ed7dcb2af674838cfa9 SIZE (rust/crates/cfg-if-1.0.3.crate) = 8719 SHA256 (rust/crates/fancy-regex-0.13.0.crate) = 531e46835a22af56d1e3b66f04844bed63158bc094a628bec1d321d9b4c44bf2 SIZE (rust/crates/fancy-regex-0.13.0.crate) = 85930 SHA256 (rust/crates/heck-0.5.0.crate) = 2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea SIZE (rust/crates/heck-0.5.0.crate) = 11517 SHA256 (rust/crates/indoc-2.0.6.crate) = f4c7245a08504955605670dbf141fceab975f15ca21570696aebe9d2e71576bd SIZE (rust/crates/indoc-2.0.6.crate) = 17164 SHA256 (rust/crates/libc-0.2.175.crate) = 6a82ae493e598baaea5209805c49bbf2ea7de956d50d7da0da1164f9c6d28543 SIZE (rust/crates/libc-0.2.175.crate) = 788728 SHA256 (rust/crates/memchr-2.7.5.crate) = 32a282da65faaf38286cf3be983213fcf1d2e2a58700e808f83f4ea9a4804bc0 SIZE (rust/crates/memchr-2.7.5.crate) = 97603 SHA256 (rust/crates/memoffset-0.9.1.crate) = 488016bfae457b036d996092f6cb448677611ce4449e970ceaf42695203f218a SIZE (rust/crates/memoffset-0.9.1.crate) = 9032 SHA256 (rust/crates/once_cell-1.21.3.crate) = 42f5e15c9953c5e4ccceeb2e7382a716482c34515315f7b03532b8b4e8393d2d SIZE (rust/crates/once_cell-1.21.3.crate) = 34534 SHA256 (rust/crates/portable-atomic-1.11.1.crate) = f84267b20a16ea918e43c6a88433c2d54fa145c92a811b5b047ccbe153674483 SIZE (rust/crates/portable-atomic-1.11.1.crate) = 185506 SHA256 (rust/crates/proc-macro2-1.0.101.crate) = 89ae43fd86e4158d6db51ad8e2b80f313af9cc74f5c0e03ccb87de09998732de SIZE (rust/crates/proc-macro2-1.0.101.crate) = 53886 SHA256 (rust/crates/pyo3-0.24.2.crate) = e5203598f366b11a02b13aa20cab591229ff0a89fd121a308a5df751d5fc9219 SIZE (rust/crates/pyo3-0.24.2.crate) = 1113754 SHA256 (rust/crates/pyo3-build-config-0.24.2.crate) = 99636d423fa2ca130fa5acde3059308006d46f98caac629418e53f7ebb1e9999 SIZE (rust/crates/pyo3-build-config-0.24.2.crate) = 34019 SHA256 (rust/crates/pyo3-ffi-0.24.2.crate) = 78f9cf92ba9c409279bc3305b5409d90db2d2c22392d443a87df3a1adad59e33 SIZE (rust/crates/pyo3-ffi-0.24.2.crate) = 76458 SHA256 (rust/crates/pyo3-macros-0.24.2.crate) = 0b999cb1a6ce21f9a6b147dcf1be9ffedf02e0043aec74dc390f3007047cecd9 SIZE (rust/crates/pyo3-macros-0.24.2.crate) = 8874 SHA256 (rust/crates/pyo3-macros-backend-0.24.2.crate) = 822ece1c7e1012745607d5cf0bcb2874769f0f7cb34c4cde03b9358eb9ef911a SIZE (rust/crates/pyo3-macros-backend-0.24.2.crate) = 72654 SHA256 (rust/crates/quote-1.0.40.crate) = 1885c039570dc00dcb4ff087a89e185fd56bae234ddc7f056a945bf36467248d SIZE (rust/crates/quote-1.0.40.crate) = 31063 SHA256 (rust/crates/regex-1.11.2.crate) = 23d7fd106d8c02486a8d64e778353d1cffe08ce79ac2e82f540c86d0facf6912 SIZE (rust/crates/regex-1.11.2.crate) = 166265 SHA256 (rust/crates/regex-automata-0.4.10.crate) = 6b9458fa0bfeeac22b5ca447c63aaf45f28439a709ccd244698632f9aa6394d6 SIZE (rust/crates/regex-automata-0.4.10.crate) = 622754 SHA256 (rust/crates/regex-syntax-0.8.6.crate) = caf4aa5b0f434c91fe5c7f1ecb6a5ece2130b02ad2a590589dda5146df959001 SIZE (rust/crates/regex-syntax-0.8.6.crate) = 358808 SHA256 (rust/crates/rustc-hash-1.1.0.crate) = 08d43f7aa6b08d49f382cde6a7982047c3426db949b1424bc4b7ec9ae12c6ce2 SIZE (rust/crates/rustc-hash-1.1.0.crate) = 9331 SHA256 (rust/crates/serde-1.0.219.crate) = 5f0e2c6ed6606019b4e29e69dbaba95b11854410e5347d525002456dbbb786b6 SIZE (rust/crates/serde-1.0.219.crate) = 78983 SHA256 (rust/crates/serde_derive-1.0.219.crate) = 5b0276cf7f2c73365f7157c8123c21cd9a50fbbd844757af28ca1f5925fc2a00 SIZE (rust/crates/serde_derive-1.0.219.crate) = 57798 SHA256 (rust/crates/syn-2.0.106.crate) = ede7c438028d4436d71104916910f5bb611972c5cfd7f89b8300a8186e6fada6 SIZE (rust/crates/syn-2.0.106.crate) = 301514 SHA256 (rust/crates/target-lexicon-0.13.2.crate) = e502f78cdbb8ba4718f566c418c52bc729126ffd16baee5baa718cf25dd5a69a SIZE (rust/crates/target-lexicon-0.13.2.crate) = 27923 SHA256 (rust/crates/unicode-ident-1.0.18.crate) = 5a5f39404a5da50712a4c1eecf25e90dd62b613502b7e925fd4e4d19b5c96512 SIZE (rust/crates/unicode-ident-1.0.18.crate) = 47743 SHA256 (rust/crates/unindent-0.2.4.crate) = 7264e107f553ccae879d21fbea1d6724ac785e8c3bfc762137959b5802826ef3 SIZE (rust/crates/unindent-0.2.4.crate) = 7422

Collapse this list.


Packages (timestamps in pop-ups are UTC):
py39-tiktoken
ABIaarch64amd64armv6armv7i386powerpcpowerpc64powerpc64le
FreeBSD:13:latest--------
FreeBSD:13:quarterly------0.5.20.5.2
FreeBSD:14:latest--------
FreeBSD:14:quarterly------0.5.20.5.2
FreeBSD:15:latest--n/a-n/a-0.5.2_1-
FreeBSD:15:quarterly--n/a-n/a---
FreeBSD:16:latest--n/a-n/a---
 

py311-tiktoken
ABIaarch64amd64armv6armv7i386powerpcpowerpc64powerpc64le
FreeBSD:13:latest0.11.00.11.0-0.9.0_40.11.0---
FreeBSD:13:quarterly0.9.0_30.9.0_3-0.9.0_30.9.0_3---
FreeBSD:14:latest0.9.0_40.11.0-0.9.0_40.11.0---
FreeBSD:14:quarterly0.9.0_30.9.0_3-0.9.0_30.9.0_3---
FreeBSD:15:latest0.9.0_40.11.0n/a0.9.0_4n/a---
FreeBSD:15:quarterly--n/a-n/a---
FreeBSD:16:latest-0.9.0_5n/a-n/a---
 

Dependencies
NOTE: FreshPorts displays only information on required and default dependencies. Optional dependencies are not covered.
Build dependencies:
  1. py311-setuptools>=62.4 : devel/py-setuptools@py311
  2. py311-setuptools-rust>=1.5.2 : devel/py-setuptools-rust@py311
  3. py311-wheel>=0 : devel/py-wheel@py311
  4. rust>=1.90.0 : lang/rust
  5. python3.11 : lang/python311
  6. py311-build>=0 : devel/py-build@py311
  7. py311-installer>=0 : devel/py-installer@py311
Test dependencies:
  1. python3.11 : lang/python311
Runtime dependencies:
  1. py311-regex>=2022.1.18 : textproc/py-regex@py311
  2. py311-requests>=2.26.0 : www/py-requests@py311
  3. python3.11 : lang/python311
This port is required by:
for Run
  1. devel/py-ddtrace
  2. misc/chatblade
  3. misc/py-aider-chat
  4. misc/py-langchain-openai
  5. misc/py-litellm
  6. misc/py-openai-whisper
  7. misc/py-transformers

Configuration Options:
===> The following configuration options are available for py311-tiktoken-0.11.0_1: BLOBFILE=off: Blob file support ===> Use 'make config' to modify these settings
Options name:
textproc_py-tiktoken
USES:
cargo python:3.9+
FreshPorts was unable to extract/find any pkg message
Master Sites:
Expand this list (2 items)
Collapse this list.
  1. https://files.pythonhosted.org/packages/source/t/tiktoken/
  2. https://pypi.org/packages/source/t/tiktoken/
Collapse this list.

Number of commits found: 28

Commit History - (may be incomplete: for full details, see links to repositories near top of page)
CommitCreditsLog message
0.11.0_1
03 Oct 2025 08:16:50
commit hash: 97b5de4ffc2a512051f354fbcb2a184f3e002b2fcommit hash: 97b5de4ffc2a512051f354fbcb2a184f3e002b2fcommit hash: 97b5de4ffc2a512051f354fbcb2a184f3e002b2fcommit hash: 97b5de4ffc2a512051f354fbcb2a184f3e002b2f files touched by this commit
Mikael Urankar (mikael) search for other commits by this committer
lang/rust: Bump revisions after 1.90.0

PR:		289709
0.11.0
07 Sep 2025 23:04:12
commit hash: e901c585fbcb554bf49a920d3d53a961f2b2b0aecommit hash: e901c585fbcb554bf49a920d3d53a961f2b2b0aecommit hash: e901c585fbcb554bf49a920d3d53a961f2b2b0aecommit hash: e901c585fbcb554bf49a920d3d53a961f2b2b0ae files touched by this commit
Po-Chuan Hsieh (sunpoet) search for other commits by this committer
textproc/py-tiktoken: Update to 0.11.0

Changes:	https://github.com/openai/tiktoken/releases
		https://github.com/openai/tiktoken/blob/main/CHANGELOG.md
0.9.0_5
01 Sep 2025 08:25:04
commit hash: b21956d7db4b5a41c0e0c57c5cdca79ee89be9c6commit hash: b21956d7db4b5a41c0e0c57c5cdca79ee89be9c6commit hash: b21956d7db4b5a41c0e0c57c5cdca79ee89be9c6commit hash: b21956d7db4b5a41c0e0c57c5cdca79ee89be9c6 files touched by this commit
Mikael Urankar (mikael) search for other commits by this committer
lang/rust: Bump revisions after 1.89.0

PR:		288923
0.9.0_4
03 Jul 2025 08:46:01
commit hash: f3f7e555b06d9a87d63c047ce3e82e936a11f2fecommit hash: f3f7e555b06d9a87d63c047ce3e82e936a11f2fecommit hash: f3f7e555b06d9a87d63c047ce3e82e936a11f2fecommit hash: f3f7e555b06d9a87d63c047ce3e82e936a11f2fe files touched by this commit
Mikael Urankar (mikael) search for other commits by this committer
lang/rust: Bump revisions after 1.88.0

PR:		287766
0.9.0_3
05 Jun 2025 07:52:53
commit hash: 3ed29b818520b4c5691c1c49d13fd6b89c3142efcommit hash: 3ed29b818520b4c5691c1c49d13fd6b89c3142efcommit hash: 3ed29b818520b4c5691c1c49d13fd6b89c3142efcommit hash: 3ed29b818520b4c5691c1c49d13fd6b89c3142ef files touched by this commit
Mikael Urankar (mikael) search for other commits by this committer
lang/rust: Bump revisions after 1.87.0

PR:		286829
0.9.0_2
08 Apr 2025 08:41:13
commit hash: 2a5976aebeefd09d36fd4ac2e7e2eefde5491c0dcommit hash: 2a5976aebeefd09d36fd4ac2e7e2eefde5491c0dcommit hash: 2a5976aebeefd09d36fd4ac2e7e2eefde5491c0dcommit hash: 2a5976aebeefd09d36fd4ac2e7e2eefde5491c0d files touched by this commit
Mikael Urankar (mikael) search for other commits by this committer
lang/rust: Bump revisions after 1.86.0

PR:		285840
0.9.0_1
24 Feb 2025 07:55:52
commit hash: cf7c4b037d7e9cadfa539f3b9cd45396c1a00dfbcommit hash: cf7c4b037d7e9cadfa539f3b9cd45396c1a00dfbcommit hash: cf7c4b037d7e9cadfa539f3b9cd45396c1a00dfbcommit hash: cf7c4b037d7e9cadfa539f3b9cd45396c1a00dfb files touched by this commit
Mikael Urankar (mikael) search for other commits by this committer
lang/rust: Bump revisions after 1.85.0

PR:		284884
0.9.0
19 Feb 2025 11:54:48
commit hash: 4830cc4be071f5a69237855f8cdd7176768adb01commit hash: 4830cc4be071f5a69237855f8cdd7176768adb01commit hash: 4830cc4be071f5a69237855f8cdd7176768adb01commit hash: 4830cc4be071f5a69237855f8cdd7176768adb01 files touched by this commit
Po-Chuan Hsieh (sunpoet) search for other commits by this committer
textproc/py-tiktoken: Update to 0.9.0

Changes:	https://github.com/openai/tiktoken/releases
		https://github.com/openai/tiktoken/blob/main/CHANGELOG.md
0.8.0_3
20 Jan 2025 11:06:50
commit hash: fccaa1b326310b7f33556fe133b509260a90fe7fcommit hash: fccaa1b326310b7f33556fe133b509260a90fe7fcommit hash: fccaa1b326310b7f33556fe133b509260a90fe7fcommit hash: fccaa1b326310b7f33556fe133b509260a90fe7f files touched by this commit
Mikael Urankar (mikael) search for other commits by this committer
lang/rust: Bump revisions after 1.84.0

PR:		283962
0.8.0_2
01 Dec 2024 09:24:18
commit hash: 1d1fd15a3e499eed9e604dac94bab73c2fea4b2fcommit hash: 1d1fd15a3e499eed9e604dac94bab73c2fea4b2fcommit hash: 1d1fd15a3e499eed9e604dac94bab73c2fea4b2fcommit hash: 1d1fd15a3e499eed9e604dac94bab73c2fea4b2f files touched by this commit
Mikael Urankar (mikael) search for other commits by this committer
lang/rust: Bump revisions after 1.83.0

PR:		283000
0.8.0_1
08 Nov 2024 08:24:20
commit hash: ca048c2884dc9822310a87e2dfebb8b1647095e2commit hash: ca048c2884dc9822310a87e2dfebb8b1647095e2commit hash: ca048c2884dc9822310a87e2dfebb8b1647095e2commit hash: ca048c2884dc9822310a87e2dfebb8b1647095e2 files touched by this commit
Mikael Urankar (mikael) search for other commits by this committer
lang/rust: Bump revisions after 1.82.0

PR:		282516
0.8.0
14 Oct 2024 05:25:48
commit hash: b9ec26d2939593c8f7a67c1a5af171d2ef88a61bcommit hash: b9ec26d2939593c8f7a67c1a5af171d2ef88a61bcommit hash: b9ec26d2939593c8f7a67c1a5af171d2ef88a61bcommit hash: b9ec26d2939593c8f7a67c1a5af171d2ef88a61b files touched by this commit
Po-Chuan Hsieh (sunpoet) search for other commits by this committer
textproc/py-tiktoken: Update to 0.8.0

Changes:	https://github.com/openai/tiktoken/releases
		https://github.com/openai/tiktoken/blob/main/CHANGELOG.md
0.7.0_3
10 Sep 2024 11:00:34
commit hash: 89ab81d0e2eca6b298b0efd3f4577d468abf18d5commit hash: 89ab81d0e2eca6b298b0efd3f4577d468abf18d5commit hash: 89ab81d0e2eca6b298b0efd3f4577d468abf18d5commit hash: 89ab81d0e2eca6b298b0efd3f4577d468abf18d5 files touched by this commit
Mikael Urankar (mikael) search for other commits by this committer
lang/rust: Bump revisions after 1.81.0

PR:		281300
0.7.0_2
10 Sep 2024 10:58:07
commit hash: c0d07f1127793ffc97a6c1db78423d0d16bdc5efcommit hash: c0d07f1127793ffc97a6c1db78423d0d16bdc5efcommit hash: c0d07f1127793ffc97a6c1db78423d0d16bdc5efcommit hash: c0d07f1127793ffc97a6c1db78423d0d16bdc5ef files touched by this commit
Mikael Urankar (mikael) search for other commits by this committer
Author: Siva Mahadevan
*/*: remove STRIP_CMD calls in rust based ports

This is not needed after bc4fedc1fec0d359365c04d43be9e32bf101a50e

PR:		246993
Differential Revision:	https://reviews.freebsd.org/D46503
0.7.0_2
26 Aug 2024 08:08:01
commit hash: 7a55f9542531c541f36ae5cf3a58d6ce92ec7bffcommit hash: 7a55f9542531c541f36ae5cf3a58d6ce92ec7bffcommit hash: 7a55f9542531c541f36ae5cf3a58d6ce92ec7bffcommit hash: 7a55f9542531c541f36ae5cf3a58d6ce92ec7bff files touched by this commit
Mikael Urankar (mikael) search for other commits by this committer
lang/rust: Bump revisions after 1.80.1

PR:		280490
0.7.0_1
18 Jun 2024 10:59:14
commit hash: 21476ca8a9b5feb3485a3fa29299c0b447c90350commit hash: 21476ca8a9b5feb3485a3fa29299c0b447c90350commit hash: 21476ca8a9b5feb3485a3fa29299c0b447c90350commit hash: 21476ca8a9b5feb3485a3fa29299c0b447c90350 files touched by this commit
Mikael Urankar (mikael) search for other commits by this committer
lang/rust: Bump revisions after 1.79.0

PR:		279707
0.7.0
19 May 2024 17:15:56
commit hash: 741835f949140931e2525cc378007fbc212c27a3commit hash: 741835f949140931e2525cc378007fbc212c27a3commit hash: 741835f949140931e2525cc378007fbc212c27a3commit hash: 741835f949140931e2525cc378007fbc212c27a3 files touched by this commit
Po-Chuan Hsieh (sunpoet) search for other commits by this committer
textproc/py-tiktoken: Update to 0.7.0

Changes:	https://github.com/openai/tiktoken/releases
		https://github.com/openai/tiktoken/blob/main/CHANGELOG.md
0.6.0_2
13 May 2024 11:03:24
commit hash: 2322b661000b69fcc75fe68fec25e5b4e48c2a8ecommit hash: 2322b661000b69fcc75fe68fec25e5b4e48c2a8ecommit hash: 2322b661000b69fcc75fe68fec25e5b4e48c2a8ecommit hash: 2322b661000b69fcc75fe68fec25e5b4e48c2a8e files touched by this commit
Mikael Urankar (mikael) search for other commits by this committer
lang/rust: Bump revisions after 1.78.0

PR:		278834
0.6.0_1
23 Mar 2024 09:41:46
commit hash: d632fbe55e45559d03a0bef7211913964aeba45acommit hash: d632fbe55e45559d03a0bef7211913964aeba45acommit hash: d632fbe55e45559d03a0bef7211913964aeba45acommit hash: d632fbe55e45559d03a0bef7211913964aeba45a files touched by this commit
Mikael Urankar (mikael) search for other commits by this committer
lang/rust: Bump revisions after 1.77.0

PR:		277786
0.6.0
21 Feb 2024 15:08:04
commit hash: 4e17347e94db300992ec8544fd7cc7705c5d8c18commit hash: 4e17347e94db300992ec8544fd7cc7705c5d8c18commit hash: 4e17347e94db300992ec8544fd7cc7705c5d8c18commit hash: 4e17347e94db300992ec8544fd7cc7705c5d8c18 files touched by this commit
Po-Chuan Hsieh (sunpoet) search for other commits by this committer
textproc/py-tiktoken: Update to 0.6.0

Changes:	https://github.com/openai/tiktoken/releases
		https://github.com/openai/tiktoken/blob/main/CHANGELOG.md
0.5.2_2
19 Feb 2024 11:59:23
commit hash: 7f5a7b0e524c8d32727681a4324905e350dc433ccommit hash: 7f5a7b0e524c8d32727681a4324905e350dc433ccommit hash: 7f5a7b0e524c8d32727681a4324905e350dc433ccommit hash: 7f5a7b0e524c8d32727681a4324905e350dc433c files touched by this commit
Mikael Urankar (mikael) search for other commits by this committer
lang/rust: Bump revisions after 1.76.0

PR:		276920
0.5.2_1
09 Jan 2024 12:15:54
commit hash: 109c3d4629b84972e660b689d169ac0761c1a519commit hash: 109c3d4629b84972e660b689d169ac0761c1a519commit hash: 109c3d4629b84972e660b689d169ac0761c1a519commit hash: 109c3d4629b84972e660b689d169ac0761c1a519 files touched by this commit
Mikael Urankar (mikael) search for other commits by this committer
lang/rust: Bump revisions after 1.75.0

PR:		276033
0.5.2
14 Dec 2023 05:48:32
commit hash: d3c7291cd247fef7bb9faed43d35b38c37dd1c6fcommit hash: d3c7291cd247fef7bb9faed43d35b38c37dd1c6fcommit hash: d3c7291cd247fef7bb9faed43d35b38c37dd1c6fcommit hash: d3c7291cd247fef7bb9faed43d35b38c37dd1c6f files touched by this commit
Po-Chuan Hsieh (sunpoet) search for other commits by this committer
textproc/py-tiktoken: Update to 0.5.2

Changes:	https://github.com/openai/tiktoken/releases
		https://github.com/openai/tiktoken/blob/main/CHANGELOG.md
0.5.1_2
08 Dec 2023 15:04:09
commit hash: b2afd89fb26437c5d9810b131370525352e63f71commit hash: b2afd89fb26437c5d9810b131370525352e63f71commit hash: b2afd89fb26437c5d9810b131370525352e63f71commit hash: b2afd89fb26437c5d9810b131370525352e63f71 files touched by this commit
Mikael Urankar (mikael) search for other commits by this committer
lang/rust: Bump revisions after 1.74.1

PR:		275243
0.5.1_1
24 Oct 2023 10:22:32
commit hash: dd10dc88a7cf54f9aa164bf40cc5e481021401d6commit hash: dd10dc88a7cf54f9aa164bf40cc5e481021401d6commit hash: dd10dc88a7cf54f9aa164bf40cc5e481021401d6commit hash: dd10dc88a7cf54f9aa164bf40cc5e481021401d6 files touched by this commit
Mikael Urankar (mikael) search for other commits by this committer
lang/rust: Bump revisions after 1.73.0

PR:		274499
0.5.1
22 Sep 2023 03:19:31
commit hash: 227e43a0b0373cf0e65d381134ed5561db9a05a8commit hash: 227e43a0b0373cf0e65d381134ed5561db9a05a8commit hash: 227e43a0b0373cf0e65d381134ed5561db9a05a8commit hash: 227e43a0b0373cf0e65d381134ed5561db9a05a8 files touched by this commit
Po-Chuan Hsieh (sunpoet) search for other commits by this committer
textproc/py-tiktoken: Update to 0.5.1

Changes:	https://github.com/openai/tiktoken/releases
		https://github.com/openai/tiktoken/blob/main/CHANGELOG.md
0.4.0_1
08 Sep 2023 09:06:24
commit hash: aba8da2be191461126505c67726fa87345ae63f8commit hash: aba8da2be191461126505c67726fa87345ae63f8commit hash: aba8da2be191461126505c67726fa87345ae63f8commit hash: aba8da2be191461126505c67726fa87345ae63f8 files touched by this commit
Mikael Urankar (mikael) search for other commits by this committer
lang/rust: Bump revisions after 1.72.0

PR:		273331
0.4.0
21 Aug 2023 17:01:58
commit hash: 6c34be9ab1b27410b400e03f8281f03d4f956fa3commit hash: 6c34be9ab1b27410b400e03f8281f03d4f956fa3commit hash: 6c34be9ab1b27410b400e03f8281f03d4f956fa3commit hash: 6c34be9ab1b27410b400e03f8281f03d4f956fa3 files touched by this commit
Po-Chuan Hsieh (sunpoet) search for other commits by this committer
textproc/py-tiktoken: Add py-tiktoken 0.4.0

tiktoken is a fast BPE tokeniser for use with OpenAI's models.

Number of commits found: 28