python/python-tiktoken
SHA256
14
0
Fork 0
forked from pool/python-tiktoken
Code Pull Requests Activity

- Update to version 0.8.0:

Browse Source 
* Alas, emulation
  * Attempt to fix emulated builds
  * Partial sync of codebase
  * Add possessive quantifiers to avoid catastrophic backtracking (#258)
  * update README to mention gpt-4o
  * Sync codebase
  * Simplify byte_pair_merge (#255)
  * Inline custom mapping function in _byte_pair_merge
  * Avoid calling byte_pair_encode for existing tokens
  * Performance improvements

OBS-URL: https://build.opensuse.org/package/show/devel:languages:python/python-tiktoken?expand=0&rev=5
This commit is contained in:
Dirk Mueller
2024-11-21 09:21:27 +00:00
committed by Git OBS Bridge
parent 206a42ca14
commit d8c0e09456
7 changed files with 26 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
python-tiktoken.spec
View File

@@ -18,7 +18,7 @@
%{?sle15_python_module_pythons}
Name: python-tiktoken
Version: 0.7.0
Version: 0.8.0
Release: 0
Summary: Fast BPE tokeniser for use with OpenAI's models
License: MIT