python/python-tiktoken
SHA256
14
0
Fork 0
forked from pool/python-tiktoken
Code Pull Requests Activity

Accepting request 1225495 from devel:languages:python

Browse Source 
- Update to version 0.8.0:
  * Alas, emulation
  * Attempt to fix emulated builds
  * Partial sync of codebase
  * Add possessive quantifiers to avoid catastrophic backtracking (#258)
  * update README to mention gpt-4o
  * Sync codebase
  * Simplify byte_pair_merge (#255)
  * Inline custom mapping function in _byte_pair_merge
  * Avoid calling byte_pair_encode for existing tokens

  * Performance improvements

OBS-URL: https://build.opensuse.org/request/show/1225495
OBS-URL: https://build.opensuse.org/package/show/openSUSE:Factory/python-tiktoken?expand=0&rev=2
This commit is contained in:
Ana Guerrero
2024-11-21 14:19:04 +00:00
committed by Git OBS Bridge
parent 206a42ca14 d8c0e09456
commit b1c6b8a7ee
7 changed files with 26 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
_service
View File

@@ -3,7 +3,7 @@
<param name="url">https://github.com/openai/tiktoken.git</param> <param name="url">https://github.com/openai/tiktoken.git</param>
<param name="versionformat">@PARENT_TAG@</param> <param name="versionformat">@PARENT_TAG@</param>
<param name="scm">git</param> <param name="scm">git</param>
<param name="revision">0.7.0</param> <param name="revision">0.8.0</param>
<param name="match-tag">*</param> <param name="match-tag">*</param>
<param name="versionrewrite-pattern">v(\d+\.\d+\.\d+)</param> <param name="versionrewrite-pattern">v(\d+\.\d+\.\d+)</param>
<param name="versionrewrite-replacement">\1</param> <param name="versionrewrite-replacement">\1</param>

4
_servicedata Normal file
View File

@@ -0,0 +1,4 @@
<servicedata>
<service name="tar_scm">
<param name="url">https://github.com/openai/tiktoken.git</param>
<param name="changesrevision">63527649963def8c759b0f91f2eb69a40934e468</param></service></servicedata>

14
python-tiktoken.changes
View File

@@ -1,3 +1,17 @@
-------------------------------------------------------------------
Thu Nov 21 09:21:10 UTC 2024 - Dirk Müller <dmueller@suse.com>
- Update to version 0.8.0:
* Alas, emulation
* Attempt to fix emulated builds
* Partial sync of codebase
* Add possessive quantifiers to avoid catastrophic backtracking (#258)
* update README to mention gpt-4o
* Sync codebase
* Simplify byte_pair_merge (#255)
* Inline custom mapping function in _byte_pair_merge
* Avoid calling byte_pair_encode for existing tokens
------------------------------------------------------------------- -------------------------------------------------------------------
Fri Sep 20 06:53:00 UTC 2024 - Daniel Garcia <daniel.garcia@suse.com> Fri Sep 20 06:53:00 UTC 2024 - Daniel Garcia <daniel.garcia@suse.com>

2
python-tiktoken.spec
View File

@@ -18,7 +18,7 @@
%{?sle15_python_module_pythons} %{?sle15_python_module_pythons}
Name: python-tiktoken Name: python-tiktoken
Version: 0.7.0 Version: 0.8.0
Release: 0 Release: 0
Summary: Fast BPE tokeniser for use with OpenAI's models Summary: Fast BPE tokeniser for use with OpenAI's models
License: MIT License: MIT

3
tiktoken-0.7.0.tar.zst
View File

@@ -1,3 +0,0 @@
version https://git-lfs.github.com/spec/v1
oid sha256:de3cb95697d244d4f50e74e086372bc76656e969c5adb24413b0581f70382d9d
size 28819

3
tiktoken-0.8.0.tar.zst Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:36f052fe061b5555044b19b4078e8b7192f6f2a3f908490eb722f1bd3dd51f19
size 30248

4
vendor.tar.zst
View File

@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1 version https://git-lfs.github.com/spec/v1
oid sha256:dbf935f57c364527d839375ae4ff5d58646d86aabc06b00ee4a3d00a7b8a9d2b oid sha256:9bc22788c4bf6265b4be0f02aa5e958bb2c5694050c5ede75888d9757bff9475
size 6264358 size 2748582