kevin/ai-bot
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

/context 增加 token 统计：tiktoken 精确分词与窗口使用百分比

Browse Source
This commit is contained in:
kevin
2026-08-14 19:10:48 +08:00
parent bb5bc619c9
commit ba2ee4b0a0
6 changed files with 124 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
go.mod
+2
View File
@@ -14,11 +14,13 @@ require (
require (
filippo.io/edwards25519 v1.2.0 // indirect
github.com/dlclark/regexp2 v1.10.0 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/mattn/go-isatty v0.0.24 // indirect
github.com/mattn/go-runewidth v0.0.3 // indirect
github.com/ncruces/go-strftime v1.0.0 // indirect
github.com/pkoukk/tiktoken-go v0.1.8 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
github.com/tidwall/match v1.1.1 // indirect
github.com/tidwall/pretty v1.2.1 // indirect