mirror of https://github.com/openclaw/openclaw.git
a5147d4d88
- Use code-point length instead of UTF-16 length in estimateStringChars() so that CJK Extension B+ surrogate pairs (U+20000+) are counted as 1 character, not 2 (fixes ~25% overestimate for rare characters). - Change long-line split step from maxChars to chunking.tokens so that CJK lines are sliced into token-budget-sized segments instead of char-budget-sized segments that produce ~4x oversized chunks. - Add tests for both fixes: surrogate-pair handling and long CJK line splitting. Addresses review feedback from Greptile and Codex bots. |
||
|---|---|---|
| .. | ||
| clawdbot | ||
| memory-host-sdk | ||
| moltbot | ||