Skip to content
HNHacker News·
Not on the current live radar

Generate fonts where every LLM token is the same width

AI summary

A project aims to generate fonts where every LLM token has the same width, transforming a font and tokenizer into a token-space font. The ctok tool, an unofficial approximation of Claude's tokenizer, has limitations including a 256-step search limit and issues with invisible characters affecting token boundaries. Audit results show ctok versions v3, v4.7, and v4.8 have various count/width and semantic failures, indicating it does not reliably agree with Claude's actual tokenization. These issues remain unfixed, suggesting a need for more expert implementation.

Why this one

This report details specific failures in ctok versions v3, v4.7, and v4.8, unlike other accounts that might only mention its unofficial status.

Time & source

Times shown in UTC

Display time zone: UTC

Local time zone unavailable; showing UTC.

IngestedOffset at this time: UTC+0Sep 27, 2026, 00:00 UTC

Ingested
Sep 27, 2026, 00:00
Source type
Unclassified

Full text isn't available here.

Read at source →
Source·Hacker News·ampdot.mesh.host