Unpacking Tokenization: Evaluating Text Compression and its Correlation with Model Performance

Omer Goldman | Avi Caciularu | Matan Eyal | Kris Cao | Idan Szpektor | Reut Tsarfaty |

Paper Details:

Month: August
Year: 2024
Location: Bangkok, Thailand and virtual meeting
Venue: F | i | n | d | i | n | g | s | - | A | C | L |

Citations

URL

No Citations Yet

Field Of Study