How do different tokenizers perform on downstream tasks in scriptio continua languages?: A case study in Japanese

Takuro Fujii | Koki Shibata | Atsuki Yamaguchi | Terufumi Morishita | Yasuhiro Sogawa |

Paper Details:

Month: July
Year: 2023
Location: Toronto, Canada
Venue: ACL |