Preprint finds common multilingual AI metrics can skew cross-language comparisons
A new preprint reports that several normalized metrics used to compare language models across languages can introduce biases linked to tokenization, encoding and orthographic differences. It argues that sentence-level negative log-likelihood over semantically equivalent sequences produces more consistent comparisons.