Inside Tiny Aya: What an SAE Sees in a Model Built for 70+ Languages
We trained and released Sparse Autoencoders on all four Tiny Aya regional variants and asked what an SAE sees inside a model built for 70+ languages: a representational-density gap, a mostly-shared feature basis, and an identity-versus-quality null.