Wiola 13M, a Gated Spiral Attention Architecture for Parameter Efficient Small Language Models
arXiv:2608.14604v1 Announce Type: new Abstract: Small language models in the ten to one hundred million parameter range are attractive for on device inference, rapid experimentation, and controlled scientific study, yet most of them reuse the standard transformer block without adaptation to the small scale regime. We present Wiola, a decoder only language model whose novelty…
Lire l'article