Large language models have become everyday collaborators, churning out answers, drafting emails, and even steering decisions in software that touches real lives. Yet beneath the surface lies a stubborn problem: these models can be confidently wrong, and in high-stakes domains—healthcare, autonomous systems, law—that confidence can be dangerous. The field has long chased a principled way to measure and bound what these models don’t know. Now, a team from SRI International’s Neuro-Symbolic Computing and Intelligence Research Group has proposed a compact, scalable recipe for uncertainty that actually scales to the biggest models we have today. It’s called ScalaBL, short for Scalable Bayesian Low Rank Adaptation via Stochastic Variational Subspace Inference. And it does something surprisingly simple and powerful: it learns uncertainty in a tiny corner of the model, then maps those uncertainties onto the full network when it matters.
ScalaBL isn’t a dramatic blueprint-shift for how LLMs are built. It’s a clever reimagining of where we do Bayesian thinking inside a model fine-tuned with LoRA, a parameter-efficient fine-tuning technique. The work, led by Colin Samplawski with colleagues Adam D. Cobb, Manoj Acharya, Ramneet Kaur, and Susmit Jha, tackles a scalability bottleneck head-on. By treating a small, r-dimensional subspace as the space in which uncertainty lives, the researchers can perform probabilistic inference with only a handful of extra parameters per layer—thousands, not millions. The payoff is big: competitive uncertainty quantification on large models, without the memory and compute burden that previously blocked Bayesian approaches at scale. In other words, you get a more honest model without paying a crippling price in resources.
