How far should a conversation with a machine bend to your taste?
When you ask a modern AI assistant for help—whether it’s to plan a trip, solve a coding problem, or explain a concept—the default is a one‑size‑fits‑all voice. That can feel efficient, but it often misses the subtle, personal rhythms that make human conversations feel alive. A recent study from NAVER Labs Europe, led by Thibaut Thonet and colleagues, asks: what if personalization could be achieved not by asking you to reveal a long, invasive portrait, but by using a compact, questionnaire‑style footprint that stays on your device? Their answer arrives in the form of FaST, a highly parameter‑efficient approach to personalize Large Language Models (LLMs) with limited data. In other words: can a small, well‑designed questionnaire unlock personalized AI without turning you into a dataset?
To explore this question, the researchers built two new datasets that feel almost like prototypes for real life: DnD, a fantasy role‑playing scenario where user preferences come from a character’s voice and actions, and ELIP, a practical setup in which a conversational assistant should answer questions in a user’s preferred style. The bold claim is not merely “we can tailor responses better.” It’s “we can tailor them with surprisingly little data, and in a way that keeps your data private.” The study combines clever feature discovery, a compact reward model, and an iterative generation strategy to tune a model to a specific user—with a footprint so small that it could run on modest hardware or even on a personal device.
Highlights: a) a fixed, user‑agnostic questionnaire can drive meaningful personalization; b) a feature‑aware reward model (FaRM) learns from high‑level, interpretable traits discovered automatically from data; c) FaST outperforms many traditional approaches in both predicting preferences and generating personalized responses, even with fewer than 100 annotations per user; d) the framework emphasizes transparency by explaining which features influence a given response; e) the method shows promise for improving accessibility and fairness by better serving under‑represented user profiles.
The PPALLI problem: a practical form of personal AI with tiny data
The paper introduces Personalized Preference Alignment with Limited Data, or PPALLI, a scenario in which a single, fixed questionnaire—comprising contexts and several candidate responses per context—serves as the sole source of user preference data. A user marks one preferred response for each context. The system then fine‑tunes a generation model to align with those preferences. The twist is in scale and practicality: you don’t annotate dozens or hundreds of scenarios per user. You provide at most a few dozen to under 100 items, yielding a compact set of preference tuples. This is a deliberate choice to respect privacy and reduce the cognitive load on users while still enabling useful personalization.
The value proposition is twofold. First, a fixed questionnaire means the system can learn a user’s preferences without needing new prompts or bespoke per‑user data every time you chat. Second, by learning a model per user, the approach can adapt to individual tastes while keeping sensitive preferences on the user’s device—an attractive property in an era of privacy concerns around cloud‑based personalization.
FaST: turning a data‑sparse problem into a feature‑rich map
FaST, the centerpiece of the study, stands for Feature‑Aware Sampling and Tuning. It is a two‑stage, highly parameter‑efficient pipeline designed to personalize LLMs under data constraints. The beauty of FaST lies in its insistence on high‑level features that the system can interpret, not raw, opaque signals about preferences. Here’s how it unfolds in broad strokes:
1) Feature discovery. From the user‑agnostic questionnaire, an LLM (GPT‑4o in their experiments) generates a set of global features that characterize the different responses. These features are meant to be interpretable and domain‑agnostic; they could capture the tone, verbosity, humor, or technical depth of a response, among many others. Importantly, the features are discovered without peeking at which responses the user chose—the features describe the space of possible responses, not the user’s particular choices.
2) Feature function definitions. Each feature gets a scoring function—called a feature function—that rates a candidate response on that feature on a 1‑to‑5 scale. The scoring itself uses a lightweight prompt to an LLM, designed so that the next token is the numeric score and so the model’s uncertainty about the score is captured. This yields a richer, probabilistic view of how well a response matches a feature than a single point estimate would.
3) A compact reward model. The heart of FaST is FaRM, a Feature‑Aware Reward Model that computes a user‑specific score for a given context and candidate response by combining the per‑feature scores with a small set of learned weights. Concretely, the model computes a weighted sum of feature scores, then feeds that into a softmax over all candidate responses to obtain probabilities. The weights are learned by maximizing the likelihood that the user would pick their own preferred response, over the fixed questionnaire. The weights are the only parameters to learn in FaRM, making the approach highly data‑efficient and less prone to overfitting than full fine‑tuning.
4) Generation via sampling and tuning. Armed with FaRM, FaST generates a pool of candidate responses from the base LLM, ranks them with FaRM, and fine‑tunes the model using the ranked samples. This is done with a family of strategies—SFT (supervised fine‑tuning), DPO (direct preference optimization), and variants like Rejection Sampling Fine‑Tuning (RFT) and Online‑DPO—depending on what works best for the data at hand. The approach emphasizes stability and data efficiency: it forgoes the heavy hardware and hyperparameter tuning required by some reinforcement‑learning‑based methods, in favor of a simpler, more robust loop that can perform well with small datasets.
Two worlds, one idea: DnD and ELIP
To test FaST, the researchers built two datasets that span playful fantasy and practical everyday use. The DnD dataset imagines ten distinct characters in a Dungeons & Dragons‑style universe, facing 129 situations with three possible actions each. For every (character, situation) pair, there is a preferred action, forming a set of 1,290 preference tuples. The ELIP dataset borrows from the ELI5 tradition—100 open‑ended questions drawn from the real world open‑ended Q&A domain, each with four potential responses generated by an LLM, and eight different user profiles that encode preferences along three dimensions: expertise, informativeness, and style. The ELIP data yields 800 (user, question) preference tuples. Together, the datasets show how FaST operates across a synthetic fantasy domain and a grounded, real‑world question‑answering scenario.
These datasets aren’t just “proof of concept” toys. They’re designed to reflect the practical realities of personalization: the question‑answer pair space is fixed and shared, users provide only a limited set of preferences, and the system should perform well even when many contexts are unseen. They also give researchers a transparent way to interpret what the model is prioritizing when personalizing a conversation—an issue that often gets murky in more opaque RLHF pipelines.
From features to meaning: interpretable personalization
One of FaST’s most compelling promises is interpretability. By discovering high‑level features from the questionnaire and learning weights over those features per user, the system can tell you which features most influence a given response. For example, in one demonstrated case, a user profile named “Grog” on the DnD dataset learned weights that valued features like direct combat preparedness, exploration, and a humorous tone, while de-emphasizing abstract or overly technical explanations. In ELIP, a user named “AAA,” who wants child‑friendly, concise, and humorous responses, shows a feature weight pattern that emphasizes metaphorical analogy, humor, relatability, and visual imagery.
The upshot is a form of transparency that is rare in large‑scale personalization pipelines. If your assistant is tailoring its tone to your preferences, FaST can, in principle, reveal which features are guiding those choices. It’s not just “make the AI nicer,” but “here are the knobs we’ve turned and why.” That matters because it gives users a degree of control—an important lever in building trust and preventing unwanted manipulation. The researchers even discuss ethical safeguards, arguing that a feature‑based approach makes it easier to opt out of or adjust the dimensions that matter most to users.
How FaST performs: data efficiency that actually shows up in practice
In the paper’s experiments, FaRM—the feature‑weighted reward model—delivered strong predictive accuracy for user preference on unseen contexts, often outperforming traditional fine‑tuning approaches that require far more data. On the DnD dataset, FaRM achieved top validation and test accuracy among several baselines, even as little as eight or sixteen contexts were used for training. On ELIP, FaRM remained robust across data sizes and backbones, maintaining strong performance when the training data dwindled. The authors highlight a striking efficiency: FaRM weights for eight ELIP users were learned in about seven seconds on a CPU, while a full fine‑tuned model could take tens of minutes on a modern GPU.
What about actual personalized generation, not just predicting the preferred option in a fixed questionnaire? Here FaST shines again. The researchers evaluated personalized generations with a battery of metrics derived from LLM judges: a 0–5 personalization score (higher is better) and winrates in head‑to‑head comparisons against baselines. Across both DnD and ELIP, FaST variants—particularly those using Online‑DPO or RFT for fine‑tuning—delivered stronger personalization scores and higher winrates than a wide range of baselines, including traditional reward models and even oracle baselines that knew the user profile in depth. The results were not a hollow victory of “better scores”; in several cases FaST approached or matched the oracle upper bounds, despite having access only to a few dozen preference annotations.
There’s a tale in the numbers about robustness too. The FaST approach maintained performance when the data footprint shrank to a handful of items. In other words, you don’t need vast swaths of annotated data to get meaningful personalization. That matters for real‑world deployments where users may not want to or be able to provide continuous feedback. The implication is clear: you can ship lighter, privacy‑savvy personalization that still respects individual flair and style.
A practical, human‑centred path to fairer AI personalization
Beyond the engineering magic, the paper makes a pointed ethical argument. Personalization in AI carries serious risks: how we present information, how we steer opinions, and how we risk echo chambers or manipulation. FaST’s feature‑based design is presented not just as a technical workaround, but as a path toward transparency and accountability. By exposing the feature dimensions that steer the assistant’s behavior, users can see which levers are being pulled and decide whether to adjust or opt out of certain features. The authors argue that this kind of openness is a contrast to black‑box reward models that can obscure influence. The ethical tone is not naive—FaST acknowledges risks but invites governance by design: better control, better user agency, and a built‑in pathway toward inclusive personalization that can accommodate under‑represented user profiles.
In practice, that last point matters. Personalization, if done responsibly, can help marginalized users see AI that respects their needs and constraints rather than chasing majority preferences. The paper provides evidence that, when tailored through high‑level features, the method can improve alignment with diverse user bases and thereby contribute to more equitable interactions with AI systems.
Limitations, tradeoffs, and future horizons
No study is a perfect crystal ball, and the FaST work is no exception. The authors openly discuss several limitations. A key one is the dependence on the base LLM’s ability to generate a sufficiently diverse set of candidate responses for sampling. If the model’s distribution is narrow, the system may struggle to discover outputs with higher FaRM rewards. They note this isn’t unique to FaST; it’s a general constraint of sampling‑based fine‑tuning methods. They propose future work that could broaden the sampling space, such as prompting the policy to generate responses jointly as a list to ensure diversity.
Another limitation is the evaluation framework. The heavy reliance on LLM judges—state‑of‑the‑art language models performing human‑like scoring—introduces variability and potential biases. The authors mitigate this by using multiple metrics: pointwise personalization scores, pairwise win rates, and Elo rankings derived from many head‑to‑head comparisons. Still, human studies would strengthen the bridge from automated judgments to real‑world user satisfaction.
The datasets themselves are carefully designed but still synthetic in the sense of being generated with language models. They provide crucial, controlled environments for studying PPALLI, but real‑world deployment would bring a broader spectrum of contexts and more nuanced preferences. The authors acknowledge that their ELIP profiles capture three pragmatic dimensions (expertise, informativeness, and style) and that real users may express richer preference structures. The pathway forward could involve broader user studies and more diverse domains beyond fantasy and explain‑like‑me scenarios.
Finally, the paper touches on the computational and operational dimension of per‑user personalization. Training one model per user can be manageable for a handful of users or on edge devices, but scaling to millions of users would require careful infrastructure design, privacy safeguards, and possibly hybrid strategies that balance on‑device learning with server‑side orchestration. The authors’ emphasis on parameter efficiency and on‑device potential is a thoughtful nod toward scalable, privacy‑aware personalization—yet it also signals the need for follow‑ups that explore deployment in more demanding, real‑world ecosystems.
What this means for the future of personalized AI
The FaST approach exemplifies a broader shift in AI toward human‑centered customization that respects privacy while delivering meaningful, interpretable personalization. A few takeaways feel especially resonant in today’s AI conversations:
• Data‑efficient personalization is not a contradiction to quality. The combination of data‑driven feature discovery and a compact, interpretable reward model lets systems align with individual tastes without drowning in data requirements.
• If personalization is to be trusted, it must be transparent. FaST’s feature‑driven design isn’t just an engineering trick; it’s a design philosophy that invites users to understand which aspects of tone or approach are guiding the assistant’s behavior. In a world where an AI can simulate human conversation, having a map of the levers human readers can study is invaluable for accountability.
• Personalization can be inclusive. The authors’ emphasis on under‑represented user profiles isn’t a theoretical footnote. It’s a practical claim that tailored assistants can help users whose preferences diverge from mainstream patterns, potentially reducing the digital divide in how people experience AI companions.
• Privacy‑by‑design can coexist with usefulness. By enabling per‑user models that can, in principle, live on the user’s device, FaST points toward a future where personalization is not a cloud‑centric commodity but a personal tool that respects the boundaries people set around their data.
On the science side, FaST is an invitation to rethink how we think about alignment. Instead of relying on ever‑larger reward models trained on vast streams of feedback, FaST shows that a well‑engineered, feature‑driven approach can achieve robust personalization with modest data and modest compute. It’s not a repudiation of big models or long surveys; it’s a reminder that the most human touches—tone, humor, clarity, empathy—often live in compact, interpretable dimensions that we can measure and tune.
Where NAVER Labs Europe stands in this story
The study originates from NAVER Labs Europe, with Thibaut Thonet as the corresponding author and a team including Germán Kruszewski, Jos Rozen, Pierre Erbacher, and Marc Dymetman. The project sits at the intersection of practical AI engineering and thoughtful human‑centered design. It isn’t just about making AI better at mimicking human preferences; it’s about giving people more reliable, understandable ways to guide how AI behaves in everyday conversations. By releasing the DnD and ELIP datasets, the authors also invite the research community to probe, challenge, and extend these ideas in new domains—an open invitation to iterate toward ever more capable yet trustworthy personalized AI.
NAVER Labs Europe’s contribution is timely because it addresses a core bottleneck in industry: personalization that scales without compromising privacy or reliability. The FaST framework offers a blueprint for building personal assistants that feel less like generic help desks and more like tuned, considerate partners who understand when to be concise, when to joke, when to escalate a technical explanation, and how to adapt to a user’s unique blend of traits.
In the end, the work asks a subtle but powerful question: what if the key to truly personal AI isn’t a flood of data or a grand, opaque reward signal, but a tiny, well‑designed probe into what matters most to each person? FaST gives a careful, ambitious answer: yes, it’s possible to listen to a user’s preferences with respect, clarity, and humility—and to turn that listening into AI that feels more aligned, more trustworthy, and more human.
Closing thought
As the field of AI continues to grow toward ever larger language models and more aggressive training regimes, FaST reminds us of something essential: personalization is not merely a feature; it’s a relationship. The paper’s data‑efficient, interpretable approach offers a pathway to building AI that respects your privacy, speaks in your voice, and helps you accomplish your goals without pretending to be someone you are not. It’s the kind of progress that makes you a little more willing to invite a machine into your daily life—provided the machine is honest about which knobs it’s turning and why. If this is the trail we’re on, the next steps will be less about squeezing more performance out of a black box and more about crafting AI that understands people as individuals, not as data points on a chart—an aspiration that feels both technically exciting and morally humane.
For those who want to know where this work came from and who steered the ship, NAVER Labs Europe’s Thobet Thonet and his colleagues have laid a clear map. The dialogue between high‑level features, compact reward learning, and careful evaluation in DnD and ELIP marks a promising direction for the design of personal AI assistants—one where you, the user, remain at the center of a conversation that respects your boundaries, celebrates your quirks, and grows with you over time.
