Memes aren’t just images with funny captions. They’re cultural bones, remixable templates that carry ideas, attitudes, and jokes across millions of feeds in a heartbeat. In a new study from Vrije Universiteit Amsterdam, researchers push past the knee-jerk pixel-level comparisons that often miss the point of memes. They propose a way to cluster memes not just by how they look, but by the underlying templates and the multiple dimensions that make a meme feel related or unrelated. The result is a method that respects the messy, multimodal reality of memes—where text, visuals, people, and social context all matter—and still finds coherent groupings that humans recognize in everyday online life. The work is led by Tygo Bloem and Filip Ilievski, and it’s rooted in a simple, ambitious idea: you don’t need a giant database of templates to understand memes; you can discover the templates from the memes themselves, then match everything else to those templates using a kaleidoscope of similarity signals.
What makes this approach feel particularly timely is its insistence on modularity. Rather than collapsing meme similarity into a single number, the authors separate four dimensions—form, visual content, text, and identity—and pair each with both global and local features. Think of it as analyzing a symphony not just by its melody, but by timbre, rhythm, the crowd in the background, and the celebrity or character repeatedly appearing in the notes. By combining these dimensions, the method can handle remixing, substitutions, and partial similarities that typically confound standard clustering. And because it learns from the memes themselves rather than relying exclusively on a predefined knowledge base, it can adapt as memes evolve in real time. Across a data set drawn from Know Your Meme (KYM) and Reddit, the study shows that this template-guided, multi-dimensional clustering yields more coherent, human-aligned groups than traditional, bottom-up clustering. It’s a small advance in software, but it feels like a cultural upgrade in how we quantify a phenomenon that’s reshaping online discourse.
A Modular Multi-Dimensional View of Memes
Traditional meme analysis has tended to lean on two poles: global features that summarize an image, and local features that note distinctive bits like logos or faces. The problem, as the paper argues, is that memes inhabit several planes at once. A ‘Stonks’ meme, for example, might share a template image and font (form), display similar stock-market imagery (visual content), echo a catchphrase (text), or reappear with the same fictional character (identity). If you only chase one of these dimensions, you’ll miss most of the intricate relationships that make memes feel like connected jokes rather than random jumbles. The authors formalize this intuition by dividing similarity into four categories: form, visual content, text, and identity. Each category is then measured with a mix of global and local features. The result is a family of adjacency matrices, one per feature type, that can be combined in plenty of ways to reveal how memes relate along different roads to similarity.
Form signals capture the surface-level look of memes—the layout, color palette, typography, and the way text sits on an image. The researchers combine perceptual hashing (PHASH), color histograms, and SURF-based local descriptors to capture those low-level cues. Visual content digs deeper, aiming to detect recognizable objects, scenes, and expressions inside memes with a transformer-based embedding (ViT). Meanwhile, text brings in the words that often define a meme’s identity, using OCR to extract the overlay and a BERT-based embedding to measure how catchphrases or templated phrases travel. Finally, identity looks for the same people or characters appearing across memes, a signal that’s become a backbone for many meme families in recent years.
This division is more than a neat taxonomy. It’s a practical design choice that lets the system study how a meme can stay the same in form while changing its content, or keep a person’s face while swapping the joke around them. The combination of global features (broad image understanding) and local features (specific, repeatable details) lets the method respect both the big-picture semantical shifts and the tiny, often crucial variations that make a meme recognizable across replicas and remixes.
From Templates to Dynamic Meme Retrieval
The heart of the approach rests on a two-step dance. First, the method identifies a set of core meme templates by clustering a subset of memes that are highly similar across multiple dimensions. Those templates act as anchor points—a compact, robust representation of what a meme family is “about.” Then, the rest of the memes are matched to these templates, using a composite similarity score that sums the signals from each dimension. The beauty of this design is that you don’t need a giant, complete database of known templates; the system discovers its own templates from the data it sees.
Once templates are in place, the paper describes a method for incremental, template-driven clustering. Each meme is assigned to the template it most resembles, with an overall ranking that favors imagery with high initial coherence. In practice, this means you get tight, semantically meaningful clusters early on, and broader exploration later as more memes are added. The authors also show how this setup can power dynamic meme retrieval. They enrich each template with descriptors generated by a multimodal language model, index those descriptors using FAISS, and then perform template-aware searches that can pivot on dimension—identity to find all memes with the same character, or visual content to locate memes that share a scene, for example. The result is a practical prototype for a similarity-aware meme search engine that could be tuned to surface the most relevant memes in a given social context or for a given user query.
The case studies in the paper illustrate the practical payoff. A template built around the “WAT Grandma” meme shows how identity signals pull in related memes even when the surface visuals drift, while text can reveal connections that a blind visual search would miss. In another example, a template anchored to a controversial “Remove kebab” image demonstrates how different similarity dimensions converge to reveal related memes with similar taints or lexical cues, even when the surrounding visuals diverge. These examples underline a core point: in a world where memes mutate at the speed of scrolling, a multi-dimensional, template-led approach offers both resilience and clarity, enabling retrieval systems that respect the messiness of real online culture.
What The Results Reveal About Online Memes
The authors test their approach on a data set drawn from KYM’s meme corpus and Reddit’s r/memes, totaling roughly 20,000 memes. They evaluate two core claims: first, whether template-based clustering yields more consistent, coherent clusters than standard bottom-up clustering; second, how well human judgments align with the machine’s similarity dimensions. Across multiple experiments, the results lean strongly in favor of the template-based, multi-dimensional approach. When clustering 11,000 memes, the template-based method achieved notably higher consistency with known KYM templates than standard methods, even as the dataset grew. In one set of experiments, the best template-based configuration reached a consistency score around 0.87, compared with about 0.54 for the strongest standard approach. That gap isn’t just a number. It signals that template anchoring helps keep clusters semantically tight as noise and remixing accumulate in the wild.
The entropy analyses reinforce the same story from a purity perspective. Clusters formed with the template-based method tended to be purer, exhibiting far lower entropy than those from standard clustering. In other words, the memes grouped together by templates showed less cross-pollination across unrelated templates. The strongest gains appear when the authors combine all the feature dimensions; the four-dimension, template-based approach consistently yields the most coherent clusters, especially as more memes are included. An important nuance emerges: identity features drive early accuracy—memes featuring the same person or character often cluster tightly—whereas over time, relying on identity alone can misfire as new memes arise. Text signals, by contrast, are trickier; while they can anchor certain catchphrase-based templates, overlays and variations often dilute a pure textual signal. The take-home message is not that one signal rules them all, but that the right mix—especially one anchored to templates—delivers sturdier clustering, more in line with human intuition.
Beyond numbers, the study probes human alignment with the machine’s decisions. In human validation tasks, judges were asked to say whether a set of memes shared a relation and, if so, which dimension best explained the relation. The results showed that when judges recognized a relation, their choices largely matched the intended dimension for each feature set, with some caveats around text. The upshot is that the approach doesn’t just produce tidy clusters; it creates a taxonomy of relationships that humans can map onto, at least in a majority of cases. That alignment matters. It means the framework can serve as a useful bridge between computational analysis and social-scientific interpretation of memes, a space where nuance and context matter as much as similarity scores.
Toward Safer, Smarter Moderation and Cultural Insight
One of the most compelling implications of this work is practical: it offers a more nuanced lens for moderation and analytics. Memes are often used to spread toxic ideas or orchestrate campaigns, but their meaning hides inside multiple layers—the image, the text, the identity of the people involved, and the broader cultural references they evoke. By isolating and re-combining these layers, moderators and researchers can detect patterns that would be invisible to a single-metric method. The authors even suggest future work that would attach toxicity scores to templates themselves, enabling a smoother flow from template discovery to automated content assessment. It’s a promising path toward scalable, context-aware moderation that can adapt as meme culture evolves, rather than being locked to a fixed set of templates or a single similarity metric.
But the authors are careful about the darker side of this kind of technology. They acknowledge potential misuse—memes are a form of communication that can be weaponized just as easily as they can illuminate shared humor. They release the code publicly to encourage responsible use and emphasize that no background culture or intent is baked into the model itself. The paper also highlights the ethical complexities of data sources, annotation, and the uneven landscapes of meme communities. In that sense, the work isn’t merely a technical achievement; it’s a step toward a more reflective, interdisciplinary approach to studying digital culture—one that invites social scientists, linguists, designers, and platform moderators to collaborate with computer scientists rather than work in silos.
Looking ahead, the authors envision a future where the clustering framework feeds into broader tools: dynamic search engines for memes, evolution-tracking dashboards for meme genres, or even sentiment and stance analyses that map how humor travels through political or social movements. The practical blueprint already exists in the paper’s “Dynamic Meme Retrieval” section, where templates are enriched with descriptive metadata and indexed for fast, dimension-aware search. If you’ve ever wanted a way to trace how a meme’s joke morphs while still clinging to a core template, this is the kind of engineering that could make it possible—without flattening the rich, improvisational texture of online humor.
In the end, Bloem and Ilievski’s study is a reminder that memes are not static artifacts but living, evolving artifacts of culture. Their core idea—discover templates from data and then match everything else to those templates through a modular, multi-dimensional lens—offers a way to respect both structure and drift. It’s a tool for scholars who want to study how ideas spread, for moderators who want to keep platforms safe without crushing creativity, and for everyday readers who want to understand the memes that keep turning up in their feeds with surprising persistence. It’s not a revelation that memes are simple; it’s a recognition that the right tools can reveal the hidden grammar of a global language spoken in GIFs, captions, and the occasional punchline that lands just right.
As the study’s authors, based at Vrije Universiteit Amsterdam, remind us in a practical, almost democratic gesture, their work is public, modular, and expandable. It invites other researchers to plug in new feature extractors, new similarity dimensions, or new knowledge bases as the meme ecosystem continues to evolve. If memes are the climate of online culture, then this approach is a new set of weather instruments: it helps us map the temperature of a joke, the humidity of a reference, and the wind of a remix that carries ideas across digital continents. The next wave of meme research may well hinge on how readily we can combine templates with flexible similarity signals to keep pace with the speed and whimsy of internet culture, while still surfacing what matters—shared meanings, common frames, and the human impulse to laugh together, even as the memes change shape around us.
