What “Attention Is All You Need” Actually Changed

In 2017, a team of researchers at Google published a paper called “Attention Is All You Need.” The premise was deceptively simple. Instead of processing information sequentially, a model could look at everything at once and learn which parts mattered most. It could assign weight. It could prioritize. It could decide what deserved attention and what didn’t.

That single idea reshaped the trajectory of artificial intelligence. Models built on attention mechanisms could capture relationships across entire passages of text, not just adjacent words. They could hold context. They could understand that a word at the beginning of a sentence might matter more than the word right next to it. The result was a leap in capability so dramatic that nearly every modern AI system traces back to that paper.

But the deeper insight wasn’t just architectural. It was conceptual. Attention gave machines a way to separate signal from noise. To look at a universe of information and decide what mattered. That idea—that intelligence is partly about knowing what to ignore—turned out to be far bigger than machine learning.

Attention isn’t just how modern AI works. It’s increasingly how modern products work.

Attention becomes product design

Every product makes an implicit decision about what deserves attention. A dashboard shows every metric equally and overwhelms the user. A well-designed interface surfaces the one number that matters and lets the rest recede. The difference isn’t the data. It’s the editorial judgment about which part of the data deserves to be seen first.

Users don’t need more information. They need the right information. They need a system that has already done the work of deciding what matters in this moment, for this person, in this context. That’s what attention means in product design. Not showing everything. Showing the right thing.

Consider how many products fail at this. Search engines that return ten blue links when you needed one answer. Dashboards that display fifty metrics when one decision matters. News apps that present five hundred stories when you have three minutes. The problem isn’t a lack of data. It’s a lack of prioritization.

Users don’t need more information. They need the right information.

The products that feel intelligent—the ones that create a sense of relief rather than burden—are almost always products that have made a strong decision about what to attend to. They reduce noise. They increase signal. They treat the user’s attention as the scarce resource it actually is.

The pattern in my own work

I didn’t set out to build products around a theory of attention. But looking back at the last decade, the pattern is hard to miss. Each project I’ve built was, at its core, a system for deciding what deserves attention.

Project Sapiens helps people focus on the right AI concept instead of every concept. The problem isn’t a lack of AI tools or educational content. There is an overwhelming surplus of both. The problem is figuring out which concept matters for the person, at their level, right now. Attention means filtering the curriculum before it reaches the user.

Hotline Sports helps fans find the one game they can actually watch instead of showing every game. The sports broadcast landscape is a maze of subscriptions, blackouts, and regional restrictions. Showing a fan every game being played today is useless. Showing them the one game they can access, on the service they already pay for, is attention as product design.

Pronunciation Intelligence helps voice AI pay attention to the one detail people always notice—their name. A voice assistant can get weather, traffic, and calendar right. But if it mispronounces someone’s name, that’s the only thing they remember. Attention means knowing which details are small in the data but large in the experience.

AI Brand Visibilityhelps AI systems pay attention to the information that best represents an organization. When a model forms an understanding of a company, it draws on whatever signals are available. If the strongest signals are noise, the understanding is noise. Attention means making sure the right information is the most visible information.

Each product reduces noise. Each product increases attention. The domain changes. The principle doesn’t.

Reflection

I didn’t notice the pattern while it was forming. Each project felt like its own problem. Pronunciation was a speech problem. Hotline Sports was a sports media problem. Project Sapiens was an education problem. AI Brand Visibility was a discovery problem. Different domains, different users, different technologies.

But looking back, they all shared one structure. Each one took a landscape of overwhelming information and built a system to decide what mattered. Each one reduced noise. Each one increased attention. The products looked different from the outside. Inside, they were doing the same thing.

That’s what makes “attention” such a useful mental model for building products. It reframes the work. The question stops being “what can we show the user?” and becomes “what should we attend to?” The answer is almost never “everything.” It’s usually one thing, chosen well, presented clearly, with the rest available but not demanding.

Looking back, I don’t think I’ve spent the last decade building AI. I think I’ve spent the last decade building systems that help people—and increasingly AI—pay attention to the right things.

Different products. Same question. What deserves attention?

Tony Marks

Founder of Project Sapiens

Builder of AI systems for the Olympic Games, NBA, NFL, airlines, and enterprise platforms. Currently exploring attention as a product design principle, personalized AI interfaces, and human-centered AI.

Back to homepage