Know the Layer Beneath

An abstraction is only as powerful as the person wielding it. And an AI coding agent is nothing but an abstraction layer.

I was listening to Boris Cherny from Anthropic talk about this and it really clicked for me. As AI takes over more of the actual coding, the engineers who stay sharp are the ones who understand what’s happening one level beneath. Not because they need to write it themselves, but because they know when something is off, and they know how to fix it.

Think about it. LLMs, agentic tools, Claude Code, Cursor, these are just the latest abstraction layer. Same story as frameworks, ORMs, cloud services. Every generation had the same choice: just use it, or actually understand it.

The engineers who knew React internals wrote better React. The ones who understood how databases execute queries wrote better queries. Same thing here. If you understand how LLMs reason, how context windows work, how agents plan and where they fail, you’ll build dramatically better software with them.

I push this a lot in my team. Don’t just prompt the agent and accept the output. Understand why it made that decision. Know its failure modes. Go read about how it actually works.

That’s the difference between someone who uses AI and someone who actually controls it.

But wait, is an LLM really an abstraction layer?

Most engineers don’t think about it this way. When you say “abstraction layer” people think of React, or an ORM, or a REST API. Something technical and structured. LLMs feel different, more like a tool you talk to than a layer you build on.

But that’s exactly what makes them easy to misuse.

An abstraction layer hides complexity beneath a simpler interface. You call query() instead of writing raw SQL. You use fetch() instead of managing TCP connections. The whole point is that you don’t have to think about what’s underneath. And that’s fine, until something breaks or until you hit the limits of what the abstraction was designed for.

LLMs are the same. You send a prompt, you get a response. Clean interface, powerful output. But underneath, there’s a model that was trained on patterns in text, that has no real understanding of your codebase, that doesn’t actually “think” the way you do, that works within a context window with hard limits, and that has a strong tendency to sound confident even when it’s wrong.

If you don’t know that, you’ll trust the output blindly. And that’s where things go wrong.

What knowing the layer beneath actually changes

Here are some concrete examples of how understanding LLMs makes you a better user of them.

Context windows and why your agent gets confused

Every LLM has a context window, a limit on how much text it can “see” at once. When you’re using a coding agent on a large codebase, it doesn’t see the whole thing. It sees a slice. If that slice doesn’t include the right files, the agent will make decisions based on incomplete information, and it won’t tell you that. It’ll just confidently produce something wrong.

If you know this, you start structuring your prompts differently. You point the agent to the right files explicitly. You break large tasks into smaller scoped ones. You stop asking it to “refactor the whole module” and instead guide it step by step.

Why LLMs hallucinate and what to do about it

LLMs generate the next most probable token based on patterns in their training data. They don’t look things up. They don’t verify. They predict. This means they can produce library methods that don’t exist, API signatures that are wrong, or explanations that sound right but are subtly off.

Engineers who don’t know this get burned by it constantly. Engineers who do know it treat LLM output like code from a junior developer: promising, but always reviewed. They verify the generated code actually works, they check the docs when the agent uses a specific API, and they don’t ship anything they don’t understand.

How agents plan and where they fail

Agentic tools like Claude Code or Cursor agents don’t just generate one response. They reason through a task in steps, sometimes using tools like file search or terminal commands along the way. But they can get stuck in loops, make wrong assumptions early that compound through the whole chain, or take a reasonable-looking path that leads somewhere broken.

Engineers who understand this structure their tasks to give the agent clear checkpoints. They don’t let it run for 20 steps before reviewing. They stay in the loop, steer when it goes off track, and know when to stop and restart with a cleaner prompt rather than watching it dig deeper into a bad direction.

The pattern is always the same

Every abstraction rewards the people who understand it. The engineers who knew what SQL was doing behind the ORM wrote faster queries. The ones who understood the browser’s rendering pipeline wrote smoother animations. The ones who knew how React’s reconciler worked wrote components that didn’t cause unnecessary re-renders.

The engineers who understand how LLMs actually work, what they’re good at, where they fail, and why, will get dramatically more out of them than everyone else.

Not because they write better prompts. Because they understand what they’re working with.

Go one level deeper. It’s always worth it.