Much-improved AI tooling is changing how we build software, and I want to take a peek into how the future of software engineering may unfold under its influence. What better place for that than with tech’s most “AI-pilled” teams: the AI labs themselves.
So, I visited the two leading AI labs to see how teams and engineers do things day to day. In this article and in an upcoming follow-up, I’ll share what I learned about how AI is reshaping software engineering principles many of us are accustomed to – and what’s stayed mostly the same despite the AI wave.
In a later article, we’ll compare findings from Anthropic and OpenAI to see what their ways of working might mean for the overall direction of software engineering.
Thanks to Anthropic for showing me inside their lab in San Francisco. I talked with four people:
, Head of Engineering for Claude Platform, whose organization owns the infrastructure that Claude runs onKatelyn Lesse, creator of Bun, now at Anthropic on Bun and Claude CodeJarred Sumner, who works across Claude Code engineering and educationThariq Shihipar, at Anthropic’s Applied AI organization in a role resembling a sales engineer, working with customers like Cursor, Cognition, and PerplexityDavid Hershey
Thanks to them, I got a sense of where things are headed at the leading AI lab – and possibly for the wider industry.
Before we continue, The Pragmatic Engineer will be on summer break for the next week and a half. This means no Thursday article this week, and no articles next week. I appreciate your understanding and support!
Back to today’s deepdive, we cover:
Complex & long: Claude Managed Agents. One of the most complicated projects took the Claude Platform team six months to ship, and created a new primitive to use at the agent infra level. Infra projects still need re-architecting mid-way through and take time to get right.**Twelve-month project done in 11 days: Bun rewrite to Rust.**Migrating a 500K+ line project to another language used to take a small team a year, making it impractical. With Fable and $165K of tokens, it recently took the creator of the project less than two weeks.Changing engineering practices. Inside the AI lab with more than 3,500 employees, prototyping is more fluid, verification is more time-consuming than implementation, code review and testing are increasingly done by AI.**Team-level changes.**Design is more ongoing and less upfront, teams work on more projects, a maximum of two engineers per project, and more.**Still the same:**two-pizza teams, planning is important, PRDs are relevant in complex projects, context switching is a challenge, the ratio of time spent on coding vs testing not changing that much.**Changing the “standout” software engineer archetype?**Deep understanding, including of a layer below what you work on, is valuable, along with the ability to coordinate work.**Will AI replace software engineering?**The more hands-on software engineers get with AI at the lab, the less they fear their jobs are going away.
The Claude Platform team’s most complex project in the past year was building Claude Managed Agents, a pre-built harness for production agents that runs in the cloud on infrastructure managed by Anthropic, or on your team’s own infrastructure, with any sandbox you choose. The project took around six months from idea until launch in April. Katelyn Lesse, head of engineering for Claude Platform, shared the story.
This team sits between the model/accelerator layer (Claude models operate on GPUs) and the product/application layer (with products like Claude Code and Claude Cowork):
Katelyn on what the Platform team does:
“We’re on the ‘token hot path.’ The prompt comes in, then we tokenize it. Then, things like safeguards and billing all happen within our layer.”
What the Claude team calls “Platform,” I think more of as “API.” Claude Platform operates the API, and owns responsibilities an API would have. Of course, the team does more than that, and Claude Managed Agents is one case we cover here.
**The platform layer is being migrated from Python to Rust. **Originally, this layer was written for Python for the “usual” reasons at AI companies: it’s a convenient language and AI researchers use Python already, which enables quick iteration. But Python is single-threaded, and at scale, when the API is under high load, it’s not as performant as Rust.
The project came together due to customers wanting their own “harness infrastructure”, says Katelyn:
“We started with a model where you get an API to define an agent, then you get an API to start a session with an agent. The reality of what the world wants and needs right now is people running their own infrastructure. So, we started to build a self-hosted sandbox.
But then, what we started to hear from lots of customers is that they’re trying to hack harnesses together, running their own “harness infrastructure,” and this gave us the idea for Claude Managed Agents.”
In this project, Katelyn said the single biggest matter was planning:
“There are products you can jump straight to prototyping, but then there are ones where you need to start by architecting it properly. For example, if we build a TypeScript CLI – which is pretty trivial for what needs to be built – we could go straight to prototyping. But with Claude Managed Agents, we needed to first figure out what we are doing.
Of course, we did some upfront prototyping for Managed Agents: hacking and spiking things. But prototyping itself was more about understanding the requirements.
Our planning process looked more like a typical pre-AI planning process.You know how every team has the project, where everyone comes up with some version of the same idea and people keep floating and circling it around until you finally do it? Managed Agents was this for our team. When we started the project, we had documents dating back up to two years about ideas and suggestions.
Post-planning, when the project officially kicked off, a PRD (product requirements document) was created:
“In the end, it was the Product Manager and the Tech Lead on our API Agents team who decided to pull the trigger and kick off this project. We’d get in a room, go through it, and get aligned. But it wasn’t just us: we’d have to align with teams around the business, other cloud providers, and other engineering teams. For example, we have a sandboxing team inside of the Platform org: and so this team was consulted on the design of Managed Agents, given this product would spawn a lot of sandboxes.
Just like before, we had a PRD, it was a Google Doc.We used a Google Doc because we needed to coordinate all interested people. This has not gone away.Similarly, my product counterpart and I run product reviews.”
Some processes from before AI, like the PRD, are still useful in complex projects today, for getting large groups of people on the same page.
With planning complete, the team decided to do a “spike” and stress-test the idea and architecture, by building the backend of Claude Code on the web. Remote execution of code with an agent harness was a similarly shaped problem to the one they wanted to solve for customers. The thinking was to start by solving it for the Claude Code team before tackling it in a more generic way for customers.
**Internal teams are more fluid than before AI. **Katelyn:
“Pre-AI, we might have hit the Claude Code team up with a bunch of big requirements documents, and they would have then hit us back with another set of documents. Now it was much easier: someone on our team built a few components, took it over to the Claude Code team, and they started to hack around it. We could figure out how this component plugs into this part of their product, and the other way around. It was just a faster and easier process, getting this first internal version of the product up and running.
Aligning with other teams on interfaces remains important, and it’s easier.
Back in the day, you’d have to come with a fully spec’d interface to use. Now, we could do it a lot more fluidly: we could stand up a stub service that shadowed traffic to start with, and iron out the interfaces with the Claude Code team as we went. They did some hacking on it and gave feedback, we made changes while building out the service under the interface, then went back to make it work.”
They launched a service for Claude Code’s mobile app to spin up a sandbox, boot up Claude Code and run it. The service went to production and the Claude Platform team took the learnings.
It’s likely a familiar scenario many engineers can relate to, that after planning a project and getting underway, you see that you’re going to need to change the architecture. It happened on this project, too.
**The platform team ended up re-architecting Managed Agents based on learnings from the Claude Code “spike.” **Re-architecting meant decoupling the “brain” of Claude and its harness from the “hands” (sandboxes & tools that perform actions) and the “session” (the log of events). Each became an interface that made few assumptions about each other.
The team also built an abstraction around vaults and credentials. Credentials can safely be stored inside a vault. All calls using credentials are made via a proxy which has a session token. It is the proxy that fetches the right credentials from the vault: the credentials are never seen by the agent, sandbox, or session. Credentials are only injected at the egress boundary when the service is invoked:
Internal “dogfooding” helped surface hard problems to solve. A few examples:
Reliability and scalability: these are really hard to do well for agents because if connection to the sandbox is lost, the whole agent dies and you lose state
Credentials and access control: also hard and problematic, especially when first building the service
The Managed Agents team shared more about this re-architecting project.
**The project took about six months, by no means a rapid process. **Katelyn emphasized that pre-AI, a project like this would have probably been in the realm of two years. Managed Agents is one of the biggest projects the Claude Platform team has built, and more complex than it looks: for example, adding support for running agents on AWS, GCP and Azure.
As covered before, Jarred Sumner is the creator of Bun, a popular JavaScript runtime with 22 million monthly downloads currently and Claude Code as a dependency.
Bun is written in Zig, a performant, productive language. However, it’s not memory safe and memory issues kept coming up. Jarred thought that rewriting the project to an also-performant, memory-safe language like Rust could be an option – except that rewrites like this turned out as follies in the past. Jarred (emphasis mine:)
“Historically, rewrites are a terrible idea. Excluding comments, Bun is 535,496 lines of Zig. A rewrite in another language would take a small team of engineers a full year. It would mean freezing bugfixes, security fixes or feature development for that time. The least risky approach to getting something shippable would be a mechanical port from Zig to Rust, with the minimal number of behavioral changes, using the exact same test suite we already use for testing Bun.
Fortunately, Bun’s own test suite is written in TypeScript which means it doesn’t depend on the runtime’s programming language.
A year of zero user-facing impact was not an option we could consider.So, enforcement through code style to fix stability issues was our best bet, and was our plan when we added Rust-inspired smart pointers to Bun’s codebase.But honestly, I didn’t want to do it. Homegrown smart pointers offer worse ergonomics than Rust, with none of the guarantees.”
But then, Jarred asked if AI could do the heavy lifting and wondered how much the migration could be sped up. **In the end, he completed the rewrite from start to merge in 11 days, using 64 parallel agents and $165,000 in tokens at API price. **Here’s Jarred on how his AI-heavy rewrite compared:
“By hand, I think this would’ve taken three engineers with full context on the codebase about a year, during which time we wouldn’t be able to improve Node.js compatibility, fix bugs, fix security issues or implement new features. We never would’ve done that. The realistic alternative was to do nothing and keep fixing the bugs at the top of this post forever.”
There was a lot more to the project than typing out the “...make zero mistakes” prompt:
Jarred made a detailed plan and style guide on how to migrate
He set up the project so agents would not use Git worktrees which he found slow, but worked on different files in the same codebase
He created an orchestration system where each AI agent came up with suggestions of what to change, but did not make a change to the file to avoid conflicts; an orchestrator AI agent created the commits
The most time and tokens went on fixing the compile bugs, tests, and verifying that things worked
Bun itself has a very robust test harness: when all tests pass, it’s a high-confidence signal that the rewrite works
Crucially, Jarred is
theultimate domain expert in Bun: he created the project and knows the codebase better than anyone
The rewrite has been shipped to production and powers Claude Code today.
We cover a lot more on this in What can we learn from Bun’s rapid Rust rewrite with AI?
So, what has changed in how teams build software at Anthropic, compared to the pre-AI days? That’s the question of this article, and it seems that many things are different. Let’s go through it:
Some things as normal as breathing at AI labs like Anthropic stand out as different with an outside perspective:
**Everyone runs multiple AI agents all the time.**Running 3-10 parallel agents is a given. Folks I talked with had their agents running in the background or cloud.**No token budget, usage not tracked.**One major difference between AI labs and everyone else is that therereallyis no token limit or token leaderboards that promote tokenmaxxing; people already use agents all the time.**Very high autonomy.**Work is becoming more structured inside AI labs, but there’s still massive autonomy compared to Big Tech and most startups. When everyone has unlimited tokens, it’s pretty easy to prototype any idea.
It was several times faster to prototype early approaches for Claude Managed Agents. Similarly, “spiking” the Claude Code mobile backend implementation was much faster than pre-AI, Katelyn told me.
Jarred made a point about the split between implementation and validation in his 11-day rewrite to Rust. Roughly, it was:
The “implementation” part of rewriting the code from Zig to Rust took about 15% of the time, while 85% went on fixing things up: getting it to compile, fixing tests, verifying that it worked.
Thariq:
“We see that few tokens are spent on actual implementation. Most are spent on discovery of unknowns, prototyping, mocking, and then in verification and testing.”
Jarred’s Bun rewrite echoes this: he spent more tokens on fixing up the implementation and verifying that it worked than on the implementation itself!
Jarred:
“Critiquing the code and testing it with agents is a new approach we do a lot more of
.I think a lot about trust when you merge a lot of code. How do you merge 100+ PRs a day, and make sure the code works? At this pace, you need to trust the code without the ability to read it all yourself. And I think it’s a few things:
Code review: it needs to be really good and automated. I’m clearly tooting our own horn here, but I find Claude’s code review to be really good. Claude’s code revie…
