Editor’s note

We’ve spent time over the past few weeks talking about prompts, context, harnesses, and loops. But there’s another part of the AI engineering stack that doesn’t get nearly as much attention: the context your coding agents depend on.

This week, Maxim Salnikov explores why agent context deserves to be treated like any other software dependency, and how Agent Package Manager (APM) aims to solve the configuration drift that’s quietly emerging across AI development teams.

If you’d like to go deeper after reading, Maxim has also put together a free interactive guide on Agent Package Manager, which we’ve linked at the end of this article. I’ll leave the floor to him.

You know the feeling, because you’ve felt it before… in a different decade. A new engineer joins, and getting their environment right means a wiki page, a Slack thread, and copying files out of someone’s home directory. Two developers swear they have the same setup, but their tools behave differently, and nobody can say why.

Before npm, pip, and Cargo, that was simply how software felt. Dependencies lived wherever each developer happened to put them — a lib/ folder copied between machines, a README that said “install these seven things first.” It mostly worked, until it didn’t. Package managers ended it by turning scattered files into a declared dependency: one manifest that names what you need, one lockfile that pins exactly what you got, one command that restores it anywhere.

The files that configure your AI coding agents are in that same pre-package-manager state right now.

The context you never treated as code

Call it your agent context: the instructions and coding standards, the reusable prompts, the skills and agent personas, the plugins, the Model Context Protocol (MCP) server configurations. It shapes the code an agent writes, how it reviews a diff, and which external tools it can reach. That makes it a real project dependency — one of the highest-leverage ones you have, because it multiplies across every line the agent touches.

Yet most teams hand-maintain it as loose files scattered across per-tool directories and personal machines, with no declared source of truth. So it drifts. One tool’s instructions say to use the Money value object; another still mentions raw decimals. Three copies of the same review prompt live in three home directories, each checking a different threat model. Neither is wrong. Neither is portable.

A shared README doesn’t save you: documentation describes intent but never restores state, so the drift returns the moment someone edits a local file. This is “works on my machine,” aimed squarely at your agents — and it’s invisible, because nothing broke. The build is green. The agent just quietly got a little less consistent, or a little more dangerous, on one laptop.

Three files and one habit

The fix is not a new methodology. It’s the boring, proven shape that worked the first time. Agent Package Manager (APM), an open-source tool from Microsoft, borrows the manifest-plus-lockfile pattern and points it at agent context. There are three files and one habit:

• apm.yml — the manifest. The single human-authored source of truth, naming the primitives and MCP servers your project depends on, and which tools to target.

• apm.lock.yaml — the lockfile. Machine-generated, never hand-edited, pinning every dependency to an exact source ref and a content hash, so two developers install byte-identical context.

• apm-policy.yml — install-time governance, for when you’re ready for it.

• apm install — the habit. The one command that reads the manifest and materializes the declared context into each tool’s native location.

The move that makes it click is materialization. A manifest is a recipe, not a meal; apm install turns the declaration into the real files each tool reads natively — compiled into .github/ for Copilot, .claude/ for Claude Code, .cursor/ for Cursor — then stays out of the way at runtime. Your three-harness maintenance burden becomes one.

A 60-second quick start

Enough theory. Here is the entire loop against a real, public collection — web-ai-agent-skills, a maintained set of skills for the browser’s built-in AI APIs. Every command below is in the APM docs:

# 1. Install the CLI (macOS/Linux; use the PowerShell one-liner on Windows)

curl -sSL https://aka.ms/apm-unix | sh

# 2. In your project, once:

apm init

# 3. Add a skill as a pinned dependency:

apm install webmaxru/web-ai-agent-skills/skills/prompt-api

That last line is the whole pitch. It resolves the skill straight from its git source, pins it in apm.lock.yaml by commit and content hash, and compiles it into the location your agents already read. Commit the two files, and the next developer who runs apm install gets byte-identical context. The Prompt API skill is a declared dependency now, not a snippet someone pasted from a stale gist — and swapping prompt-api for any other entry works the same way.

What actually changes

Onboarding stops being a ritual and becomes git clone then apm install — the new hire’s agent is configured identically to yours before their first coffee. Drift stops being something you find during a security review: the lockfile makes it impossible, reproducing known-good context down to the bytes. None of this makes your agents smarter. That’s the point — it makes them consistent, the precondition for every other improvement, since you can’t tune a context you can’t reproduce.

The uncomfortable part is that we already learned this lesson once. Agent context deserves the same declaration and review discipline you already demand of package.json or requirements.txt. The tools exist. The only question is how long you keep paying the drift tax before you declare the dependency you already have.

And if authoring that first manifest feels like one more chore, there is a fitting last move: let an agent do it. The Agent Package Manager skill teaches your coding agent to run apm init, add and pin dependencies, validate the manifest, and manage the lockfile — installed, of course, exactly like any other skill:

apm install webmaxru/ai-native-dev/skills/agent-package-manager

Which is the whole idea coming full circle: the package manager for agent context, installed and operated by the agent itself.

That’s it for this one. We’ll pick up the conversation next week.

Until then, keep building.

Tanya D’cruz
Editor-in-Chief

But adding a graph does not remove the difficult questions. How does an agent know when it has gathered enough evidence? What happens when relationships change, memories become unreliable, or the graph itself sends the model down the wrong path? And when is a graph genuinely the right architecture rather than simply the more fashionable one?

For the first episode of the Agentic Engineering podcast, I spoke with David Knickerbocker, founder and intelligence architect at Verdant Intelligence and author of Network Science with Python. David has spent decades working across databases, cybersecurity, open-source intelligence, and network science. His answers return repeatedly to one idea: GraphRAG and agents are still software systems. If we want to trust them, we need to make their evidence visible, test their individual parts, and resist rebuilding systems that already work.

When do you genuinely need GraphRAG?

I don’t see graphs as an inherently more complicated alternative to RAG. Networks already exist in nature, organisations, and social systems; graphs simply represent how things connect. They can feel difficult because fewer teams have experience with network science, but the basic idea—one thing connected to another—is straightforward.

That does not mean every team should replace its RAG system. If you have invested in a mature similarity-based system and it solves the problem well, there is no reason to solve the same problem twice. Use GraphRAG when relationships, paths, and connected evidence are central to the problem, and your team is prepared to work with them. If the existing system works, keep it.

Does a natural-language GraphRAG endpoint remove complexity, or merely hide it?

A natural-language endpoint should hide implementation work without hiding the evidence. I design systems to return the context behind an answer: the source material, the relevant data, and the reasoning behind a predicted relationship. For example, when one of my tools converts text into a graph, it preserves both the original passage and the reason an edge was created.

There will always be some trust involved when an agent uses an external service, just as there is when we use a search engine or API. That trust should develop through repeated, dependable use—not blind faith. Developers may not need to understand the database schema or write Cypher, but they should still receive enough information to verify what the endpoint returned.

How does an investigative agent know when it has investigated enough?

We need to teach agents something my managers used to tell me: good enough is good enough. An agent should have a defined purpose, a limited mandate, and a clear point at which it reports back. An agent checking for concerts does not need to investigate indefinitely; it can check the relevant venues and artists, return the results, and stop.

An open-source intelligence agent may need to search more widely, compare conflicting sources, and detect manipulation, so its definition of “enough” will be different. The stopping condition has to match the task. The important thing is to decide what good enough means before the agent begins, rather than allowing it to keep reasoning and spending without a boundary.

How should GraphRAG represent information that changes over time?

I treat graphs as temporal. Information changes, and the completeness of our information changes with it. Rather than overwriting an old relationship, I would preserve what was claimed, where it came from, and when it was recorded. The history is part of the information.

In my systems, the highest-level object is a claim rather than a fact because the internet contains many competing claims and facts require judgement. A graph should preserve those differences instead of making every relationship look timeless and definitive. Timestamps, sources, and changing versions allow an agent to reason about what was believed at a particular point rather than treating the latest entry as the only truth.

How do we prevent an agent from poisoning its own memory?

Agent memory needs the same quality assurance we expect from any other piece of software. If a weak inference is stored, the system should retain its source and confidence rather than retrieving it later as an established fact. Independent agents can check one another, although using the same model everywhere may reproduce the same blind spots.

The more granular the memory, the more granular the testing needs to be. If you cannot inspect, validate, correct, or test an agent’s memory, you cannot determine whether it is reliable. Agentic work does not remove the need for software engineering; it makes careful testing at the component level even more important.

Can the knowledge graph, GraphRAG platform and agent really evolve separately?

They can evolve separately if their dependencies are explicit, versioned and tested. I give the major components of my systems their own semantic versions so each can be developed, released and, when necessary, rolled forward without rebuilding everything at once. Iteration creates infrastructure; big-bang development usually creates demonstrations.

A major graph-schema change may require the GraphRAG interface to change, just as a database change could break an application long before generative AI existed. But if the agent communicates through a stable natural-language contract, it may continue asking the same questions while the implementation underneath evolves. Separation does not mean independence; it means managing the connections deliberately.

Are we measuring GraphRAG correctly?

Counting relevant retrieved facts is not enough if the answer depends on a complete chain of evidence. A system may retrieve three of four necessary facts and still produce a confident but fundamentally incomplete conclusion. Teams need to test whether the system identified every question it needed to ask and recovered the evidence required to answer each one.

That also means testing the layers separately. Does the language model decompose the original request correctly? Does the GraphRAG endpoint return the right data for each subquestion? Does the agent assemble that evidence into a supported answer? When something is missing, we should identify the layer that lost it instead of labelling the entire system inaccurate.

Is the AI hallucinating, or is the graph misleading it?

I use a phrase in my training: similarity is not sameness. A similarity-based system may retrieve extra information that looks related but does not support the answer. If the model incorporates that noise as evidence, it can produce a hallucinated conclusion.

GraphRAG can fail differently. A query may miss a relationship, fail to traverse a node type, or return an event without its time or source. That is not necessarily the model inventing information; it may be a problem in the data, schema, query, or traversal. The only way to tell is to inspect the evidence path and determine which layer introduced the error.

David’s case for GraphRAG is not that a graph should replace every RAG pipeline. It is almost the opposite: keep the parts that already work, introduce graphs where relationships genuinely matter, and make every new layer earn its place.

That restraint matters as agents become more capable of searching, remembering, and acting without constant direction. More intelligence in the workflow also creates more places for evidence to become incomplete, outdated, or distorted. Reliability, therefore, depends less on whether a system carries the latest label and more on whether its builders can see what it did, test each part, and recognise when it has done enough.

The most useful place to begin may not be a large architectural migration. It may simply be a small experiment, built slowly enough that you can still understand what the agent sees and why it reaches the conclusions it does.

Enjoyed today’s podcast and want more engineering conversations? Pull up a chair.

Subscribe now

That’s it for this one. We’ll pick up the conversation next week.

Until then, keep building.

Tanya D’cruz
Editor-in-Chief

Editor’s note

Last week, Ken Huang broke down agentic engineering into three disciplines: harness, context, and loops. But I couldn’t leave that discussion just at that.

Because if you talk to builders, it’s clear that the work is still moving. A couple of years ago, it was all about prompting. Then we got better at tweaking context. But now, we’re starting to move beyond that. Toward loops.

So this week, Mona picks this apart from a practitioner’s lens. I’ll leave the floor to her.

In 2023, everyone wanted to be a prompt engineer. In 2026, the head of Claude Code at Anthropic says he doesn’t prompt Claude anymore. His words: “My job is to write loops.”

That single shift from prompting an agent to designing the loop that prompts it is the biggest change in how professionals work with AI since ChatGPT launched. And most people haven’t caught up yet.

Let me walk you up the ladder, one level at a time.

Level 1

Prompt engineering, the words you send

Prompt engineering is where all of us started. You learn that “write me a blog post” gets mediocre output, but “write a 900-word blog post for enterprise cloud architects, in a direct second-person voice, with a contrarian hook” gets something usable.

Prompt engineering is about how you express the task. Be specific. Give examples. Assign a role. Ask for a format. But it operates on one thing: the message you type into the box.

The limitation is obvious. A prompt is a single instruction to a model that has no idea who you are, what your codebase looks like, or what happened five minutes ago. You can only cram so much into words.

Level 2

Context engineering, everything the model sees

The next realization: the model’s output quality depends less on your clever phrasing and more on what information it has access to when it responds.

Context engineering is about curating everything in the model’s window — the instructions, the documents, the data, the tool definitions, the conversation history, the intermediate results. It asks a different question from prompt engineering:

• Prompt engineering asks: how do I say it?

• Context engineering asks: what should the model know?

This is why retrieval (RAG), memory systems, and structured tool outputs became the center of gravity for enterprise AI teams in 2024–2025. Teams needed to feed the model the right context at the right time, and what was just as important was keeping the wrong context out.

But context engineering has ceilings of its own:

• The window is finite, and attention degrades before it fills.

  Even massive context windows don’t mean equally useful attention across every token. Overload the window, and the model starts missing details buried in the middle. More context is not better context.

• It’s a snapshot, not a conversation with reality.

  Context is assembled before the model responds. Real tasks surprise you mid-execution — a dependency is missing, an API returns something malformed, a test fails. No amount of upfront curation can anticipate what only shows up when you run the thing.

• The model can know everything and still do nothing.

  Context engineering makes the model informed. It doesn’t make it capable. It can’t execute your code, check the result, or fix what broke. Knowledge without the ability to act is a very well-read consultant with no hands.

• Curation doesn’t scale by hand.

  Someone has to decide what goes into the window for every task. Past a certain complexity, assembling the context itself needs to be automated, which is the door the next two levels walk through.

Level 3

Harness engineering, the environment the agent runs in

Then agents arrived, and a prompt plus context stopped being enough.

An agent doesn’t just answer — it acts. It reads files, runs code, calls APIs, and checks its own work. For that, it needs an environment: tools it’s allowed to use, a filesystem it can touch, permissions that constrain it, feedback mechanisms that tell it whether the tests passed.

That environment is the harness. Harness engineering is designing the executable world around a single agent run — the system prompt, the tool set, the memory, the guardrails, the verification steps. If context engineering decides what the model knows, harness engineering decides what the model can do and how it finds out whether it did it well.

If you haven’t subscribed yet, pull up a chair.

Subscribe now

If you’ve used Claude Code, Cursor, or any serious coding agent, you’ve benefited from harness engineering, whether you knew the term or not. The difference between an agent that flails and an agent that ships is usually not the model. It’s the harness.

But even a great harness has a human bottleneck: you. You’re still the one deciding what the agent works on next, reading the output, and firing the next run. One turn after another. The agent is a power tool, and you’re holding it the whole time.

Level 4

Loop engineering, the cycle that drives it all

Loop engineering removes you from the turn-by-turn seat.

Instead of prompting the agent, you build a small system (sometimes a shell script, sometimes a scheduled job, sometimes a few hundred lines of orchestration code) that runs the agent in a repeating cycle:

You define the goal and the stopping condition. The loop does the iterating. It runs on a schedule (including while you sleep) or until the goal is met.

The idea crystallized fast in mid-2026. Peter Steinberger argued that the real skill had moved from prompting agents to designing their loops; Addy Osmani gave the practice its name and structure in an essay the following day; and inside Anthropic, the

Claude Code team was already describing their daily work the same way. The framing that stuck: the harness equips a single agent run — the loop is what keeps running the harness, spawning helper agents, checking results, and feeding itself the next task.

Each level wraps the one below it. The loop runs the harness. The harness carries the context. The context frames the prompt. Nothing on the ladder becomes obsolete. Rather, the leverage keeps moving up.

How to schedule a loop

1. Open your project in Claude Code.

2. Use the /loop command followed by your interval and the prompt or skill you want to run.

The /loop command is the core, with three modes depending on what you provide:

• /loop 5m check the deploy: Runs your prompt on a fixed schedule (Claude converts the interval to a cron expression)

• /loop check the deploy (no interval): Claude picks the delay dynamically after each iteration, between one minute and one hour, waiting less when things are active and more when quiet

• Bare /loop: Runs a built-in maintenance prompt that continues unfinished work, tends the current PR (review comments, failed CI, merge conflicts), and does cleanup passes. You can replace this default with your own loop.md file at project level (.claude/loop.md) or user level (~/.claude/loop.md).

But here’s the part nobody puts in the headline

First: loops multiply whatever you give them. A well-designed loop multiplies a strong engineer. A badly designed one multiplies a bad decision just as fast — with less of you watching. Two people can build the identical loop: one uses it to move faster on work they deeply understand, the other uses it to avoid understanding the work at all. The loop can’t tell the difference. You can.

Second: the unglamorous work is making it stop. Ask practitioners about loops and the first war stories you’ll hear aren’t about architecture: they’re about cost. Autonomous loops that burned through hundreds of dollars overnight. Companies imposing hard monthly caps on agent spend after blowing annual AI budgets in a quarter. The production rule is simple: every loop ships with hard guards (budget limits, iteration caps, verification gates, audit logs) or it doesn’t ship.

Everything in this article assumes one foundational skill: knowing how to build an agent in the first place. The loop is only as good as the agents it orchestrates.

That’s what my latest book, AI Agents on AWS (co-authored with Bunny Kaushik, published by Packt), teaches. You’ll learn how to design agents that reason, use tools, and act autonomously; how to wire up the harness around them — memory, guardrails, verification; and how to take them from a notebook experiment to production on AWS.

Add it to your shelf

When an agent has to serve real users across real channels, with real credentials, tools, permissions, memory, failures, latency, and cost constraints, the problem changes. It stops being only a prompting problem. It becomes an engineering problem.

This is where agentic engineering begins.

Agentic engineering recognizes that an AI agent is not just a model wrapped in an interface. It is a runtime system. It needs boundaries, identity, observability, recovery paths, and a way to explain what happened when something goes wrong. It must be designed like infrastructure, not like a clever script.

Three practices define this shift: harness engineering, context engineering, and loop engineering.

Harness engineering is the discipline of building the runtime around the agent. It includes gateways, tools, permissions, hooks, retries, telemetry, memory, sandboxes, and recovery paths. The harness is what lets an agent act safely in a real system. Without it, autonomy becomes fragility.

Context engineering is the discipline of selecting, shaping, compressing, refreshing, and protecting the information given to the model. Context is no longer just “stuff in the prompt.” It is an operational resource with a budget, lifecycle, security posture, and failure mode.

Loop engineering is the discipline of designing the cycles through which agentic systems act, learn, improve, and produce software. At the runtime level, this means structuring the agent’s observe-decide-act-evaluate cycle with termination conditions, policy gates, retry limits, rollback paths, and human escalation points.

But loop engineering goes further.

One loop is the recursive self-improvement loop. A production agent system should be able to detect weakness, diagnose failure, propose improvement, test that improvement, validate it against policy, and only then adopt it. This does not mean giving agents uncontrolled permission to rewrite themselves. It means creating bounded, observable, auditable improvement cycles where agents can help harden the system without bypassing engineering discipline.

Another loop is the continuous software factory. In agentic engineering, software development itself becomes a loop: requirements become designs, designs become code, code becomes tests, tests become deployment signals, deployment produces telemetry, telemetry informs the next requirement, and the cycle continues. Agents can participate at every stage, but the factory still needs gates, evidence, review, and rollback.

This is one of the central themes of my upcoming book, OpenClaw AI in Production. OpenClaw treats agent behavior as something that must be routed, bounded, observed, and corrected through architecture. The Gateway becomes the control plane. The request pipeline separates context assembly, authorization, directive handling, tool execution, and model invocation. Hooks allow cross-cutting concerns such as telemetry, policy, billing, and remediation to operate outside the core agent logic.

That architecture matters because agent failures rarely appear as simple crashes. A stale memory, a slow tool, a partial outage, a malformed directive, or an over-permissive policy can quietly distort the loop. The agent may keep acting, but each step moves it further from safe and useful behavior.

Production systems need to detect that drift.

This is why observability for agents has to go beyond CPU, latency, and HTTP response codes. A 200 OK from a model provider does not tell us whether the agent used the right tool, respected the right policy, retrieved the right memory, or stopped at the right time. Agentic systems need semantic observability: traces that show the agent’s observable decisions without leaking sensitive data or private reasoning.

The deeper opportunity is self-correction. If an agent platform can detect degraded state, isolate the failing component, reduce privileges, pause risky workflows, retry safely, or route through a fallback path, then we move from passive monitoring to active resilience. If the software factory can learn from its own incidents, tests, deployments, and user feedback, then engineering itself becomes more adaptive.

The future will not be won by prompts alone. It will be built by engineers who understand harnesses, contexts, and loops - and who can turn model intelligence into systems that are secure, observable, resilient, and continuously improving.

OpenClaw AI in Production is my attempt to map that territory in detail: from Gateway-centered architecture and policy enforcement to distributed state, semantic observability, self-correcting stacks, fault injection, high-throughput design, and decentralized deployments.

Agentic engineering as a discipline is still young. But one thing is already clear: building agents that work in demos is very different from building agents that survive, learn, and improve in production.

This article is a glimpse into the ideas behind Ken’s OpenClaw AI in Production. If you’re interested in building AI systems that are resilient by design rather than optimistic by default, the book expands on the engineering principles that separate production platforms from impressive prototypes.

Add it to your shelf

Context complexity exists across five distinct levels, evolving from zero-context basic prompts to highly advanced semantic blueprints. Prompt engineering resides at the shallowest level, treating context as a mere text prefix. Context engineering, conversely, treats context as a structural, programmatic architecture.

To build a robust multi-agent system (MAS), we must transition from linear text parsing to multidimensional semantic structures. We can achieve this by implementing Semantic Role Labeling (SRL) to map complex data relationships natively. By utilizing the Model Context Protocol (MCP), we can define rigorous protocol message formats that specialist agents such as a Researcher, Writer, and Orchestrator use to communicate without ambiguity.

Instead of hoping a black-box model infers the correct workflow, we architect a semantic blueprint that explicitly guides the system’s reasoning process, completely decoupling the immutable enterprise data layer from the probabilistic reasoning layer.

Architecting the glass-box context engine

The antidote to black-box unpredictability is the context engine: a transparent, glass-box architecture of context and reasoning. In this framework, the engine’s core intelligence is compartmentalized into distinct, observable modules: the Planner, the Executor, and the Tracer.

• The Planner: Acts as a meta-controller. It utilizes a discoverable Agent Registry to dynamically route tasks, intelligently switching between semantic search and strict data filtering based on the precise context of the request.

• The Executor: Relies on a Dual RAG architecture, simultaneously processing factual data via a strict Knowledge Base and procedural instructions via a Context Library.

• The Tracer: Allows human operators to parse the ExecutionTrace object to render token metrics, dependency resolutions, and view the internal “thinking” steps of the agent.

Within a finite reaction field, agent decisions must be bounded by plausibility to avoid cognitive dissonance. We enforce these boundaries using operators like the Semantic Switch ($\sigma$) and the Triton Planner Kernel. These tools apply explicit, rule-bound kinematics over the continuous probabilities of the neural field, acting as the constraint satisfaction engines that pure LLMs lack.

Enterprise guardrails: Scaling with Dual RAG

Bringing these systems into production requires moving AI directly to the data. By leveraging architectures of modern enterprise databases with vector and relational support, we can build Sovereign AI systems that are dock-agnostic, portable multi-agent systems directly to immutable enterprise databases.

This paradigm shift enables hyper-contextual capabilities:

• Hybrid querying: Combining strict SQL scalar filters (such as experience levels or salary caps) with semantic vector searches to generate grounded, policy-compliant recommendations.

• Converged Spatial-RAG and GraphRAG: Integrating physical geolocation (Oracle Spatial) and social dimensions (SQL Property Graphs) directly into the vector search pipeline to evaluate meaning, physical proximity, and relationship mappings simultaneously.

• Micro-context engineering: Deploying specific Summarizer agents to proactively manage API costs, reduce context overhead, and actively manage token limits.

• Policy-driven moderation: Implementing a two-stage moderation gatekeeper that flags anomalies, prevents data poisoning, and adapts to real-world legal compliance limits.

Conclusion: From cost center to value multiplier

The debate over how to achieve true agentic reasoning will not be settled by building ever-larger black boxes or writing increasingly convoluted text prompts. The future belongs to those who recognize and engineer around the structural boundaries of artificial intelligence.

By embracing context engineering, we shift AI from an unpredictable stochastic experiment into a verifiable, transparent asset. Through glass-box architectures, Dual RAG pipelines, and the rigorous application of spatial and semantic boundaries, we can finally build human-centered AI systems that are genuinely business-ready. It is time to stop prompting the machine and start engineering the context.

Less hype, more engineering. Pull up a chair.

Subscribe now

AI-assisted development is becoming the new standard. Learn how to use OpenAI Codex effectively in real engineering projects through a live, instructor-led bootcamp.

Date: Saturday, July 25, 2026
Time: 9:00 AM – 3:00 PM EST
Format: Live Virtual Bootcamp
Hosted by: Packt Publishing

What you’ll learn:

• How to use OpenAI Codex as a practical coding partner

• How to plan, build, debug, test, and review code with AI assistance

• How to apply Codex across real-world developer workflows

• How to complete hands-on projects and coding exercises

• How to improve code quality, productivity, and delivery speed using AI-assisted development

How this helps your career:
AI coding tools are becoming part of everyday software development. Learning how to work effectively with Codex can help developers become faster, more confident, and more valuable in AI-enabled engineering teams. This bootcamp helps participants strengthen their practical development workflow, improve productivity, and build skills that are increasingly relevant across modern software roles.

Certificate included:
Participants will receive a Packt certificate, which can be added to their LinkedIn profile, resume, or professional portfolio to showcase their learning and commitment to practical AI-assisted development.

Best suited for:
Technically professionals, developers, software engineers, technical leads, and AI builders who want to use OpenAI Codex to simplify coding workflows and improve engineering productivity.

This bootcamp helps you stay ahead so you can ship more, grow faster, and take on higher-impact work.

Register Here

Rick Spencer, General Manager for Technology and Product at SUSE, has looked hard at how the industry is measuring AI’s effect on engineering. “I consider that garbage vanity metrics,” he says, calling them unhelpful.

His argument for what to track instead is one of the more clarifying things an engineering leader can hear right now, because it separates the numbers that look like progress from the numbers that actually represent it.

Output is cheap; impact is what counts

The core of Spencer’s position is a distinction between output and impact, and it matters because the two come apart precisely when AI enters the picture. AI makes output cheap as the lines of code, pull requests, and token counts all mount up when agents are doing the writing, which makes them exactly the wrong thing to measure if what you care about is value delivered. “We’re really tending away from measurements that measure output and utilization, and we’re trying to focus on impact,” he says. A leaderboard that ranks engineers by how much their agents produced does not tell you who is solving the hardest problems or keeping customers safe. It tells you who is generating the most volume, and in an AI-assisted world, that number is close to meaningless.

There is also a structural reason the standard tooling does not fit SUSE, and it applies to more organizations than it might first appear. Much of the available measurement tooling assumes a particular shape of company. “They really assume you’re a proprietary software company where everyone’s working on a single code base,” Spencer explains, “which is just not how an open-source enterprise works.” His engineers work across hundreds, sometimes thousands, of repositories, where the maintenance work on each one differs enormously. A per-developer comparison across that landscape measures the shape of the work far more than it measures the contribution of the engineer, which is why he treats developer-to-developer comparison as fundamentally low value rather than merely imperfect.

The reporting burden itself is part of his objection, and it is a point leaders setting up AI dashboards should sit with carefully. A measurement regime that requires engineers to generate weekly utilization reports spends the very time it claims to be optimizing. “I’d rather have them working than reporting,” Spencer says. The instrument meant to measure productivity eventually becomes a tax on it.

What SUSE tracks instead

Rejecting vanity metrics only helps if there is something better to put in their place. And Spencer shares how SUSE measures business impact in terms that connect directly to what customers actually receive. “How fast are CVEs being addressed, how fast are patches being backported, how fast are our L3 responses getting closed while maintaining the same NPS score,” he underscores, listing what his teams track. The common thread is that each one is an outcome the customer feels, not an activity the engineer performs. AI has been applied to exactly these areas, so measuring the speed and quality of those outcomes tells you whether the AI is doing anything worth its cost, which is the actual question worth asking.

This shift from output to outcome reframes what a metric is for in the first place. A CVE response time captures whether the organization is keeping customers safe faster than it used to. A backport speed captures whether stable releases are getting their fixes without the manual grind that used to gate them. These numbers move because the underlying work got genuinely better, not because more text was generated, and that is the property that makes them trustworthy. They are also far harder to game, because the only way to improve them is to actually improve the thing the customer depends on.

Give managers visibility, not a leaderboard

None of this means SUSE ignores cost or utilization entirely, and the distinction Spencer draws here is the one that keeps the approach from collapsing into either negligence or surveillance. The company is building dashboards that give engineering managers visibility into their team’s cost and utilization, but the purpose is coaching rather than ranking. The unit of analysis is the team, and the question it answers is diagnostic. Spencer gives the example of a manager with an eight-person team noticing the numbers and asking the right kind of question. “We’re burning a lot of tokens. What are we actually doing that’s burning that many tokens? I’m not sure we’re getting value out of that.” The inverse matters just as much, where purchased seats for a code assistant sit unused, and the manager asks whether there are places the team should be drawing value that it is currently leaving on the table.

The governance side of that picture, including how SUSE keeps agents and their costs inside a boundary it can stand behind, is covered in a companion piece, How SUSE Runs AI Without Losing Control.

The difference between this and a leaderboard is not subtle, and it is the heart of the leadership lesson. A leaderboard exposes individuals and turns measurement into a game engineers play against each other, a game Spencer is explicit has nothing to do with customer value. Team-level cost visibility used for coaching does the opposite. It gives a manager the information to guide the team toward better use of the tools without making any individual engineer feel watched. “We’re really trying to decentralize and allow engineering managers to guide their teams on getting the most value out of the AI,” he says, “without it becoming like a leaderboard game where developers feel like they’re exposed.” The data exists to help the manager help the team, not to rank the team against itself.

The principle holding it together

What makes Spencer’s approach more than a list of preferred numbers is the principle holding it together, which is that measurement should serve the work rather than distort it. Every choice he describes follows from that one idea. Impact comes before output because output is the thing AI inflates. Team-level diagnostics come before individual leaderboards, because the goal is coaching rather than competition. Business outcomes come before activity counts, because outcomes are what customers actually receive. The decentralization to engineering managers reflects the same conviction that the people closest to the work are best placed to judge whether the AI is helping, given the right information and trusted to use it well.

The deeper point for any leader standing up AI measurement is that the easy numbers and the useful numbers are not the same, and AI has widened the gap between them. The figures that are simplest to collect, lines of code, tokens, and per-head utilization, are the ones AI has made least meaningful. The figures that matter, the speed and quality of the outcomes customers depend on, take more thought to define and more care to track. Spencer’s argument is that the effort is the job. “Let’s focus on the impact,” he says, “the business impact, not on the utilization.” For engineering leaders deciding what belongs on a dashboard as agents reshape their teams, that is the distinction worth getting right before the vanity metrics calcify into the way the organization sees itself.

This creates a pervasive dissonance: the expectation of industrial reliability built atop probabilistic generation.

The current noise suggests that if we simply scale the parameters, refine the system prompts, or throw more compute at the problem, true autonomous reasoning will spontaneously emerge.

To cut through this noise, we must confront an uncomfortable reality. Unconstrained probabilistic generation cannot serve as the kinematics for reliable robotic or enterprise execution. If we are to build truly agentic systems, we must move beyond the brittle, zero-context art of prompting and embrace the rigorous, transparent discipline of context engineering.

The limits of the translation lattice

At their core, LLMs do not guarantee structured reasoning across steps. They act as translation lattices operating on lossy text compressions, fundamentally disconnected from the non-verbal, experiential grounding that characterizes human thought. They are magnificent at semantic association but possess no innate understanding of physical or logical bounds. When an agent relies solely on an LLM’s internal weights to reason, it runs headfirst into the reality of the Probability Decay Theorem (PDT), a multiplicative reliability collapse effect. The theorem was derived through multiple real-world implementations by the author.

The PDT governs the multiplicative collapse of chained rules. In single-turn interactions, an LLM might succeed 95% of the time. But in agentic workflows requiring multi-step execution, the probability of success does not simply carry an additive exception load PDT₁. Instead, it collapses under multiplicative chained probability PDT₂. A chain of ten probabilistic bets, each with a 95% success rate, yields an overall reliability that is entirely unacceptable for real-world enterprise operations.

If an autonomous agent executes a sequence of tasks where each step is 95% reliable, the multiplicative collapse of the Probability Decay Theorem dictates that it takes just 14 consecutive steps 0.95¹⁴ ≈ 0.488 for the overall success rate to plummet below 50%, rendering the entire workflow less predictable than a blind coin toss. This shows that after enough chained steps, reliability rapidly degrades below usable thresholds, even when individual steps appear highly accurate.

LLMs operate on statistical associations in text without explicit grounding in physical or logical constraints. This means that mapping raw signals to lexical labels will always incur an irreducible loss due to the polysemy inherent in language. These models are structurally bounded by the imprecision of polysemy. Treating their hallucinations as mere software bugs to be patched by longer text prompts ignores the mathematical reality of the medium.

The embodied reality factor

To understand how to fix this, we must look at the lineage of modern embodied AI, tracing back to industrial automated systems. Physical robotics and real-world supply chains cannot operate on unconstrained probabilistic generation. Instead, algorithmic logic must be ruthlessly translated into the thermodynamic and spatial kinematics of the physical world. This absolute reliance on physical bounds forms the applied foundation for defining hard limitations such as spatial collisions, reaction times, and hardware constraints, which are not errors. They are the foundational constraint forces that dictate the accuracy of a signal. This principle inherently solves the simulation-to-reality (Sim2Real) transfer gap. If we want our digital agents to interact reliably with the real world, we must enforce digital representations of physical limits to ensure their reasoning remains strictly bounded by the reality factor of the environment.

If reliability cannot emerge from probabilistic generation alone, then it has to come from somewhere else. That somewhere else isn’t another prompt. It’s context, engineered as a first-class component of the system.

That’s where we’ll begin in Part 2. See you next Thursday!

His career has moved fluidly between academia and industry, from teaching advanced AI courses at the American University of Beirut to advising teams on turning machine learning ideas into systems that can be trusted.

He is also known for his candid take on the current AI landscape, especially the gap between meaningful engineering and what he often calls AI slop.

In this conversation, Ammar challenges one of the most common assumptions in agent development: that a correct answer is evidence of a successful agent. He explains why reliability lies in the path an agent takes, not just in the result it produces, and why evaluation must evolve from output scoring to a discipline that measures behaviour and trustworthiness in production.

Most teams think they’re evaluating agents, but they’re actually not. Where do you see the biggest illusion of evaluation today?

The biggest illusion is that teams think they are evaluating an agent when they are only evaluating the final answer.

That works well for a chatbot. But an agent is different. It plans, chooses tools, passes arguments, reads observations, retries, stops, and sometimes takes action. A final-answer score hides most of the actual failure surface.

An agent can produce a good-looking answer after calling the wrong tool, wasting ten steps, misreading a tool result, or ignoring a failed call. From the outside, the answer may look acceptable. From a reliability perspective, the run is not acceptable.

So the illusion is: “the answer looked right, therefore the agent worked.” However, what you need to know is whether the path was valid, efficient, grounded, and safe.

You break evaluation into component, trajectory, outcome, and adversarial layers. Where do most teams underinvest, and what failures does that lead to?

Most teams underinvest in trajectory evaluation and adversarial evaluation.

Outcome evaluation is the easiest layer to reach for because the final answer is visible. Component evaluation is also fairly intuitive once tools are involved: did it choose the right tool, did it pass valid arguments, did the plan make sense?

Trajectory evaluation is harder because you need structured traces and assertions over the run itself. But this is where many production failures live: loops, duplicate calls, silent retries, no recovery after a tool failure, unnecessary detours, high latency, high token cost. Two agents can produce the same answer, but one gets there in four clean steps, and the other gets there through an expensive, brittle path. Output scoring treats them as equal. Production does not.

Adversarial evaluation is also underbuilt. Teams may try a few prompt-injection examples manually, but they rarely turn those attacks into a versioned regression suite. That leads to a false sense of safety. A guardrail can look very strong against the examples it was designed for, while still being fragile against slightly different payloads.

Agent failures only show up after deployment. What’s the hardest failure mode to catch early, even with a good evaluation setup?

The hardest failures are the ones that look like successful runs.

A tool returns something plausible but incomplete. The agent takes a reasonable-looking path. The final answer is fluent. No exception is thrown. But the answer is weakly grounded, the evidence is stale, or the agent skipped a recovery step after a bad observation.

These failures are hard because they do not announce themselves as failures. They show up later as retries, edits after the answer, escalations, user abandonment, or quiet loss of trust.

The other hard category is drift. A hosted model changes, a tool schema changes, retrieval content shifts, or user traffic moves into a different distribution. Nothing “breaks” in the traditional software sense, but the agent becomes less reliable. This is why offline evals need to connect to production monitoring. A test suite is necessary, but it is not the whole system.

LLM-as-a-judge is becoming a default pattern. Where does it actually work well, and where does it quietly break?

LLM-as-a-judge works well when the task is bounded, the rubric is explicit, and the judge has the evidence they need. It is useful for rubric-based scoring, regression checks, pairwise comparisons, and multi-dimensional outcome evaluation, especially when you calibrate it against human labels.

The important part is that the judge itself has to be evaluated. I would not trust a judge just because it is an LLM. I would look at correlation with human labels, agreement rates, mean absolute error, and performance by rubric dimension.

Where it quietly breaks is when teams use it as an uncalibrated oracle. Judges often reward verbosity, prefer answers in a certain style, miss missing citations, or give a strong score to an answer that is polished but not grounded. Overall scores can also hide weak dimensions. For example, a judge may be decent on safety or format, but poor on groundedness, which is often the dimension that matters most for a research or retrieval-heavy agent.

So I see LLM judges as useful evaluators, not authorities. They need rubrics, evidence, calibration, and periodic human audit.

If you had to audit an agent system in production with very limited time, what signals or metrics would you look at first to decide if it’s reliable?

Aggregate success rate is often the last metric I look at. I would start with the traces behind the failures and the production signals that users generate when the agent is not working.

The first signals I would inspect are abandonment rate, retry rate, escalation rate, clarification rate, thumbs down, and edit-after-answer rate. Those are often more honest than a dashboard success metric.

Then I would look at trace-level reliability: number of tool calls, duplicate calls, loop-like behaviour, failed tool calls, recovery after failure, latency, and token cost. A reliable agent should not only get the answer right; it should get there through a path that is stable and explainable.

I would also check whether offline evals are tied to production: are failed production examples converted into regression tests? Are adversarial cases versioned? Are judge scores calibrated against human labels? Are there no-go gates for safety, groundedness, cost, latency, and step count?

With limited time, I am looking for one thing: whether the team has connected offline evaluation, online monitoring, and regression gates. If those are disconnected, reliability is usually more assumed than measured.

As organizations continue to explore what AI can and should do, voices like Ammar’s help bring the discussion back to the questions that matter: What problem are we really solving? Can the system be trusted? And are we building something meaningful, or simply adding more noise to an already crowded field?

If you’d like to continue exploring these ideas, Ammar will be speaking at Agent Evals Bootcamp on June 27th, where he will turn these ideas into a hands-on framework for evaluating agents across tool use, planning, trajectories, outcomes, regressions, and adversarial failure modes before deployment.

Of course, this won’t do for us. What’s the point of building custom AI pipelines if they sit locked in a dark room? To make that logic useful, we need to connect it to the real world. We build user authentication, set up secure access controls, write adapters for communication apps like Telegram or Discord, and configure reliable task schedulers.

By combining LangChain with the OpenClaw ecosystem via the LangClaw framework, you can bridge this gap in an afternoon.

Why combine OpenClaw and LangGraph?

OpenClaw began as a configuration-first, out-of-the-box personal AI runtime. It provides developers with a pre-wired ecosystem that includes direct messaging connectors for platforms like WhatsApp and Telegram, persistent memory layers, and native browser automation tools.

Instead of writing code to manage how an agent communicates with a chat application or remembers previous conversations, users simply write markdown files (SKILL.md) to define functions. OpenClaw handles the operational runtime, API routing, and state storage automatically.

While this structure is ideal for standard automation, engineering teams often require deeper programmatic control over their agentic reasoning. They need complex, multi-step state machines, advanced semantic search pipelines, and conditional execution paths.

This is where LangChain excels, and it is exactly why the two systems are paired together.

LangClaw as the glue layer

LangClaw acts as the architectural bridge between these two worlds. It is a declarative, Pythonic framework that compiles complex LangChain reasoning loops directly into an OpenClaw-compatible runtime engine.

Instead of managing external configuration markdown files, you use clean Python decorators to build tools, establish role-based access controls, and schedule autonomous background tasks.

If you’re the kind of person who reads something like this and immediately wants to see the code, we’ve documented a complete LangGraph + OpenClaw build that turns these ideas into a working corporate intelligence agent, covering everything from subagent orchestration and scheduled research tasks to permissions, messaging, and token-efficient utility commands.

Get the playbook here

Let’s build one.

The first step is creating the primary agent and establishing its responsibilities through a system prompt.

import os
from datetime import datetime
from langclaw import Langclaw
from langclaw.gateway.commands import CommandContext
# Initialize the master agent application interface
app = Langclaw(
 system_prompt=(
 “## Corporate Intelligence Agent\n”
 “You are a corporate intelligence analyst. You track market trends “
 “and draft precise outreach sequences based on current events.\n”
 “Delegate deep multi-source research tasks to the web-researcher subagent.”
 ),
)

Tools allow the agent to access capabilities beyond language generation. In this example, the agent can retrieve market intelligence data.

# Custom Tool: Data Extraction Layer
@app.tool()
async def fetch_market_news(sector: str) -> dict:
 “”“Extract breaking market developments for a specific commercial sector.”“”
 # This acts as a standard tool available to your core reasoning models
 return {
 “timestamp”: datetime.now().isoformat(),
 “sector”: sector,
 “headline”: “Warehouse Automation Demands Surge Amid Labor Reshuffling”,
 “body”: “Logistics providers face margin pressures, driving immediate infrastructure upgrades.”
 }

Rather than placing all reasoning responsibilities inside a single agent, LangClaw allows specialised subagents to handle focused tasks.

# Subagent Delegation: Isolated Reasoning Context
app.subagent(
 “web-researcher”,
 description=”Performs deep multi-step web analysis and synthesis.”,
 system_prompt=”You are an investigative researcher. Identify technical pain points and list them.”,
 tools=[”web_fetch”, “web_search”],
 output=”channel”, # Streams output directly back to the active user channel
)

Role-based permissions make it possible to expose different capabilities to different groups of users.

# Role-Based Access Control: Security Gateways
app.role(”premium_analyst”, tools=[”*”])
app.role(”basic_user”, tools=[”fetch_market_news”])

Agents do not need to wait for user prompts. Using cron schedules, they can execute tasks automatically.

# Automated Execution: Background Scheduler
@app.cron(”0 8 * * 1-5”) # Executes Monday through Friday at 8:00 AM
async def automated_morning_brief():
 “”“Scans market trends autonomously and publishes digests to the user feed.”“”
 raw_data = await fetch_market_news(sector=”Logistics”)
 instruction = f”Synthesize this market report for the main channel: {raw_data[’body’]}”
 # Compile runtime structure down to LangGraph for invocation
 compiled_agent = app.compile()
 analysis = await compiled_agent.ainvoke(
 {”messages”: [{”role”: “user”, “content”: instruction}]},
 config={”configurable”: {”subagent”: “web-researcher”}}
 )
 # Broadcast the completed report directly to your active chat channel
 await app.gateway.send_message(
 channel=”management_feed”,
 text=f”🔔 **Automated Market Digest**\n\n{analysis[’messages’][-1].content}”
 )

Not every task requires an LLM. For predictable workflows, slash commands can bypass the model entirely.

# Slash Command: High-Speed Utility Routing
@app.command(”template”, description=”Generate static outreach copy without AI overhead”)
async def outreach_template_cmd(ctx: CommandContext) -> str:
 “”“Returns a structured outreach framework directly, bypassing the LLM entirely.”“”
 args = ctx.message.text.split(’”’)
 client_name = args[1] if len(args) > 1 else “Colleague”
 # Bypassing the LLM completely saves computation costs and provides sub-millisecond responses
 return (
 f”📩 **System Template Generated**\n\n”
 f”Hello {client_name},\n\n”
 f”I noted your team’s footprint in the logistics sector. Given recent automation trends, “
 f”I wanted to connect to share notes on how regional hubs are optimizing their margins.\n\n”
 f”Regards,\n[Internal System]”
 )

Once the components are assembled, launching the runtime is straightforward.

if __name__ == “__main__”:
 app.run()

Once the runtime is launched, the agent becomes more than a collection of functions and workflows. It becomes a persistent service that can receive requests, execute tasks, and communicate results through your chosen channels.

Let’s see how that works.

What happens when the agent goes live

When you invoke app.run(), LangClaw spins up a persistent asynchronous gateway server. It does not wait for a single terminal input; it establishes a permanent network listener.

Interaction happens entirely through your chosen deployment endpoints:

• Production chat channels: By adding tokens for Telegram, Discord, or Slack into your .env configuration file, your script logs into those platforms as a live bot user.

• LLM invocations: When a user types a natural language question into the chat, LangClaw pipes the string into your LangChain model, runs the necessary code tools, and formats the response.

• The slash command route: When a user types /template “Alex”, the gateway identifies the leading slash character. It isolates the instruction, skips the LLM reasoning step, runs the native Python function, and returns the text.

  ---

Hosting your agent for less than £5 per month

Deploying a traditional enterprise application often brings significant server management costs. However, because this framework relies on an event-driven architecture and asynchronous IO libraries, it requires minimal computing resources.

You can host this entire stack for under £5 per month using modern cloud infrastructure.

Virtual Private Servers (VPS)

Platforms like DigitalOcean, Hetzner, or Linode offer entry-level Linux instances with 1GB of RAM and 1 CPU core. Because LangClaw uses a non-blocking asyncio event loop, a single micro-instance can easily manage hundreds of concurrent chat messages and automated cron tasks without breaking a sweat.

Process Management

To ensure your agent runs continuously, you wrap the script using a Linux system process supervisor like systemd or Supervisor. If the server reboots or encounters an unhandled API timeout, the operating system brings the agent back online instantly, providing 99.9% uptime with zero manual maintenance.

We’ve set up an agent, we’ve talked about messaging and about the advantages of full control, about the hosting. Building your automation stack with this methodology yields three immediate operational improvements:

• Zero-token utilities: By routing routine structural actions through slash commands, you bypass LLM token fees entirely. This provides instantaneous responses while eliminating API runtime costs for predictable tasks.

• Decoupled architecture: Your core business logic remains completely separate from your communication channels. If your team decides to migrate from Telegram to Slack, you alter a single environment variable. The underlying tools, subagents, and schedules require no modifications.

• Production readiness: You move from an abstract local script to an enterprise-grade agent. The system manages its own user security roles, schedules its own data retrieval cycles, and operates autonomously around the clock.

Let us know how you get on!

You can watch the video here!

The common assumption is that small models are simply the cheaper version of frontier models. Same idea, lower cost. But Maxime’s experience has been that they’re often harder to work with. Not because they’re worse, but because they expose problems that larger models can often hide.

When most people build AI systems today, they’re testing them with frontier models. Those models are smart enough to compensate for weak prompts, incomplete logic, or edge cases that nobody thought about. Small models don’t give you that luxury. As Maxime put it, they can fail on surprisingly basic tasks, and when they do, entire workflows can break.

That changes the engineering. You need stronger guardrails, better error handling, and less reliance on the model figuring things out for itself. In some ways, small models force you to build more robust systems because they expose weaknesses earlier.

The tradeoff is that you get something valuable in return. Small models can run on phones, laptops, and local infrastructure. They’re private, can work offline, and give you a level of control that isn’t always possible with centralized frontier models.

Maxime doesn’t see this gap lasting forever, however. He pointed to techniques like distillation and reinforcement learning as reasons small models are steadily improving, particularly when it comes to agent capabilities.

Agentic Engineering grew out of a desire to create something useful for people trying to make sense of agentic AI and beyond. If you’d like to support that direction, please tell your friends about us.

Share Agentic Engineering

Those questions still matter as foundation models remain the foundation of everything happening in AI today. But it feels like the centre of gravity has shifted (slightly at first and then all at once).

The work happening today isn’t about making a model better at answering questions. It’s about figuring out how to build systems around those models that can reason, retrieve information, use tools, coordinate actions, and operate live workflows.

We had our own theories about where the AI conversation was heading, but theories are cheap. If Agentic Engineering was going to be built around the questions people actually care about, there was only one group worth asking.

So we turned to the people reading this

A few weeks ago, we asked you what you wanted more of from AI coverage. While the responses covered a wide range of interests, the overall message was consistent (and I’m not surprised).

You weren’t asking for more AI news.

You weren’t asking for more model rankings.

And you certainly weren’t asking for more hype.

The most common frustrations with AI newsletters today were too much hype, a lack of practical depth, and repetitive coverage that often feels disconnected from how AI is actually being used. What you wanted instead was real-world workflows, implementation advice, clear explanations of complex topics, and lessons from teams building with these technologies in practice.

The interesting part wasn’t that AI agents ranked first. Honestly, I expected that.

What caught my attention was everything sitting underneath it.

Open-source AI. Infrastructure. RAG systems. Multi-agent systems. Agent orchestration. Evaluation.

At some point, it stopped looking like interest in a technology category and started looking like interest in an engineering discipline.

Taken together, those responses point toward something bigger than a content preference. They point toward a shift in the conversation itself.

Once AI systems begin interacting with databases, applications, workflows, enterprise systems, and even other agents, an entirely new set of questions emerges.

• How should these systems be evaluated?

• How do you observe and debug them?

• What does governance look like when AI can take actions rather than generate outputs?

• How much autonomy is too much?

• How do you move from an impressive demo to something people can actually trust?

These aren’t purely research questions. They’re engineering questions that sit at the intersection of software engineering, AI engineering, operations, product design, governance, and systems thinking.

That’s why Agentic Engineering was born

Not because every workflow needs an agent. Not because every company should rush to deploy autonomous systems. And certainly not because we’re interested in chasing the latest buzzword.

Agentic Engineering exists because a distinct engineering discipline is beginning to emerge around how these systems are designed, evaluated, deployed, governed, and integrated into real-world environments.

Where we go from here

Over the coming months, we’ll be exploring that discipline through expert-written analysis, interviews with practitioners working close to the technology, research breakdowns, case studies, and conversations with the people building these systems every day.

The truth is that nobody has all the answers yet. The playbook for building reliable agentic systems is still being written. Best practices are emerging in real time. Some ideas will prove durable. Others won’t survive contact with production.

We’re watching a new engineering discipline take shape, and we’d like to document it as honestly as possible. The breakthroughs, the lessons, the contradictions, the failures, and the things that genuinely move the field forward.

If you’re trying to make sense of where agentic systems go from here, you’re in the right place 🙂

That’s what made this roundtable session so interesting for my co-host, Ali Abidi, and me.

We spent an hour digging into these questions with three people who’ve helped shape how the industry thinks about AI engineering: Chip Huyen, Paul Iusztin, and Maxime Labonne.

Chip Huyen brought the systems-level perspective she’s become known for through Designing Machine Learning Systems and AI Engineering: why reliability, orchestration, tooling, memory, evaluation, and infrastructure choices are quickly becoming more important than flashy demos.

Paul Iusztin and Maxime Labonne, co-authors of LLM Engineer’s Handbook, brought deeply practical perspectives from the frontlines of modern AI engineering, especially around modular architectures, production workflows, multimodal systems, post-training, fine-tuning, and the engineering discipline needed to move beyond PoC purgatory and into systems that can actually survive real-world use.

The conversation stayed honest throughout. Nobody pretended this space is “solved.” The discussions revolved around trade-offs, failure points, bottlenecks, orchestration chaos, and the messy engineering layer that starts appearing the moment agents interact with real systems and real users.

Huge thanks again to Chip, Paul, and Maxime for such a thoughtful conversation. Ali and I had an absolute blast hosting this one.

Thanks for reading!

Agentic Engineering grew out of a desire to create something useful for people trying to make sense of agentic AI and beyond. If you’d like to support that direction, please tell your friends about us.

Share Agentic Engineering

Once agents begin using tools, retrieving information, maintaining memory, looping through decisions, and interacting with external systems, things stop looking like a simple chatbot pretty quickly. You start ending up with something closer to an operational system that needs planning, structure, and supervision to keep functioning coherently.

And that’s the part of the conversation that still feels oddly underdiscussed.

A lot of attention goes toward models and autonomy, but much less toward the layer responsible for coordinating all of this.

That layer is the orchestrator. And the more complex agentic systems become, the more important the orchestration layer becomes.

Join Angentic Engineering

For builders navigating the move from models to agents

An AI orchestrator serves as the central hub that coordinates interactions between the model, tools, memory stores, APIs, and other external systems. It ensures that AI agents operate effectively and in a controlled manner.

AI orchestrators can help in the following:

• Managing complexity: AI workflows often involve multiple coordinated steps such as retrieval, reasoning, and action execution. Orchestrators automate and structure these processes, making systems easier to scale and maintain.

• Enhancing scalability: Orchestrators handle high loads by distributing tasks, caching responses, and parallelizing operations—critical for handling multiple users or token-intensive tasks.

• Ensuring context awareness: Since LLMs have limited memory, orchestrators integrate vector databases and memory systems to help agents retain information and deliver more coherent, personalized experiences.

• Facilitating tool integration: Orchestrators streamline the use of APIs, search engines, and databases by managing task execution and ensuring smooth interaction between LLMs and external tools.

• Improving reliability and monitoring: From logging to human-in-the-loop feedback, orchestrators offer tools to catch errors, prevent hallucinations, and ensure systems run securely and reliably.

To better understand the need for an AI orchestrator in the context of AI agents, we need to introduce three key features of AI agents: autonomy, abstraction, and modularity.

Subscribe now

Autonomy

Autonomy refers to an AI agent’s capacity to operate independently, making decisions and executing actions without human intervention. This self-directed behavior enables AI agents to perform tasks, adapt to new situations, and pursue goals based on their learned experiences.

The autonomy of an AI agent implies that the steps that the agent is going to take are not necessarily known in advance.

For example, let’s consider a non-agentic workflow as simple as the following:

Whenever we prompt an LLM, we are doing an API call, which is the only step that comprises this workflow. Even in the scenario of a retrieval augmented generation (RAG) workflow, steps are known in advance

Let’s now consider an autonomous agentic approach. Let’s say we have an agent with two tools:

• Weather tool: a function that takes two parameters: city and unit of measurement.

• Location tool: a function that takes the current position of the user, leveraging the GPS position. This function takes no parameters.

Both functions come with a natural language description, as per the AI agent’s anatomy. Our workflow design will let the agent decide which tool to invoke

Let’s say that a new user asks, “What’s the weather for tomorrow?” The following things will happen:

1. The agent will read through the descriptions of its tools and understand that it needs to invoke the weather tool. However, it’s missing the two parameters, but thanks to its autonomy, it can look around to retrieve them. It soon understands that it can leverage the location tool to get the first parameter: the output of that function will serve as the parameter of the weather tool.

1. For the second parameter, the agent cannot make it alone: it needs to ask the user. So it does, asking the user which kind of unit of measurement is needed. Once the user responds, the agent is able to properly invoke the tool with both parameters.

1. The agent observes the output of the weather tool and ascertains that it now knows the final answer for the user.

Can you imagine how many “if…else” statements would have been needed to replicate such a degree of autonomy in a standard Robotic Process Automation (RPA) process? And even if we could manage a similar scenario, what if the user asks something that has not been hardcoded in the process? Adaptability, self-critique, and self-adjustment are key features of agentic autonomy.

There are many degrees of autonomy we can provide our agents with: it all boils down to the workflow we set up and the planning strategy we instruct our agents to follow. Designing the proper agentic workflow is an architectural design conversation that is core when building your agentic state.

Abstraction and modularity

Abstraction refers to breaking down and simplifying complexity. It is what makes these systems comprehensible and scalable. ​But beyond simplification, it also enables modular design, a fundamental principle in building intelligent systems.

Modularity breaks down complex problems into smaller, reusable components, each handling a specific part of the challenge. This approach offers several advantages:

• Interchangeability: Components can be swapped, upgraded, or replaced without affecting the entire system

• Reusability: Well-designed modules can be repurposed across different projects, improving efficiency

• Scalability: Independent yet seamlessly integrated components make it easier to expand solutions

In multi-agent systems, abstraction and modularity allow for the creation of cooperative agents, each specializing in specific tasks while interacting dynamically. This mirrors human problem-solving, where we divide, delegate, and collaborate to tackle complexity effectively.

A great way to understand abstraction and modularity in agentic patterns is by looking at a multi-agent traffic management system in a busy metropolitan city, where different levels of agents handle different levels of abstraction, ensuring smooth operation without overwhelming any single entity.

At the most granular level, we have intersection controllers, which operate at a single traffic light or intersection. These agents rely on real-time data from cameras and sensors to adjust traffic signals based on vehicle congestion, pedestrian movement, and emergency vehicle priority.

They don’t worry about what’s happening in the next city block or the broader urban landscape; their only job is to optimize traffic flow at their specific location. If a sudden influx of cars appears at a junction, they may extend green-light durations to ease congestion.

Zooming out, we have district-level traffic coordinators. These agents don’t micromanage individual traffic lights but instead analyze traffic flow across multiple intersections within a neighborhood or district.

They use data from intersection controllers, GPS tracking, and public transport systems to identify congestion patterns, reroute vehicles, and balance the flow of cars across the area. If they detect excessive delays in one zone, they adjust how much time a light is on for the traffic signal for multiple intersections rather than just one.

More importantly, they direct intersection-level agents, ensuring their adjustments align with broader district-wide traffic goals.

At the highest level, we have the citywide traffic management system, responsible for optimizing the flow of millions of vehicles across the entire metropolitan area. This agent doesn’t focus on specific traffic lights or individual congestion points; instead, it allocates resources, predicts long-term patterns, and makes strategic adjustments.

Using data from weather reports, major event schedules, accidents, and public transportation networks, this agent might reroute entire roads, coordinate construction schedules to minimize disruption, or implement city-wide emergency response plans in case of major incidents.

If an accident occurs on a major highway, the citywide system redirects district-level agents to adjust traffic patterns, which in turn instructs intersection controllers to reroute vehicles efficiently.

This layered structure demonstrates the power of abstraction and modularity in multi-agent systems:

• Intersection agents handle local, real-time decisions, adjusting traffic lights and prioritizing immediate flow

• District-level agents analyze and coordinate groups of intersections, optimizing traffic across a wider region

• Citywide agents focus on the big picture, planning for long-term efficiency, emergency responses, and systemic optimizations

This mirrors how software architectures, AI systems, and even corporate structures function in the real world. Whether it’s frontline workers executing tasks, middle managers coordinating efforts, or executives setting the overall vision, abstraction enables complex systems to remain scalable, efficient, and resilient.

By designing multi-agent AI architectures with this layered approach, we ensure each agent focuses only on what it needs to handle, preventing system overload and enabling adaptive, real-time decision-making at scale, just like a smart traffic system managing a bustling city.

If this doesn’t sound like a real thing to you, let’s have a look at OpenAI’s tool called Operator, which acts as an autonomous agent, capable of performing tasks in a web browser, such as booking tickets or filling online orders.

OpenAI’s Operator follows a hierarchical multi-agent approach similar to the traffic management system. Each agent operates at a different level of abstraction, ensuring efficiency and adaptability without overwhelming any single component.

• Web controllers (low-level agents): These agents handle execution: moving the mouse, clicking buttons, and entering text. They don’t analyze or plan—they simply follow commands.

• Vision and reasoning (mid-level agents): These agents interpret the web interface. The Vision Agent processes screenshots, detecting relevant elements, while the Reasoning Agent determines the next action (clicking, typing, or scrolling). This layer abstracts away the details of execution, focusing on understanding and decision-making.

• The planner/orchestrator (high-level agent): The top-level agent oversees the entire system, ensuring that web interactions align with broader goals—whether it’s searching for information or filling out a form. It delegates tasks to mid-level agents, ensuring smooth and strategic navigation.

This structured approach highlights why abstraction matters so much in multi-agent design. Low-level agents can focus purely on execution without needing to reason about broader objectives, while mid-level agents handle interpretation, coordination, and planning across tasks. Above them, high-level agents manage overall strategy and orchestration without getting pulled into implementation details. Separating responsibilities this way allows complex systems to remain scalable, adaptable, and manageable without overwhelming any single layer of the architecture.

By leveraging this modular design, OpenAI’s Operator adapts dynamically, handling different websites without requiring manual programming. This scalable and generalizable architecture is a prime example of how multi-agent systems drive real-world AI applications.

From an architectural perspective, all these components – agents, skills, plugins – can be seen as repeatable assets in your organization. In this context, AI orchestrators ensure that these components work together without being tightly coupled, preventing complexity from overwhelming the system.

Following the preceding hierarchical example, with an AI orchestrator, you can easily define the following:

• Execution agents (low-level): These handle raw tasks such as API calls, database queries, or web scraping, executing commands without decision-making

• Reasoning agents (mid-level): They analyze data, determine actions, and select the right tools, abstracting execution details

• Orchestration and planning (high-level): The orchestrator oversees workflows, breaking down tasks, distributing them across agents, and adapting dynamically

By structuring AI systems this way, orchestrators enable adaptive, generalizable intelligence, ensuring seamless interaction between components without manual intervention.

This excerpt was adapted from AI Agents in Practice by Valentina Alto, which does an excellent job of grounding these ideas in real architectural patterns instead of treating agents as abstract autonomous magic. The book goes deep into orchestration frameworks, multi-agent systems, memory architectures, tooling, production deployment, and the operational realities of building AI agents that can function beyond demos.

If you’ve been trying to separate what’s structurally important in the agent ecosystem, this book is worth spending time with.

Explore the book

And I think that’s why the conversation around agents sometimes feels simultaneously overcomplicated and vague. In reality, though, these systems are operating at very different levels of autonomy.

So, after spending the last few articles looking at how agents are structured and orchestrated beneath the surface, I wanted to take a new angle: the different kinds of agents organizations are building, and how capabilities change as systems move from retrieval into action and eventually toward autonomy.

Join Angentic Engineering

For builders navigating the move from models to agents

Looking at agents through this lens, the differences between these systems start becoming much easier to spot.

Some are primarily designed to retrieve and synthesize information more intelligently. Others are built around executing workflows and operational tasks. And some are beginning to operate with much higher levels of autonomy, coordinating decisions and adapting dynamically as conditions change.

Broadly speaking, most AI agents today fall into three categories: retrieval agents, task agents, and autonomous agents.

Retrieval agents

Retrieval AI agents build upon the foundations of RAG but incorporate advanced agentic behaviors, making them more autonomous and adaptive. In fact, we are adding to a standard RAG pipeline an additional layer of intelligence and planning that allows the agent to “strategize” on how to retrieve the most relevant pieces of information.

Let’s consider the following example.

Let’s say we want to build an AI assistant for a doctor to quickly retrieve information about treatments. Given the doctor asks: “What are the latest treatments for Type 2 diabetes?”, let’s see how the two approaches compare:

Traditional RAG approach:

• The RAG system retrieves the top three relevant articles from the database.

• The model extracts relevant text from those articles and generates a response summarizing key treatments.

• If the retrieved documents do not fully answer the doctor’s question, the model cannot refine the search unless the doctor manually submits a new query.

Retrieval AI agent approach

• The agent retrieves an initial set of documents and analyzes them.

• It detects that some retrieved studies are outdated, so it refines its search criteria and retrieves more recent publications.

• It recognizes a gap in information regarding a specific drug and fetches a dedicated study on that drug.

• Finally, it synthesizes all retrieved sources into a comprehensive answer, ensuring relevance and completeness.

In conclusion, agentic RAG can lead to several improvements over traditional RAG. Unlike traditional RAG systems that typically rely on a single retrieval pass, retrieval AI agents can iteratively refine their searches by breaking complex queries into multiple steps and progressively improving the relevance of the information they gather. They also maintain contextual awareness through memory of previous interactions, allowing them to ask clarifying questions, adjust retrieval strategies dynamically, and respond more intelligently over time.

Another important difference is their ability to interact directly with tools such as APIs, databases, and vector search engines, enabling them to fetch structured and real-time information when needed. Rather than pulling information from a single static source, retrieval agents can also augment knowledge adaptively by combining and synthesizing information across multiple sources depending on the context of the query. In many cases, these systems can even make autonomous decisions about when additional retrieval is necessary, which sources are most relevant, and how search results should be refined to improve the overall quality of the response.

Retrieval agents are the simplest form of AI agents, yet the extra layer of intelligence is already demonstrating great improvements for the overall user experience. However, the real power of an AI agent comes to life when it can combine retrieval skills with actionable tasks.

Task agents

Task agents go beyond information retrieval by performing specific actions. These agents are designed to automate workflows and replace repetitive tasks for users. Unlike retrieval agents, they execute predefined actions in response to user commands or external triggers.

Let’s consider once again an example in the healthcare domain, this time from the perspective of the general practitioner’s office receptionist, John.

John manages a high volume of appointment requests. Patients book visits through various channels: phone calls, emails, and an online booking system. Managing last-minute cancellations and rescheduling requests is time-consuming and often leads to gaps in the schedule.

A typical process in John’s day might look like the following:

1. John receives an email from Patient X to book an appointment and shares some preferences in terms of date and time

2. John checks the availability of the specialist practitioner required and tries to match the earliest slot possible with Patient X’s preferences

3. John doesn’t find any match, hence goes back to Patient X to find alternatives

4. Finally, John and Patient X agree on a slot and the appointment is scheduled

If you think about the above steps, they are nothing but tasks that John is meant to perform to achieve the goal – scheduling the appointment in an optimized slot for both the practitioner and the patient.

Whenever we want to map and enhance a business process with an AI agent – more specifically, a task agent – a good practice is that of transposing the human tasks into agentic tasks. Let’s see, for example, how a task agent can assist John:

1. The AI agent automatically scans the email received from Patient X. It extracts key details like the Patient’s name and contact details, preferred date and time, and the specialist practitioner required.

2. The AI agent checks the availability, invoking the plugin (the tool we equip our agent with) in the clinic’s scheduling system. It matches Patient X’s preferences with the earliest available slots for the specialist practitioner. If there’s a match, it proceeds to Step 5.

3. The AI agent finds no match. Since no match is found, the AI agent generates a list of the next best available slots based on the specialist’s schedule. It also drafts a response email to Patient X, leveraging a writing skill, with suggested alternatives, but John reviews and approves it before sending.

4. Patient X responds with a new preference and either:

   • Accepts one of them and proceeds to Step 5

   • Requests new options, then the AI agent repeats Step 3

5. Once John and Patient X agree on a slot, the AI agent automatically schedules the appointment in the system, leveraging the same plugin as above. Plus, it sends a confirmation email to Patient X with the details, leveraging an email plugin. Finally, it updates the specialist’s calendar and notifies them of the booking.

As you can see, the AI agent acts as John’s assistant, handling repetitive scheduling tasks while he focuses on in-person patient interactions.

Autonomous agents

Autonomous agents represent the most advanced category of AI agents. Unlike retrieval and task agents, which operate within predefined boundaries, autonomous agents strategically orchestrate multiple tasks and retrieval processes, making real-time decisions to optimize workflows. These agents exhibit a high degree of independence, adaptability, and contextual awareness, allowing them to perform complex operations with minimal human intervention.

The core distinction of autonomous agents lies in their ability to:

1. Combine retrieval and action: They can both find information (like a retrieval agent) and act on it (like a task agent).

2. Plan and self-adjust: They dynamically adapt based on new information or changing constraints.

3. Perform multi-step workflows: They break down complex tasks into subtasks, execute them iteratively, and adjust based on results. Let’s continue with John’s clinic example. As the clinic gets busier, managing appointments, cancellations, and reschedules becomes overwhelming. A task agent helped streamline individual actions, but now an autonomous agent takes over the end-to-end scheduling process with minimal supervision. Here is how it works, step by step:

   • Intake and prioritization: The agent monitors all channels (email, portal, phone transcripts), extracts patient preferences, urgency, and specialist needs, and ranks requests based on priority. For example, a canceled appointment opens a slot, and the agent immediately matches it to Patient X, who’s been waiting for a similar time.

   • Planning and optimization: It reviews the full daily schedule, identifies conflicts or idle gaps, and builds an optimized plan—shuffling low-priority visits to make room for urgent ones.

   • Execution with feedback: The agent messages patients with options, updates calendars, books appointments, and sends confirmations—all automatically. If preferences change, it loops back, refining its actions.

   • Real-time adaptation: A doctor calls in sick. The agent halts new bookings, reschedules affected patients, and notifies staff—handling all steps autonomously unless human input is needed.

   • Continuous learning: At day’s end, it analyzes outcomes, updates patient preferences, and adjusts future prioritization logic.

The autonomous agent can plan, retrieve, decide, act, adapt, and learn—all without relying on predefined workflows. John now focuses on edge cases, while the agent intelligently handles the rest.

Autonomous agents represent the next step in AI-driven process automation. By merging retrieval AI capabilities (context awareness, real-time query refinement) with task execution skills (appointment scheduling, automated notifications), autonomous agents can fundamentally reshape business processes and daily operations.

There are different degrees of autonomy we can provide our agents with, and the decision is based upon the business scenario as well as the level of confidence we have in the accuracy of the solution.

This piece borrows from AI Agents in Practice by Valentina Alto, which I found myself bookmarking constantly while working through these recent pieces on agents.

What stayed with me most wasn’t just the frameworks or implementation details, but how clearly it separates systems that retrieve, systems that execute, and systems that can independently coordinate decisions across workflows. That sounds obvious when written out plainly, yet a surprising amount of the current ecosystem still collapses all three into the same conversation. The more separated those ideas become, the easier it gets to reason about what these systems are capable of doing.

Expore the book

But while putting that piece together, I kept noticing something interesting. Many of these systems sound new on the surface, yet a lot of the underlying ideas have existed for much longer than the current wave of agent frameworks.

Once you start looking at older work around autonomous systems and classical AI architectures, you begin noticing familiar tensions everywhere. Systems that prioritize planning versus systems that prioritize responsiveness. Architectures built around reasoning versus architectures built around reaction. Trade-offs between control, adaptability, speed, and autonomy.

So it made sense to follow up the previous piece with this one.

I’m not saying modern LLM agents map perfectly onto these older paradigms. Understanding deliberative, reactive, and hybrid architectures, however, makes today’s agent ecosystem feel a little less chaotic and a lot more interpretable.

This article explores those architectural foundations and how many of those ideas continue to echo through the systems being built today.

Join Angentic Engineering

For builders navigating the move from models to agents

If you strip things back, agentic systems are just different ways of organizing how an autonomous system perceives its environment, reasons about decisions, responds to changing conditions, and carries out actions toward a goal.

The interesting part is that there’s no single architectural approach for doing this well. Some systems prioritize planning and structured reasoning. Others are optimized for fast reactions and adaptability. And many modern systems attempt to balance both.

Broadly speaking, most agentic architectures tend to fall into three categories: deliberative, reactive, and hybrid architectures. Each one reflects a different philosophy around how intelligent systems should behave, make decisions, and interact with the world around them.

Let’s walk through them.

Deliberative architectures

Also known as knowledge-based or symbolic architectures, deliberative architectures rely on the use of explicit representations of knowledge and reasoning mechanisms to reach decisions. They typically follow a sense-plan-act cycle, where they first perceive information about the environment, then make a plan of action according to that perception and the knowledge base, and finally execute such plans of action.

The key advantage of deliberative architectures is their ability to handle tasks that involve complex reasoning, such as planning, problem-solving, and decision-making. These architectures leverage techniques such as rule-based reasoning, constraint satisfaction, and heuristic search to navigate through intricate problem spaces and formulate appropriate courses of action.

One of the critical components of a deliberative architecture is the knowledge base that stores symbolic representations of the environment, goals, constraints, and domain-specific knowledge. This knowledge base is typically encoded using formal language or logic, enabling the system to perform logical inference and reasoning. The sense-plan-act cycle in deliberative architectures typically involves the following steps:

1. Sensing: The agent perceives and acquires information about the environment through various sensors or input mechanisms.

2. Knowledge updating: The perceived information is used to update the agent’s internal knowledge base, ensuring that it maintains an accurate representation of the current state of the environment.

3. Planning and reasoning: Based on the updated knowledge base, the agent employs reasoning techniques and algorithms to formulate plans and make decisions. This may involve techniques such as constraint satisfaction, logical inference, search algorithms, or heuristic-based planning.

4. Plan execution: Once a plan or course of action has been determined, the agent executes the corresponding actions in the environment, potentially modifying the environment or achieving specific goals.

Deliberative architectures excel in handling tasks that require complex reasoning, planning, and decision-making in well-defined environments. They can effectively handle uncertainty and ambiguity through techniques such as probabilistic reasoning, fuzzy logic, or belief revision mechanisms.

However, deliberative architectures also have some disadvantages. One significant challenge is the computational cost associated with maintaining and reasoning over complex knowledge bases, which can limit real-time responsiveness in dynamic environments. Additionally, the explicit representation of knowledge can be challenging in domains where knowledge is difficult to formalize or is constantly evolving.

To address these limitations, deliberative architectures are often combined with reactive or behavior-based components in hybrid architectures, allowing both complex reasoning and rapid response to environmental changes.

Despite their limitations, deliberative architectures remain a crucial component in many intelligent systems, particularly in domains where complex decision-making, planning, and reasoning are essential, such as robotics, decision support systems, and intelligent tutoring systems.

Reactive architectures

Reactive architectures, also known as behavior-based or stimulus-response architectures, aim to provide immediate responses to stimuli from the environment. Unlike deliberative architectures, reactive architectures do not rely on explicit models of the world or complex reasoning processes. Instead, these systems directly map perceptions onto actions, typically using simple condition-action rules or neural networks as depicted in the following figure:

Some of the key properties and characteristics of reactive architectures include the following:

• Speed and responsiveness: Reactive architectures are designed to react rapidly to changes in the environment. By directly coupling perceptions to actions, they can bypass time-consuming deliberative reasoning processes, enabling swift and timely responses.

• Robustness and fault tolerance: These architectures are generally robust and less susceptible to noise or incomplete information. Their simple, standalone nature makes them less prone to catastrophic failures, as individual components or behaviors can compensate for or mitigate the effects of faulty or missing input, especially when used within a deliberative architecture.

• Handling uncertainty: Reactive architectures can effectively handle uncertainty in dynamic environments. Their ability to respond directly to environmental stimuli allows them to adapt and adjust their actions based on the current situation, without relying on precise or complete models of the entire world.

• Parallel and distributed processing: Reactive architectures often employ parallel and distributed processing using multiple reactive modules, where multiple modules operate simultaneously and independently. This decentralized approach enables efficient handling of complex tasks and provides inherent scalability and modularity.

• Emergence of complex behavior: Despite the simplicity of individual behaviors or rules, the interaction and coordination of multiple reactive components can lead to the emergence of complex, intelligent-like behavior at the system level.

While reactive architectures offer advantages in terms of speed, robustness, and handling uncertainty, they also have limitations as highlighted:

• Lack of long-term planning: Reactive architectures cannot generally plan or reason about long-term consequences. Their focus is on immediate responses to environmental stimuli, making it difficult to pursue complex, multi-step goals or strategies.

• Limited reasoning and abstraction: These architectures may struggle with tasks that require abstract reasoning, generalization, or the manipulation of symbolic representations. They are primarily designed to operate at a lower, stimulus-response level.

• Limited learning capabilities: Many reactive architectures cannot learn from experience or adapt their behavior over time. Their fixed set of rules or behaviors may not be suitable for dynamic environments or tasks that require continuous learning and adaptation.

Despite these limitations, reactive architectures are widely used in applications where real-time responsiveness, robustness, and the ability to handle uncertainty are essential, such as in robotics, video games built with AI, and control systems. Additionally, reactive architectures often serve as components within more complex hybrid architectures, complementing deliberative or learning-based systems to achieve desired levels of performance and adaptability.

Hybrid architectures

Researchers have recognized the strengths and limitations of both deliberative and reactive architectures, leading to the development of hybrid architectures that aim to exploit the advantages of both approaches. Such hybrid architectures typically employ a layered structure, consisting of the following:

• A reactive layer for fast and low-level responses. The reactive layer is responsible for handling real-time interactions with the environment, providing rapid and situationally appropriate responses to external stimuli. This layer is designed to be highly responsive, fault-tolerant, and capable of handling uncertainty, leveraging the strengths of reactive architectures.

• A deliberative layer for high-level reasoning and planning. The deliberative layer is dedicated to higher-level reasoning, planning, and decision-making processes. This layer can maintain a more comprehensive representation of the environment, goals, and constraints, enabling it to formulate complex strategies, reason about abstract concepts, and plan long-term courses of action.

The interaction between these two layers is crucial for enabling agentic systems to respond effectively to dynamic environmental contexts while maintaining the capability to plan actions and reason about them. The reactive layer can provide real-time feedback and situational awareness to the deliberative layer, informing its decision-making processes. Conversely, the deliberative layer can guide and influence the reactive layer’s behavior by providing high-level plans, goals, and constraints.

To achieve complex goals and leverage the strengths of both layers, hybrid architectures often employ the following techniques:

• Task decomposition: Break down complex tasks into subtasks that can be handled by the appropriate layer, with the reactive layer handling low-level, time-critical tasks and the deliberative layer focusing on higher-level planning and coordination

• Multiplan selection: The deliberative layer can generate multiple potential plans or strategies, and the reactive layer can dynamically select and execute the most suitable plan based on the current environmental conditions

• Planning with external modules: The deliberative layer can incorporate external modules or specialized algorithms for tasks such as path planning, resource allocation, or scheduling, leveraging domain-specific knowledge and techniques

• Reflection and refinement: The deliberative layer can reflect on the outcomes of executed plans, learn from experience, and refine its reasoning and planning processes accordingly, enabling continuous improvement and adaptation

• Memory-augmented planning: The deliberative layer can maintain a memory or history of past experiences, decisions, and outcomes, enabling it to leverage this knowledge in future planning and reasoning processes

By combining the strengths of both deliberative and reactive approaches, hybrid architectures seek to balance responsiveness and reasoning, enabling the development of more robust, autonomous, and adaptable agentic systems. These architectures leverage the power of both approaches, providing the ability to respond rapidly to dynamic environments while maintaining the capability for complex planning, reasoning, and decision-making.

The design and implementation of effective hybrid architectures remain an active area of research, as researchers strive to develop architectures that can seamlessly integrate and coordinate the deliberative and reactive components, enabling the creation of highly capable and intelligent agentic systems.

Selecting the appropriate architectural pattern for an agentic system is contingent upon the specific requirements of the application, encompassing factors such as task complexity, environmental uncertainty, and the necessity for real-time responsiveness. Deliberative architectures excel in scenarios that demand intricate reasoning and decision-making processes, while reactive architectures thrive in dynamic environments that necessitate swift and adaptive responses. Hybrid architectures strike a harmonious balance by judiciously leveraging the strengths of both paradigms, resulting in the development of more capable and adaptable agentic systems that can seamlessly navigate the complexities of their operating environments.

This article was adapted from Building Agentic AI Systems by Anjanava Biswas and Wrick Talukdar.

Reading through it, what stood out to me was how much of agent design ultimately comes back to timeless systems questions, like how decisions are made, how tasks are decomposed, how coordination happens, and how systems adapt under changing conditions.

That feels relevant right now as current conversations around agents still focus more on tooling abstractions than the systems underneath them. If you enjoy thinking about agents as systems-design problems rather than just autonomous demos, this book is worth exploring further.

Explore the book

However, most agentic systems tend to follow a small number of recurring design patterns. The differences usually come down to how an agent reasons, how it plans tasks, how it interacts with tools, and whether it operates alone or alongside other specialized agents.

Once you recognize these patterns, a lot of the ecosystem starts making more sense. You can begin to see why some systems are better suited for short reasoning tasks while others are designed for long-running workflows, delegation, or multi-step execution.

This article breaks down some of the most common patterns used in modern AI agents and how they differ.

Join Angentic Engineering

For builders navigating the move from models to agents

All AI agent design patterns contain the core three components, but have variations in how they implement the components. In particular, they differ based on how they control the control logic framework and its tooling sophistication. These include ReAct, CoT, planner-executor, and hierarchical/multi-agent.

CoT

CoT simply encourages the model to produce a step-by-step reasoning trace before generating a final answer. A literal interpretation of it would be to tell someone to think about a problem, create a plan on how they are going to solve it, and then take the steps to solve it. One limitation of CoT is that it’s unable to take action and, more importantly, unable to adapt its plan based on the results of actions.

ReAct (Reasoning + Acting)

ReAct is a pattern that we mentioned and described in pseudocode earlier. This design pattern enforces an AI agent to iteratively reason about the problem and choose a tool or action to take, feeding the result back into the continuous loop until its goal is achieved. The difference between ReAct and CoT is in their ability to act and then adjust their plan based on the results of that action. This is best for real-world agents that need to reason and interact with tools to solve complex tasks. This pattern is most often the go-to for most traditional AI agents.

Planner-execution

The planner-execution pattern separates the system performing the planning from the system performing the execution. An agent first generates a high-level plan, and then another agent (or multiple agents) executes each task in that plan. This pattern works well for very long-term complex tasks or tasks that can easily be modularized and delegated to sub-agents.

Hierarchical/multi-agent

A hierarchical/multi-agent pattern divides tasks among multiple specialized agents that are experts in that particular domain or task. This pattern can also perform work in a distributed manner, meaning that the system can finish tasks simultaneously rather than sequentially. This system is akin to a normal company, which contains a manager (a CEO) who delegates tasks to specialized workers with their own domain (CMO, COO, CFO, etc.).

For example, if we build an AI agent that resolves customer complaints, there may be an agent that receives the complaint and produces an action plan, and then specialized agents to perform those tasks, each with their own model, tooling infrastructure, and memory and knowledge (such as a customer agent that can look up customer information, a compliance agent that reviews policies, and a response agent that drafts a reply to the customer).

These patterns are not mutually exclusive. In fact, most real-world agents, especially those built with the frameworks briefly discussed next, tend to blend these approaches depending on the complexity of the task and the tools available. The OpenAI Agents SDK framework, for example, combined ReAct and hierarchical/multi-agent patterns.

In addition to the design patterns, companies have built different frameworks to develop and deploy AI agents, which differ based on the design pattern used and based on their feature set. These include the OpenAI Agents SDK, LangChain, LangGraph, AutoGen, AutoGPT, and Crew AI.

One thing that becomes clear when building agents is that there’s no single correct architecture.

That’s also part of why the agent space feels so noisy right now. Many of the frameworks being discussed often solve similar problems underneath, just with different abstractions and orchestration layers, as well as developer experiences wrapped around them.

Understanding the underlying patterns matters more than memorising framework names. Once you understand how reasoning, planning, tooling, and delegation fit together, it becomes much easier to evaluate new agent stacks without getting distracted by surface-level differentiation.

Thanks for reading!

Agentic Engineering grew out of a desire to create something grounded and useful for people trying to make sense of agentic AI beyond the weekly framework and abstraction noise.

If you’d like to support that direction, you may pledge your support below.

Pledge your support

That’s part of the reason I wanted to start this newsletter here, with fundamentals. Before the frameworks, abstractions, and “10x autonomous employee” headlines, it helps to understand the actual moving parts underneath these systems.

Join Angentic Engineering

For builders navigating the move from models to agents

AI agents follow a typical pattern and can be broken down into three fundamental components: model, tooling interface, and memory and knowledge.

Production-grade agents depend heavily on how well these components coordinate with one another. A capable model paired with poor tooling or weak memory management will, without a doubt, produce unreliable systems.

Model

The model is still the brain of the agent. It’s the part doing the reasoning, interpreting inputs, deciding what to do next, and trying to recover when things go sideways.

But the more you look at production systems, the harder it becomes to talk about models in simple leaderboard terms. The “best” model usually depends on what kind of compromises you’re willing to live with.

Some models are fast enough to feel seamless, but start struggling once tasks become more complex. Others reason well but move slowly enough to make the whole experience feel heavy. Some are great at coding, some handle multimodal workflows better, and some begin falling apart the moment you push them into long-running tasks with too much context.

And then there’s the strange reality that models don’t just differ in capability. They differ in personality almost. Different blind spots, different reasoning habits, different confidence levels, different failure modes. Some hallucinate more aggressively. Some become overly cautious. Some feel oddly persuasive even when they’re wrong.

Most AI agents are not powered by entirely different “agent models.” Many of them rely on the same families of frontier models, whether that’s GPT-class, Gemini-class, or Claude-class systems. What differs between these applications is not the model but the instructions provided to the model and how the model interacts with other components.

These “instructions” are provided to the model by defining a system prompt, which essentially defines the model’s role inside the system. When you really get down into it, a surprising amount of agent development involves refining prompts, adjusting tool descriptions, and reshaping interaction flows to make systems behave more reliably.

Control logic framework

Another important part of an AI agent is the control logic framework: the loop that lets the system observe what’s happening, reason about what to do next, take action, and then repeat that process until it reaches a goal.

What’s interesting is that this loop often does not happen “inside” the model itself. In many cases, it’s the surrounding agent framework that keeps the cycle running, feeding outputs back into the model, deciding when tools should be invoked, and determining whether another reasoning step is needed.

The process looks something like this:

Read user’s goal and create action plan

For each step in action plan:

Create action inputs

Execute action

Get result

Add result to memory

Modify action plan if necessary or if goal not achieved

If goal is achieved:

Return output to user

Different agent frameworks implement this loop differently, but most follow some variation of planning, acting, evaluating, and repeating. Patterns like CoT and ReAct show up constantly underneath modern agent systems.

Some frameworks even get a little strange with it. One pass generates the answer, and another pass comes back to critique the reasoning like an internal reviewer hovering over the system’s shoulder.

Tooling interface

One of the biggest differences between a normal LLM and an AI agent is that the agent can actually interact with the outside world instead of just generating tokens inside a chat window. That capability comes from the tooling interface, which gives the agent a structured way to decide when and how actions should be executed.

An email assistant, for example, might be equipped with actions that look something like this:

Action #1: Send an email

Description: Sends an email to a user

Parameters: to_email_address, email_subject, email_body

Action #2: List all emails

Description: Lists all emails, including email_id

Parameters: search_term (optional)

Action #3: Read an email

Description: Returns the content of an email

Parameter: email_id

Once these tools are available, the agent’s model and control logic framework decide when they should be used and what inputs they need.

An email assistant, for example, might search for emails related to a “coffee expansion proposal,” read through the relevant threads, generate a summary, and then send that summary to a manager. Underneath, the agent is simply chaining together a series of tool calls and feeding the outputs back into the reasoning loop.

The mechanics differ slightly depending on the framework being used, but the overall pattern stays fairly consistent. The agent needs awareness that the tools exist, a way to execute them, and a way to pass the results back into the model.

Memory and knowledge

Memory and knowledge are another part of what makes agents intelligent instead of just reactive. The two are related but slightly different.

Memory is about what the system remembers from interactions and previous context, while knowledge is about what it can retrieve from external sources when needed. In both cases, the goal is to give the model enough relevant context to make better decisions and produce more useful responses.

Let’s talk about each concept and how it’s used to provide more context to the AI agent. There are typically two types of memory: working and long-term.

Working memory

Working memory is basically the agent’s short-term memory: the running context from the current interaction.

It’s what allows systems like ChatGPT to understand that when you ask “How big is it?” after asking about the sun, the word “it” still refers to the sun instead of something completely unrelated.

Underneath, this usually works by continuously feeding recent conversation history back into the model as context. That’s what makes follow-up questions and multi-step conversations possible in the first place.

But this memory is limited. Context windows eventually fill up, older information starts dropping out, and frameworks have to decide what information is worth keeping around. A surprising amount of agent design ends up revolving around that question alone.

Long-term memory

Long-term memory is what allows an agent to carry information across sessions instead of treating every interaction like a completely fresh start. Without long-term memory, an agent is basically stateless: every request exists in isolation. With memory, the system can begin recalling preferences, past interactions, recurring tasks, or relevant context from earlier conversations.

This usually means storing information in some external database or memory layer that the agent can retrieve later through tools. So an email assistant, for example, might remember how you typically write messages, who you communicate with most often, or the kinds of follow-ups you usually prefer. Your AI agent may have the following tooling, which enables it to store and recall memories:

Action #1: Store information

Description: Stores important information about the user

Parameters: information

Action #2: Read information

Description: Retrieves important information about the user

Working memory is usually the easy part. Most systems already know how to keep track of the current conversation. Things get more interesting once agents need to remember information across sessions or retrieve context from external knowledge sources in real time. That’s usually where extra infrastructure starts creeping in: vector databases, retrieval pipelines, memory stores, ranking systems, all the machinery needed to stop the agent from feeling like it has permanent short-term memory loss.

Knowledge is slightly different from memory, though. Memory is about recalling previous interactions. Knowledge is about pulling in relevant information from outside sources like documents, databases, file repositories, or internal company systems.

And the moment agents start operating inside real workflows instead of isolated chats, that distinction starts mattering a lot more.

Training knowledge

Training knowledge refers to the information already baked into the model through its training data. It’s why an LLM can answer something like “How big is the sun?” immediately without needing to search the web or retrieve a document first.

That capability is still the thing that makes these systems feel a little unreal sometimes. A model can absorb enormous amounts of general knowledge and then reshape it fluidly around whatever the user is asking.

But once you move into AI agents, that baked-in knowledge starts becoming less useful on its own.

Agents usually need information that’s contextual, recent, proprietary, or tied to a specific workflow. And the model’s training knowledge can’t help much there. It doesn’t know what happened after its cut-off date, it can’t see your internal documents, and it definitely doesn’t know why your finance team suddenly renamed twelve folders last Thursday.

Which is why modern agents now depend on retrieving information dynamically instead of relying purely on what the model already knows.

Retrieved knowledge

Retrieved knowledge is where agents start becoming genuinely useful instead of just broadly knowledgeable. Unlike training knowledge, which is frozen at training time, retrieved knowledge happens in real time. The agent pulls relevant information from documents, databases, APIs, internal systems, or the web, depending on what the situation requires.

An HR agent is only useful if it can retrieve actual company policies instead of confidently inventing them. A sales agent needs live CRM context, not generic training data. The intelligence of the system comes less from what the model already “knows” and more from how well it can retrieve and use external context while operating.

Once you break agents down into these moving parts, the current ecosystem starts feeling less like a race toward “AI autonomy” and more like an ongoing experiment in coordinating increasingly complicated systems.

This article was adapted from Building Agents with OpenAI Agents SDK by Henry Habib, a practical guide to designing and building real-world agentic systems using the OpenAI Agents SDK.

The book goes much deeper into:

• tools and MCP integrations,

• memory and RAG architectures,

• multi-agent orchestration and handoffs,

• tracing and guardrails,

• production considerations,

• and hands-on projects that evolve from simple workflows into full agentic systems

If you’re trying to move beyond experimenting with agents and toward actually shipping them, it’s a genuinely useful resource.

Get the book

Over the past few months, we’ve had a growing suspicion that the AI conversation was changing underneath everyone’s feet. Your responses to last week’s survey confirmed it.

You told us what’s missing. What feels overhyped. What kinds of conversations actually matter. And across hundreds of responses, one thing became clear: you want more grounded coverage of AI agents and the systems emerging around them.

Let’s be real for a minute. “Agentic AI” is suddenly everywhere, but genuinely thoughtful coverage of the space feels strangely fragmented. There still isn’t a single place that consistently treats agentic systems as an engineering and operational transformation rather than just another trend cycle.

So with that, we’re announcing the rebrand of AI_Distilled to Agentic Engineering.

The AI conversation is maturing

This isn’t just a name change. It reflects where the AI landscape itself is heading and where many of you told us you already are. The broader “AI” lens made sense over the past few years while the ecosystem was still orienting itself. But the conversation is maturing now. The deeper builder audience emerging around AI infrastructure, orchestration, evaluation, memory, tooling, and autonomous workflows needs something more focused, more practical, and far less interested in demo culture.

We believe agents are becoming the next layer of software abstraction, and we want this publication to evolve alongside that reality.

What Agentic Engineering will focus on

With Agentic Engineering, we’ll still make the space accessible to curious newcomers, but we’ll also go much deeper into the real engineering trade-offs, debates, architectures, failures, workflows, and systems shaping this next phase of AI.

And importantly, we’re taking everything we learned from building AI_Distilled into this next chapter, including what not to do.

What happens next

So over the next few days, you’ll start seeing things change around here. And the next time this publication lands in your inbox, it won’t be from AI_Distilled.

It’ll be from Agentic Engineering.