There has been a lot of productive dialogue recently about what agents are, and the best way to build them. It was arguably kicked off last December when Anthropic published Building Effective Agents, an almost universally well-received guide to the basic patterns underlying agentic software, as well as thoughts about when to apply them. Three weeks ago, Dexter Horthy went viral with his 12 Factor Agents, a thoughtful list of design principles to use when building agents. Then a few days ago OpenAI released A Practical Guide To Building Agents, which summarized their own thoughts on the same topic. All three are insightful reads.
We’ve been asked by open source users and enterprise customers for our own takes on these principles. We find ourselves happily in agreement with much of what’s been written, drawing on our own direct experience of building complex agentic systems like LlamaCloud as well as from working with our many enterprise customers and hundreds of thousands of open source users. We thought it would be useful to share what we’ve learned.
Workflows: the core of agentic systems in LlamaIndex
To understand how we think about designing agentic systems it’s worth first understanding Workflows, our system that exists in both our Python and TypeScript frameworks to orchestrate agentic execution (OpenAI’s guide coincidentally refers to agent execution paths as “workflows” throughout, so we guess it’s a pretty good name!).
Workflows are an event-based system that allows you to connect a series of execution steps implemented as vanilla functions, connected by emitting events that carry data and trigger execution of other steps. This system is fundamentally simple but powerful, allowing you to easily design chains, branches, loops, fan-outs and collections, including all of the design patterns mentioned by Anthropic in their post. (We recently released a tutorial showing how to implement each pattern.)
Workflows allow the implementation of agents as well as the use of sub-agents within each step. Decisions about control flow can be made by LLMs, via traditional imperative programming, or any combination of the two. Workflows have been well-received by our community and are increasingly fundamental to how software is written in LlamaIndex.
Blending autonomy and structure
The core insight behind the design of Workflows is one mentioned by all three of the posts: there exists a tension between a fully autonomous agent, which is given a goal and a set of tools and is allowed to figure out the path to the goal entirely on its own, and a more rigid structure such as a DAG in which every step is precisely defined. All three posts agree that successful agentic software lies somewhere between these two extremes.
This specific section of OpenAI’s Practical Guide was widely shared on social media:
This is why Workflows work the way they do: we experimented with more rigid, graph-based approaches and came away dissatisfied. Instead, Workflows provide optimized, imperative paths when appropriate while leaving LLMs free to work their magic. We see six key benefits to the hybrid pattern embodied by Workflows:
- Better reliability through predictable execution paths for critical operations
- Clearer error handling and recovery mechanisms
- Easier debugging and tracing of agent behavior
- Improved performance through optimized pathways for known scenarios
- Maintaining autonomy where it provides value
- First-class support for human oversight via human-in-the-loop
Optimizing agent design
Of course, declaring that the best pattern for an agent is a hybrid is easy if you don’t have to define exactly when structure makes sense versus when to opt for autonomy. It remains a tricky question, but we have some basic guidelines and examples to help:
When to define structure
When to provide autonomy
Agentic patterns are the future
We agree with all three authors that a pragmatic approach to agent design is the best one: use structure where it helps, provide autonomy where it shines. We think Workflows are the best option in the current marketplace of frameworks to easily implement such hybrid systems, and we encourage you to try them today! Of course, we also offer LlamaCloud, our enterprise platform with pre-built agentic solutions to common enterprise use-cases, complete with engineering assistance to get your system off the ground.
Some resources to help you get started:
- Python resources:
- TypeScript resources:
- LlamaCloud documentation