A Journey from AI to LLMs and MCP - 4 - What Are AI Agents — And Why They're the Future of LLM Applications
Free Resources
- Free Apache Iceberg Course
- Free Copy of “Apache Iceberg: The Definitive Guide”
- 2025 Apache Iceberg Architecture Guide
- How to Join the Iceberg Community
- Iceberg Lakehouse Engineering Video Playlist
- Ultimate Apache Iceberg Resource Guide
We’ve explored how Large Language Models (LLMs) work, and how we can improve their performance with fine-tuning, prompt engineering, and retrieval-augmented generation (RAG). These enhancements are powerful—but they’re still fundamentally stateless and reactive.
To build systems that act with purpose, adapt over time, and accomplish multi-step goals, we need something more.
That “something” is the AI Agent.
In this post, we’ll explore:
- What AI agents are
- How they differ from LLMs
- What components make up an agent
- Real-world examples of agent use
- Why agents are a crucial next step for AI
What Is an AI Agent?
At a high level, an AI agent is an autonomous or semi-autonomous system built around an LLM, capable of:
- Observing its environment (inputs, tools, data)
- Reasoning or planning
- Taking actions
- Learning or adapting over time
LLMs generate responses, but agents make decisions. They don’t just answer; they think, decide, and act.
Think of the difference between a calculator and a virtual assistant. One gives answers. The other gets things done.
The Core Ingredients of an AI Agent
Let’s break down what typically makes up an agentic system:
1. LLM Core
The brain of the operation. Handles natural language understanding and generation.
2. Tools / Actions
Agents can execute external commands, like calling APIs, querying databases, or running code.
3. Memory
Persistent memory lets agents recall previous interactions, facts, or task states.
4. Planner / Executor Logic
This is where agents shine. They can:
- Break down complex goals into subtasks
- Decide which tools or steps to take
- Loop, retry, or adapt based on results
5. Context Manager
Decides what information (memory, documents, tool results) gets included in each LLM prompt.
LLM vs AI Agent — Key Differences
| Capability | LLM | AI Agent |
|---|---|---|
| Input | Prompt | Prompt + tools + state |
| Memory | Ephemeral (context) | Persistent (via external memory) |
| Reasoning | Single-shot | Multi-step planning |
| Action-taking | No | Yes (tools, APIs, workflows) |
| Autonomy | None | Optional (user- or goal-directed) |
| Adaptability | Static behavior | Dynamic, can learn from feedback |
LLMs are the engine. Agents are the vehicle.
Examples of AI Agents in the Wild
Let’s explore how AI agents are already showing up in real-world applications:
1. Developer Copilots
Tools like GitHub Copilot or Cursor act as coding assistants, not just autocomplete engines. They:
- Read your project files
- Ask clarifying questions
- Suggest multi-line changes
- Run code against test cases
2. Document Q&A Assistants
Instead of just answering questions, agents:
- Search relevant documents
- Summarize findings
- Ask follow-up questions
- Offer next actions (e.g., generate reports)
3. Research Agents
Given a broad prompt like “summarize recent news on AI regulation,” agents:
- Plan a research strategy
- Browse the web or internal data
- Synthesize and refine results
- Ask for confirmation before continuing
🔄 Agents Enable Autonomy and Feedback Loops
Unlike plain LLMs, agents can:
- Use tools to gather more info
- Loop on tasks until a condition is met
- Store and recall what they’ve seen
- Chain multiple steps together
For example:
Task: Schedule a meeting with Alice
Agent:
-
Search calendar availability
-
Find Alice’s preferred times
-
Draft an email proposal
-
Wait for response
-
Reschedule if needed
That’s not a single LLM prompt—that’s an intelligent system managing an evolving task.
How Are Agents Built Today?
A number of popular AI agent frameworks have emerged:
- LangChain: Modular orchestration of LLMs, tools, and memory
- AutoGPT: Autonomous task completion with iterative planning
- Semantic Kernel: Microsoft’s framework for embedding LLMs into software
- CrewAI / MetaGPT: Multi-agent systems with defined roles
These frameworks let developers prototype powerful workflows, but they come with challenges—especially around complexity, tool integration, and portability.
We’ll explore those challenges in the next post.
Limitations of Today’s Agent Implementations
While agents are promising, current frameworks have some limitations:
- Tight coupling to specific models or tools
- Difficult interoperability between agent components
- Context juggling: hard to manage what the model sees
- Security and control: risk of unsafe tool access
- Hard to debug: agents can go rogue or get stuck in loops
To address these, we need standardization—a modular way to plug in data, tools, and models securely and flexibly.
That’s where the Model Context Protocol (MCP) enters the picture.
Coming Up Next: AI Agent Frameworks — Benefits and Limitations
In our next post, we’ll explore:
- How modern agent frameworks work
- What they enable (and where they fall short)
- The missing layer that MCP provides