Building AI Agents: From Concept to Production

Most people think of AI agents as glorified chatbots - you ask a question, the model generates an answer, and that is the end of it. But a real AI agent does not just answer - it acts. It perceives its environment, makes decisions, executes multi-step plans, and self-corrects when something goes wrong.

In my work building agentic systems at a global ratings agency and across my own products like xThreads.app and PostQueue.app, I have learned that the gap between a proof-of-concept agent and a production system is enormous. This post walks through that gap - from first principles to deployable architecture.

What you will learn: The anatomy of an AI agent · Planning, reasoning, and execution loops · Tool integration and structured APIs · Memory systems (short-term, long-term, semantic) · Production challenges: cost, safety, observability · A real-world case study from financial services

1. What Makes a System an Agent?

A chatbot responds to a prompt. An agent pursues a goal. That goal may be as simple as "find the latest news about AI" or as complex as "process this trade confirmation and update all downstream systems." The difference is not the model - it is the architecture around the model.

An AI agent has four defining characteristics:

• Autonomy: It operates without constant human intervention. You give it a goal, it works towards it.
• Reasoning: It decomposes complex goals into actionable steps, often using chain-of-thought or ReAct patterns.
• Tool use: It calls APIs, queries databases, runs code - it does not just talk, it acts.
• Memory: It remembers what it has done and what it has learned, across sessions and tasks.

2. The Core Loop: Reason → Act → Observe

Every AI agent, from the simplest to the most complex, runs the same fundamental loop. It is called the ReAct pattern - Reason and Act in alternating steps:

1. Reason: The LLM looks at the current state and decides what to do next.
2. Act: The agent executes a tool call - an API request, a database query, a file write.
3. Observe: The result comes back, and the agent incorporates it into its reasoning.
4. Repeat until the goal is achieved or the agent decides it cannot proceed.

This loop is deceptively simple. The magic is not in any single step - it is in theiteration. Each cycle gives the agent more information, letting it make better decisions on the next pass.

Minimal Implementation

Here is the ReAct loop in its simplest form:

from dataclasses import dataclass
from typing import Any

@dataclass
class AgentState:
    goal: str
    history: list[dict]  # reasoning steps + tool results
    done: bool = False

def react_loop(state: AgentState, tools: dict[str, Any], llm):
    """Run one cycle of the ReAct loop. Call in a while loop."""
    prompt = build_prompt(state)
    decision = llm.generate(prompt)  # "search('AI news')" or "done"

    if decision == "done":
        state.done = True
        return

    tool_name, args = parse_decision(decision)
    result = tools[tool_name](args)
    state.history.append({"step": decision, "result": result})

def build_prompt(state: AgentState) -> str:
    return f"""Goal: {state.goal}
Previous steps: {state.history}
What should I do next?"""

This is a real, working loop. Each iteration, the LLM decides what to do. The result is fed back in. The loop continues until the LLM decides the goal is complete.

3. Tool Integration: Giving Agents Capabilities

An agent without tools is just a chatbot with extra steps. The power of an agent comes from the tools it can use. In production systems, tools are structured interfaces - typed, documented, and sandboxed.

Structured Tool Definitions

from pydantic import BaseModel
from typing import Callable

class Tool(BaseModel):
    name: str                        # "search_web"
    description: str                 # Human-readable - LLM reads this
    parameters: dict                 # JSON Schema for the LLM
    function: Callable               # The actual Python function

    def to_openai_format(self) -> dict:
        """Convert to OpenAI function-calling format."""
        return {
            "type": "function",
            "function": {
                "name": self.name,
                "description": self.description,
                "parameters": self.parameters,
            }
        }

The description is critical. The LLM reads this field to decidewhich tool to use. A vague description means the LLM will guess. A precise description means the LLM will reliably pick the right tool.

Multi-Tool Orchestration

Production agents typically have 5–20 tools. Here is how a financial analysis agent might use multiple tools in sequence:

tools = {
    "search_sec_filings": Tool(
        name="search_sec_filings",
        description="Search SEC EDGAR for company filings by ticker",
        parameters={"ticker": {"type": "string"}},
        function=search_edgar,
    ),
    "extract_financials": Tool(
        name="extract_financials",
        description="Extract key financial metrics from a filing",
        parameters={"filing_id": {"type": "string"}},
        function=parse_financials,
    ),
    "calculate_ratio": Tool(
        name="calculate_ratio",
        description="Calculate a financial ratio from extracted data",
        parameters={"metric_a": {"type": "string"}, "metric_b": {"type": "string"}},
        function=compute_ratio,
    ),
    "generate_report": Tool(
        name="generate_report",
        description="Write analysis results to a structured report",
        parameters={"content": {"type": "string"}},
        function=write_report,
    ),
}

# The agent would call: search → extract → calculate → report

4. Memory: Short-Term, Long-Term, and Semantic

A critical limitation of LLMs is their context window. Without memory, an agent forgets everything between calls. Production systems use a layered memory architecture:

Short-Term Memory

The context window itself. We manage it by summarizing older turns:

class ShortTermMemory:
    def __init__(self, max_tokens: int = 8000):
        self.turns: list[dict] = []
        self.max_tokens = max_tokens

    def add(self, turn: dict):
        self.turns.append(turn)
        self._prune_if_needed()

    def _prune_if_needed(self):
        total = sum(len(str(t)) for t in self.turns)
        if total > self.max_tokens:
            summary = self._summarize_old_turns()
            self.turns = [{"summary": summary}, *self.turns[-3:]]

    def _summarize_old_turns(self) -> str:
        """Use an LLM to compress old turns into a summary."""
        old = self.turns[:-3]
        prompt = f"Summarize these steps concisely: {old}"
        return llm.generate(prompt)

Long-Term Memory

For persisting facts across sessions, use a vector database:

import chromadb

class LongTermMemory:
    def __init__(self):
        self.client = chromadb.Client()
        self.collection = self.client.create_collection("agent_memory")

    def remember(self, fact: str, metadata: dict = None):
        """Store a fact for future retrieval."""
        self.collection.add(
            documents=[fact],
            metadatas=[metadata or {}],
            ids=[str(uuid.uuid4())],
        )

    def recall(self, query: str, n: int = 3) -> list[str]:
        """Find relevant past facts."""
        results = self.collection.query(
            query_texts=[query],
            n_results=n,
        )
        return results["documents"][0]

5. Production Challenges: Cost, Safety, Observability

Building an agent that works on your laptop is easy. Building one that works reliably in production for real users is hard. Three challenges dominate:

Cost Control

Every LLM call costs money. An unbounded agent loop can burn through dollars in seconds:

class BudgetTracker:
    def __init__(self, max_cost_usd: float = 1.00):
        self.spent = 0.0
        self.max_cost = max_cost_usd

    def check(self, prompt_tokens, completion_tokens):
        cost = (prompt_tokens * 0.00001) + (completion_tokens * 0.00003)
        self.spent += cost
        if self.spent > self.max_cost:
            msg = f"Budget exceeded. Spent {self.spent:.2f}, limit {self.max_cost:.2f}"
            raise BudgetExceededError(msg)
        return True

    def estimate(self, goal):
        return len(goal.split()) * 0.0001

Safety Guardrails

Autonomous agents can take actions you did not intend. Guardrails are rules the agent cannot override:

GUARDRAILS = [
    "Never delete or modify production data",
    "Never send money or execute trades without human approval",
    "Never share personally identifiable information (PII)",
    "Never execute untrusted code from user input",
    "Never make more than 3 API calls per minute",
]

def apply_guardrails(action: str, context: dict) -> bool:
    """Check an action against all guardrails. Return True if allowed."""
    for rail in GUARDRAILS:
        if matches_guardrail(action, rail):
            logger.warning(f"Blocked by guardrail: {rail}")
            return False
    return True

def human_approval(action: str) -> bool:
    """Pause execution and ask a human."""
    print(f"\n⚠️  Requires approval: {action}")
    response = input("Approve? (y/n): ")
    return response.lower() == "y"

Observability

You cannot debug what you cannot see. Every agent step should be logged:

import json
from datetime import datetime

class AgentLogger:
    def __init__(self, agent_id: str):
        self.agent_id = agent_id
        self.logs = []

    def log(self, event: str, details: dict):
        entry = {
            "timestamp": datetime.utcnow().isoformat(),
            "agent_id": self.agent_id,
            "event": event,
            "details": details,
        }
        self.logs.append(entry)
        print(json.dumps(entry))  # structured logging

    def trace(self) -> list[dict]:
        """Return the full trace for debugging."""
        return self.logs

6. Real-World Case Study: Financial Analysis Agent

At a global ratings agency, I built an agentic system that helps analysts process complex financial documentation. Here is how it works end-to-end:

The agent's workflow:

1. Plan a retrieval strategy: The agent examines the analyst's query and decides which document types to search first.
2. Multi-hop retrieval: It searches financial statements, then cross-references with regulatory filings, then pulls comparable peer data.
3. Synthesize with citations: Each claim in the output is linked to the specific document and page that supports it.
4. Self-correct: If the agent detects contradictory information, it flags the inconsistency and re-queries rather than generating a false resolution.

7. Summary: What to Remember