Why I Stopped Using One Agent and Built a Multi-Agent Pipeline Instead

Why I Stopped Using One Agent and Built a Multi-Agent Pipeline Instead

When I first started experimenting with AI agents, I was captivated by the simplicity of a single, monolithic agent. It felt like magic: one prompt, one API call, one answer. But as I pushed into real-world tasks—research synthesis, complex data processing, and multi-step reasoning—the magic faded. The single agent became a bottleneck: context windows overflowed, reasoning loops stalled, and outputs lacked the depth I needed. That’s when I stopped relying on one agent and built a multi-agent pipeline instead. In this article, I’ll walk you through why I made the switch, how to set up your own multi-agent system, and the practical improvements it delivers.

The Problem with a Single Agent

A single agent—whether powered by GPT-4, Claude, or any large language model—has inherent limitations. The most obvious is context length. Even with models supporting 128K or 200K tokens, real-world tasks like analyzing a research paper or generating a business report quickly consume that space. The agent forgets early instructions, loses track of intermediate results, and produces shallow outputs.

Another issue is task specialization. One model can’t excel at everything. A single agent might be great at creative writing but terrible at structured data extraction. When I asked one agent to both summarize a technical document and extract tables, it either ignored the tables or produced inaccurate summaries. I needed different agents for different jobs.

Finally, debugging is a nightmare. When a single agent fails, you don’t know why. Was it the prompt? The model? The data? A multi-agent pipeline isolates failures to specific steps, making it easier to fix and optimize.

What Changed My Mind

I stumbled on a pattern while reading about agent architectures on platforms like Towards Data Science. The idea was simple: decompose a complex task into subtasks, assign each to a specialized agent, and orchestrate them with a lightweight coordinator. This approach, sometimes called a "pipeline" or "orchestrator" pattern, mirrors how humans solve problems—break it down, delegate, and combine results.

I also noticed that companies like OpenAI and Microsoft were moving in this direction. OpenAI’s function calling and assistants API enable multi-step workflows. Microsoft’s AI Blog discusses multi-agent systems for enterprise automation. Anthropic’s research on constitutional AI suggests that specialized agents can better adhere to complex rules. These developments confirmed that multi-agent pipelines aren’t just academic—they’re the future of practical AI.

Requirements for Building a Multi-Agent Pipeline

Before we dive into code, let’s list what you need:

• **Python 3.9+**: The backbone of our pipeline.
• **OpenAI API key** (or equivalent): For accessing GPT models. You can sign up at [platform.openai.com](https://platform.openai.com).
• **LangChain** (optional but helpful): A framework for chaining LLM calls.
• **A task**: Pick something non-trivial, like summarizing a set of articles, extracting key metrics, and generating a report.

For this tutorial, I’ll use OpenAI’s API with Python directly to keep dependencies minimal. If you prefer a framework, LangChain works similarly.

Step-by-Step Installation

Let’s set up the environment. I’ll assume you’re on macOS or Linux; Windows users should adjust paths accordingly.

1. Create a virtual environment

Isolating dependencies prevents conflicts.

python3 -m venv multi_agent_env
source multi_agent_env/bin/activate

2. Install required packages

We need the OpenAI Python client and a few utilities.

pip install openai python-dotenv requests

• `openai`: The official Python client for OpenAI’s API.
• `python-dotenv`: Loads API keys from a `.env` file.
• `requests`: For any external API calls (optional).

3. Set up your API key

Create a `.env` file in your project root.

echo "OPENAI_API_KEY=your-key-here" > .env

Replace `your-key-here` with your actual API key from [platform.openai.com](https://platform.openai.com).

4. Verify installation

Run a quick test to ensure everything works.

# test_setup.py
import os
from dotenv import load_dotenv
from openai import OpenAI

load_dotenv()
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

response = client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "Say hello in one word."}]
)
print(response.choices[0].message.content)

Execute it:

python test_setup.py

If you see "Hello" (or similar), you’re ready.

Designing the Multi-Agent Pipeline

My pipeline has three agents:

1. **Research Agent**: Gathers and summarizes information from provided texts. 2. **Data Extraction Agent**: Extracts structured data (e.g., numbers, dates, names). 3. **Report Generator Agent**: Combines summaries and data into a final report.

I use a simple Python script as the orchestrator. Each agent is a function that calls the OpenAI API with a specialized system prompt.

The Orchestrator Script

Create a file named `pipeline.py`. Here’s the structure:

import os
from dotenv import load_dotenv
from openai import OpenAI

load_dotenv()
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

def research_agent(text):
    """Summarizes the input text."""
    response = client.chat.completions.create(
        model="gpt-4",
        messages=[
            {"role": "system", "content": "You are a research assistant. Summarize the following text concisely."},
            {"role": "user", "content": text}
        ]
    )
    return response.choices[0].message.content

def data_extraction_agent(text):
    """Extracts structured data (dates, names, numbers) from text."""
    response = client.chat.completions.create(
        model="gpt-4",
        messages=[
            {"role": "system", "content": "Extract all dates, names, and numbers from the text. Output as JSON."},
            {"role": "user", "content": text}
        ]
    )
    return response.choices[0].message.content

def report_generator_agent(summary, data):
    """Generates a final report from summary and extracted data."""
    response = client.chat.completions.create(
        model="gpt-4",
        messages=[
            {"role": "system", "content": "You are a report writer. Combine the summary and data into a professional report."},
            {"role": "user", "content": f"Summary: {summary}\nData: {data}"}
        ]
    )
    return response.choices[0].message.content

def run_pipeline(input_text):
    """Orchestrates the multi-agent pipeline."""
    print("[Research Agent] Summarizing...")
    summary = research_agent(input_text)
    print(f"Summary: {summary[:100]}...")
    
    print("[Data Extraction Agent] Extracting data...")
    data = data_extraction_agent(input_text)
    print(f"Data: {data[:100]}...")
    
    print("[Report Generator Agent] Writing report...")
    report = report_generator_agent(summary, data)
    print(f"Report: {report[:200]}...")
    
    return report

if __name__ == "__main__":
    sample_text = """
    On March 15, 2024, Dr. Alice Johnson presented her findings on AI ethics at the Stanford Conference. 
    She noted that 87% of surveyed companies have implemented AI guidelines. 
    The event attracted 1,200 attendees from 45 countries.
    """
    final_report = run_pipeline(sample_text)
    print("\n=== Final Report ===")
    print(final_report)

Run it:

python pipeline.py

You’ll see each agent’s output printed step by step.

Usage Examples

Let’s use the pipeline on a real-world scenario: summarizing a set of news articles about AI from reliable sources. I’ll simulate this with a few paragraphs.

Example 1: Summarizing a technical article

Suppose you have a long text from the Microsoft AI Blog about multi-agent systems.

long_text = """
Microsoft researchers recently demonstrated a multi-agent framework for automating software development. 
The system uses specialized agents for code generation, testing, and documentation. 
Early results show a 40% reduction in development time for standard features. 
The framework is open-source and available on GitHub.
"""
report = run_pipeline(long_text)
print(report)

The pipeline will produce a concise summary, extract the 40% statistic and the GitHub reference, and generate a report.

Example 2: Analyzing multiple sources

You can extend the pipeline to handle multiple documents by looping the research agent.

documents = [
    "OpenAI announced GPT-4 Turbo with a 128K context window on November 6, 2023.",
    "Anthropic's Claude 3 Opus achieves state-of-the-art performance on reasoning tasks.",
    "Microsoft's AI Blog highlights the importance of responsible AI deployment."
]

all_summaries = []
for doc in documents:
    summary = research_agent(doc)
    all_summaries.append(summary)

combined = "\n".join(all_summaries)
final_report = report_generator_agent(combined, "No structured data extracted.")
print(final_report)

This pattern scales to hundreds of documents by adding parallel processing with `concurrent.futures`.

Example 3: Adding error handling

Real pipelines need resilience. Here’s a quick addition:

def safe_call(agent_func, input_text, retries=2):
    for attempt in range(retries):
        try:
            return agent_func(input_text)
        except Exception as e:
            print(f"Attempt {attempt+1} failed: {e}")
    return "Error: Agent failed."

Use `safe_call` instead of direct calls in `run_pipeline`.

Why This Works Better

The multi-agent pipeline solves the problems I had with a single agent:

• **Context management**: Each agent only sees its subtask, reducing context waste.
• **Specialization**: The research agent is optimized for summarization; the extraction agent for JSON output.
• **Debuggability**: If the report is bad, I know it’s the report generator, not the extraction.
• **Scalability**: I can add agents (e.g., a fact-checker, a translator) without rewriting everything.

I also save money. The research agent uses a cheaper model (GPT-3.5) for initial summarization, while the report generator uses GPT-4 for quality. In a single-agent setup, you’d pay GPT-4 rates for the entire task.

Conclusion

I stopped using one agent and built a multi-agent pipeline because it’s more reliable, more maintainable, and more cost-effective. The pipeline pattern—decompose, delegate, orchestrate—transformed my workflow from fragile to robust. With just a few dozen lines of Python and the OpenAI API, you can build your own system today. Start with a simple two-agent pipeline, then expand as your tasks grow. The future of AI isn’t one super-agent; it’s a team of specialists working together.