How I Designed a Reliable LLM Coding Agent for Production

From Unpredictable AI to Reliable Systems
Lessons from building real-world AI agent systems

📌 TL;DR

• Prompt engineering alone was never enough for stable execution
• Single-prompt systems created over-exploration, scope drift, and unreliable outputs
• I redesigned the system into Planner → Implementer → Validator
• Reliability came from contracts, validation, and retry loops—not better prompts
• Production reliability is a system design problem, not a model quality problem

1. The Problem Was Never Just the Prompt

When I first started building a local LLM coding agent, I believed the solution was simple:

Write a better prompt.

I was using:

• Mac Studio
• Ollama
• Claude Code CLI
• local Qwen models

The goal was straightforward:

Build an automated coding workflow that could operate without constant human intervention.

At first, I injected large prompts directly into the system and expected stable execution.

What happened instead was instability.

Sometimes the model explored too much and redesigned architecture instead of fixing the requested issue.
Sometimes it modified unrelated files and crossed boundaries I never intended to touch.
Sometimes it simply failed to return usable output.

The issue was not intelligence.

It was execution control.

That was the moment I stopped trying to optimize prompts and started redesigning the execution architecture.

2. The New Architecture — Planner → Implementer → Validator

The first version of the system looked like this:

Large Prompt
↓
LLM
↓
Hope for the best

This worked for demos, but it failed in production because the same request could produce different outcomes.

I redesigned the system around one principle:

Reliability must be enforced by the system.

The new structure became:

Request
↓
Planner
↓
Implementer
↓
Validator
↓
Retry if needed
↓
Converged Result

Each stage had:

• clear responsibility boundaries
• structured contracts
• validation checkpoints
• deterministic retry conditions

This was the point where the system became trustworthy.

The model was no longer asked to “figure everything out.”
It was asked to operate inside a controlled execution environment.

3. Layer 1 — Planner

The Planner exists to prevent intent drift.

Its job is not writing code.

Its job is defining the contract before execution begins.

Instead of vague instructions like:

Fix the login issue

the planner produces something like this:

Intent Contract

- Request Type
- Required Change
- Protected Boundaries
- Acceptance Criteria
- Codebase Anchors

For example:

Do:
- Add token refresh logic

Do Not:
- Change authentication API contracts

Must Pass:
- Existing login flow remains unchanged

This removes ambiguity before implementation starts.

Design decisions, protected boundaries, and success conditions are fixed in advance.

The Implementer is not asked to interpret intent.
It is asked to follow a contract.

That separation is what prevents scope drift.

4. Layer 2 — Implementer

The Implementer performs the actual code changes.

Its responsibility is intentionally narrow:

Execute the contract exactly.

The Implementer should not make new decisions during execution.

Its role is to apply the already defined contract as precisely as possible.
Design decisions, scope boundaries, and acceptance criteria should already be fixed by the Planner.

This significantly reduced the “creative failures” I saw in earlier versions.

Unexpected refactoring decreased, unnecessary architectural changes disappeared, and unrelated file modifications became far less common.

As a result, the implementation layer became intentionally predictable.

That predictability was a positive signal, because reliable production systems should reduce surprise rather than create it.

The goal of execution is not creativity.

It is consistency.

5. Layer 3 — Validator

This is the most important part of the system.

The Validator determines whether the result is actually acceptable.

It checks:

• Was the requested behavior implemented?
• Were protected contracts preserved?
• Did the output format remain valid?
• Were unintended side effects introduced?

Without validation, failure stays hidden.

The system may look successful while silently breaking important assumptions.

With validation, failure becomes explicit.

And once failure becomes visible, retries become meaningful.

This is where convergence actually happens—not inside the prompt, but inside the feedback loop.

A validator should reject not only syntactically broken output, but also structurally valid output that is semantically wrong.

That distinction is where many production systems fail.

6. Retry Logic — Convergence Over Perfection

Retries should not be random.

They should be guided by validation results.

Instead of saying:

Try again

the system should say:

Retry because Acceptance Criteria #2 failed
Retry because a protected contract was violated

This changes retries from guesswork into controlled convergence.

The goal is not to get the perfect answer on the first attempt.

The goal is to move the system consistently toward the target state.

That is what reliability means in practice.

Production systems do not need brilliance.

They need stable convergence.

7. Before vs After

Before:

Prompt
→ Result
→ Hope

After:

Intent
→ Contract
→ Execute
→ Validate
→ Retry
→ Reliable Output

The system became slower.

But it also became trustworthy.

And in production systems, trust matters more than speed—especially when autonomous execution is involved.

Fast failure is still failure.

Reliable execution is what matters.

8. What Actually Changed

The biggest change was not technical. It was philosophical.

At first, I focused on improving the model itself.
I kept asking how to make the model smarter, more accurate, and more capable.

Over time, I realized that intelligence was not the main problem.

The real problem was failure visibility.

The better question became:

How do I design a system where failure cannot be ignored?

That shift changed everything.

Reliable AI systems are not built by better prompts alone.

They are built through clear execution boundaries, explicit contracts, strong validation layers, and retry mechanisms that guide the system toward convergence.

In other words, reliability does not come from model intelligence itself. It comes from the system surrounding the model.

That is why:

Reliability is not a model capability.
It is a system property.