I Built an AI Coding Agent in Rust — Here's What Actually Happens Under the Hood

Most AI coding tools look simple from the outside.

You type something like:

"Find the authentication bug and fix it."

A few seconds later, the tool reads files, searches the codebase, edits code, runs commands, and sometimes even tests the result.

But underneath that experience, there is much more happening than a chatbot generating text.

I built Codey, a terminal-based AI coding agent written in Rust, to understand what actually needs to exist behind that workflow.

The biggest thing I learned is this:

An AI coding agent is not just an LLM with access to a shell.

It is a system that has to manage context, state, tools, models, and extensibility while allowing the agent to repeatedly observe what happened after each action.

The Five Core Components

I designed Codey around five main components:

• Sessions
• Context
• Model providers
• Agents
• Tools

Each one solves a different problem.

Sessions

A coding task is rarely a single request.

The agent might read a file, run a command, discover an error, inspect another file, and then retry with more information.

The session keeps that interaction together.

It represents the state of the ongoing task and gives the agent somewhere to store the conversation and the actions that have already happened.

Context

Context is probably one of the hardest problems in AI coding agents.

A real codebase can contain thousands of files. Sending the entire repository to the model would be slow, expensive, and mostly useless.

Instead, the agent needs to decide what matters for the current task.

That can include:

• Files relevant to the user's request
• Previous tool results
• Errors from commands
• Recent edits
• Project structure
• Conversation history

The goal is not to give the model more context.

The goal is to give it the right context.

Model Providers

Codey separates the agent from the underlying model provider.

That means the core agent does not need to care whether a model comes from one provider or another.

The provider layer handles communication with the model, while the rest of the agent focuses on planning and execution.

This separation makes experimentation easier and prevents the application from becoming tightly coupled to one API.

Agents

The agent coordinates the workflow.

At a high level, the loop looks like this:

User Request
      ↓
Understand Context
      ↓
Ask Model What To Do
      ↓
Call Tool
      ↓
Observe Result
      ↓
Update Context
      ↓
Continue or Finish

The important part is the observation step.

A tool call can fail.

A command can return an error.

A file can contain something the agent did not expect.

The agent needs to use that new information instead of blindly continuing with its original plan.

Tools

Tools are what turn a language model into something that can actually interact with a development environment.

Instead of only generating:

"You should inspect package.json."

The agent can actually read it.

Instead of suggesting:

"Try running the tests."

It can execute the test command and inspect the result.

Codey's architecture is built around tools that allow the agent to interact with files, commands, and development workflows.

Skills, MCPs, and Plugins

Another major focus of Codey is extensibility.

AI agents are becoming more capable, but no single agent can contain every integration or workflow internally.

That is where Skills, MCPs, and plugins become useful.

Skills

Skills give an agent reusable instructions and workflows.

For example, a skill could teach the agent how to review a pull request, debug a Rust project, or follow a particular deployment process.

The idea is that specialized knowledge can be loaded when it is relevant instead of permanently adding everything to the agent's context.

MCP

The Model Context Protocol, or MCP, provides a standard way for AI applications to connect with external tools and data sources.

Instead of writing a completely custom integration for every external service, an agent can connect to an MCP server that exposes those capabilities through a standard interface.

Plugins

Plugins package capabilities into reusable units.

A plugin can combine workflows, tools, Skills, and external integrations, making it easier to extend an agent without changing the core architecture.

This was important for Codey because I wanted the core agent to remain relatively small while allowing new capabilities to be added over time.

Why Rust?

Rust was a natural choice for the project because the application involves several areas where explicit architecture matters:

• Long-running processes
• Concurrent tool execution
• Terminal interaction
• State management
• External APIs
• System-level commands

The type system also encourages clear boundaries between components.

That became useful as the project grew beyond a simple terminal chatbot.

The Real Challenge Isn't Calling the Model

Calling an LLM API is relatively easy.

The harder part is building everything around it.

You have to answer questions like:

• What context should the model receive?
• Which tools should be available?
• How should failed commands be handled?
• How should state be preserved?
• How do you add new capabilities?
• How do you prevent one component from depending too heavily on another?

That is what made building Codey interesting for me.

The model is important, but the architecture around the model is what determines whether an agent feels like a useful development tool or just another chat interface.

What's Next

Codey is still an experiment in how modular AI coding agents can be designed.

The direction I am interested in is not simply adding more tools. It is building a system where context, agents, Skills, MCPs, and plugins can work together without turning the core architecture into a collection of tightly coupled integrations.

That is the difference between adding AI features to a terminal application and actually building an extensible agent system.