How to Teach an AI Agent

Here's something most people get wrong about AI agents.

They think building them requires engineering expertise. That teaching an AI means writing complex prompts, understanding model architectures, or hiring a team of ML engineers.

I get why people think this. Most AI systems are built that way. You update a knowledge base in some admin panel, wait for changes to propagate, test through an eval system days later, and hope it works.

But that's not how it should work. Teaching an AI should feel like training a new employee, not configuring software.

Let me show you what I mean.

The Real Challenge: Conversations Are Chaotic

Before we talk about teaching, you need to understand why AI agents get stuck in the first place.

Conversations are inherently high-entropy. They're natural language, which means people can ask you anything about any topic and word it in countless different ways.

If you're running operations, you know this reality. Customers don't read your FAQ before contacting you. They don't phrase questions the way you'd expect. They mix multiple issues into one message. They assume context you don't have.

Your AI agent will inevitably encounter situations where it doesn't understand how to respond. The question isn't whether your AI will get stuck. The question is: what happens when it does?

This is where most systems fail.

How Traditional Systems Handle Teaching (Badly)

Most AI platforms force you into one of three approaches when your agent gets stuck:

Approach 1: Asynchronous Knowledge Base Updates

You notice the AI answered something wrong. You go to an admin panel. You update documentation. You wait for changes to propagate. You test later. You hope it worked.

The problem? No immediate validation. You're teaching blind. You won't know if your update fixed the issue until the same situation happens again, maybe days or weeks later.

Approach 2: Prompt Engineering

You hire technical experts to modify system prompts. They write complex instructions. They test carefully because changing prompts can break existing functionality. It's expensive, slow, and requires specialized knowledge.

The problem? It's not scalable. Every time you need to teach the AI something new, you need engineering resources. That's not sustainable for operations teams.

Approach 3: Fine-Tuning

You send conversation data to your AI provider. They fine-tune the model. You wait days for updates. You have no transparency into what the AI actually learned. You can't update on the fly.

The problem? It's a black box. You have no control, no visibility, and no ability to respond quickly to changing business needs.

All three approaches share the same fatal flaws: slow feedback loops, no immediate validation, steep technical requirements.

There's a better way.

The Employee Onboarding Analogy

Think about how you'd train a new support agent on your team.

You wouldn't just give them access to your Google Drive and say "figure it out." You wouldn't send them to a week-long training course before they could help their first customer.

You'd sit them next to an experienced agent. When they encounter a situation they don't know how to handle, they'd ask for help. You'd teach them right then, in the moment. They'd apply what you taught immediately. You'd see if they understood. If not, you'd clarify.

That's how teaching AI should work. In the moment. With immediate feedback. Using plain language.

How Real-Time Teaching Actually Works

Let me walk you through a real example.

A customer messages: "I haven't been charged for my booking yet. Is there a mistake? Are you going to charge me before I arrive?"

Your AI agent recognizes this is payment-related. It can see the customer's booking details and payment status. But it doesn't know your specific policy about when charges occur.

So it escalates to you with context: "This customer is asking about payment timing. I can see they have a balance due, but I don't know when we charge customers for this type of booking."

Here's where real-time teaching happens. Right in the inbox, without leaving the conversation, you type in plain language:

"add to KB that we charge right before check-in"

That's it. No prompt engineering. No understanding of AI architecture. Just type what you want the AI to know.

The system processes your natural language input and understands what type of teaching this is. It presents you with a confirmation:

Teaching: The charge time policy is we charge guests right before check-in.

Apply to: You can choose whether this applies globally to all customers, just to specific product tiers, or only to customers matching certain criteria.

This is crucial. In complex businesses, different rules apply to different customer segments. You might charge short-term rentals 48 hours before check-in, but charge long-term rentals on the 1st of the month. Partition-level control ensures knowledge applies exactly where it's needed.

You confirm. The AI immediately regenerates its response using the new knowledge.

You see the new response in real-time: "We'll charge your card 24 hours before your check-in date. You'll receive an email confirmation when the charge processes."

Perfect. You send it. Done.

Here's what just happened that's different from traditional systems:

1. You taught in the moment, without leaving the conversation

2. You used plain language, no technical expertise required

3. You saw immediate validation, the AI showed you it understood

4. You have confidence, because you watched it apply the teaching successfully

The next time a customer asks about payment timing, your AI handles it perfectly. No engineering required. No waiting for updates. No hoping it worked.

The Three Types of AI Knowledge

Here's something that will make you much better at teaching AI: understanding that AI processes different types of information differently.

Most people just dump information at their AI and wonder why it doesn't work consistently. The secret is being explicit about what type of knowledge you're adding.

Type 1: Knowledge (Reference Information)

This is factual information the AI can look up and reference when needed.

Examples:

• "The key fob code is 555"

• "Office hours are 9 AM to 5 PM Monday through Friday"

• "The warranty period is 90 days from purchase"

How to teach it: "Add to knowledge base: [fact]"

How AI processes it: Basic retrieval. The AI searches for relevant knowledge and includes it in responses. Computationally lightweight. Good for static facts.

Type 2: Procedures (Step-by-Step Logic)

This is conditional logic. If-then statements. Decision trees. These are workflows, not facts.

Example:

"When a customer requests early check-in:

1. Check the calendar first

2. If there's no checkout on that day, allow early check-in

3. If there is a checkout that day, decline due to cleaning and turnover time"

How to teach it: Structure as conditional logic with clear steps.

How AI processes it: Uses significantly more computation to ensure step-by-step reliability. The AI treats this as a workflow to execute carefully, not just information to reference.

Why the distinction matters: If you add a procedure as general knowledge, the AI might reference it but won't follow it systematically. If you teach it explicitly as a procedure, the AI knows to execute it step-by-step.

Type 3: Guardrails (Hard Boundaries)

These are absolute rules about what the AI should never do or say. Constraint layers, not information.

Examples:

• "Never promise refunds without manager approval"

• "Don't provide medical advice"

• "Never share customer data with third parties"

How to teach it: Use explicit negative language. "Never [action]" or "Don't [behavior]"

How AI processes it: Acts as a constraint layer the AI checks before generating any response. Think of guardrails as filters that prevent specific behaviors.

Teaching Best Practices

Now that you understand the three types, here's how to teach effectively:

Be Explicit About Knowledge Type

Don't make the AI guess. State it clearly.

✅ "Add to knowledge base: [fact]"

✅ "When [situation], follow these steps: [procedure]"

✅ "Never [action]"

Use Procedural Format for Workflows

When teaching multi-step processes, structure them clearly:

❌ Poor: "Handle early check-in requests carefully and check availability first"

✅ Effective: "When customer requests early check-in:

1. Check calendar for same-day checkout

2. If available, approve early check-in

3. If not available, decline and explain turnover time needed"

Leverage Partition-Level Control

Don't apply everything globally. Think about whether this knowledge should apply to all customers or just specific segments.

Example: A property management company might have different payment policies for short-term rentals, long-term rentals, and corporate housing. Teach each policy to the right partition.

Avoid Common Mistakes

❌ Being aggressive: "I hate you for responding this way"

Why it fails: AI doesn't respond to emotional language. Just state what you want clearly.

❌ Vague instructions: "Handle this better"

Why it fails: AI needs specific guidance. What does "better" mean?

✅ Specific alternative: "When handling billing questions, always check payment status first, then explain our payment schedule"

Why This Approach Changes Everything

Think about what real-time teaching enables:

For you as an operator: You manage AI agents like you manage human team members. When they make mistakes, you correct them immediately. When they encounter new situations, you teach them how to handle it. No waiting for engineers. No dependency on technical teams.

For your business: Faster time to value. You start with basics and refine as you encounter edge cases. Your institutional knowledge gets captured systematically as the AI encounters new situations. Your automation percentage increases continuously over time.

For your customers: Better experiences. The AI improves continuously based on real interactions. Responses are consistent because knowledge is applied systematically. Issues get resolved faster because operators teach the AI instead of answering the same question repeatedly.

The Flywheel Effect

Here's what happens over time with this approach:

Your AI handles more conversations successfully. Your operators have more time to focus on complex situations. Every complex situation that gets escalated is an opportunity to teach. The AI becomes more capable. The automation rate increases.

This creates a positive feedback loop. The system becomes more valuable and more automated over time, not static like traditional systems.

You Don't Need Engineers

Let me be direct about this: you don't need to be technical to build and maintain sophisticated conversational AI.

You need to know how to train a smart employee. That's it.

If you can explain a policy to a new hire, you can teach an AI agent. If you can write step-by-step instructions for a procedure, you can teach an AI workflow. If you can set boundaries for your team about what they should never do, you can set guardrails for AI.

The barrier to building conversational AI shouldn't be technical expertise. It should be operational expertise. Understanding your business, knowing your customers, and having clear processes.

That's the empowerment message here. You should be able to jump in and build this yourself. You should be able to figure it out, even if you hire someone later to scale it. You should understand it, not outsource it to a black box.

Teaching AI without being technical isn't just possible. It's how it should work.

About the Author

Punn Kam is the founder of Conduit (YC W24), a platform built specifically for production-ready conversation agents. After working at Google on cutting-edge AI systems, Punn has helped hundreds of operators implement conversational AI that drives measurable outcomes.