What is an AI agent and how is it different from a chatbot?

An AI agent is software that uses a large language model to pursue a goal on its own. It picks the next step, calls a tool such as your CRM, calendar or inbox, reads the result and repeats until the task is done or a limit stops it. A chatbot replies to the message in front of it. An agent takes actions in your systems to finish a job.

People ask what is an AI agent for a practical reason. Nearly every software vendor now calls its product one, and the label decides what you pay and what can go wrong. A product that only answers questions should not be priced, or trusted, like one that can issue refunds in your order system.

On this page

• What is an AI agent in plain terms
• AI agent vs chatbot compared
• How an AI agent works under the hood
• How do AI agents reason and remember
• The line between workflows and agents
• Where AI agents help a small business today
• How reliable are AI agents in 2026
• Why do so many AI agent projects stall
• What an AI agent costs to run per task
• The risks of AI agents and how to control them
• Do you need an AI agent, a chatbot or a simple automation
• How to start your first AI agent pilot
• What savings should an AI agent pilot show

What is an AI agent in plain terms

An agent has three parts, according to OpenAI's practical guide to building agents. A model reasons and decides, tools let it read data or take action, and instructions set its job and its limits. OpenAI defines agents as "systems that independently accomplish tasks on your behalf." By that test, an app that uses a language model without letting it control the workflow, such as a basic chatbot, is not an agent.

For example, say a 12-person HVAC company in Ohio gets 40 service requests a week through its website form. A chatbot on the site can answer "do you service heat pumps?" An agent can read each new request, check the technician calendar and propose two open slots. It can then create the job in the field-service software, text the customer a confirmation and pass the 5 or 6 unusual requests to the office manager.

Both setups can run the same model. The tools, and the permission to use them, are what change.

The idea is older than ChatGPT. Russell and Norvig's textbook Artificial Intelligence: A Modern Approach describes an agent as anything that perceives its environment and acts upon it. What changed after 2022 is the quality of language models. They can now read messy input, like a customer email with three questions in it, and pick a sensible next action without a developer writing a rule for every case.

AI agent vs chatbot compared

The cleanest way to separate the terms is to ask who decides the next step and whether the software can change anything outside the chat window. The table compares the three kinds of tools business owners are usually choosing between.

Google Cloud's guide to AI agents adds a middle category, the AI assistant, which "can recommend actions but decision-making is done by the user." That is how most people use ChatGPT, Claude or Gemini in a browser: the tool drafts, and a person decides. The same model becomes an agent once a developer connects it to tools and lets it chain several actions without asking each time.

The difference between an agent and a chatbot sits in the setup around the model. A support bot that can only quote your help center is a chatbot. Give it the order system, a refund tool with a $50 cap and permission to close tickets, and it becomes an agent. New risks come with that access.

How an AI agent works under the hood

Every agent runs a loop. Anthropic's tool use documentation lays out the mechanics, and OpenAI and Google APIs work the same way in principle:

1. The application sends the model a goal, its instructions and a list of tools. Each tool has a name, a plain-language description and a JSON schema for its inputs, for example get_order(order_id).
2. The model replies with either a final answer or a structured tool call. In Anthropic's API that reply carries stop_reason: "tool_use" and the arguments the model chose.
3. Your code, not the model, runs the tool. The documentation states that for client tools, "your code runs the operation and sends back a tool_result."
4. The result goes back into the conversation, and the model decides the next step: call another tool, ask a person or finish.
5. The loop ends when the goal is met or a stop condition fires, such as a step limit, a spending cap or a required human approval.

Step 3 carries the most weight for a business owner. At no point does the model touch your database. It asks, and your software decides whether to carry out the request. That gives you a control point for permissions, logging and approvals on every single action.

How do AI agents reason and remember

The pattern of thinking and acting in turns comes from research. The ReAct paper by Shunyu Yao and colleagues, presented at ICLR 2023, set the template. Its models wrote a short reasoning step before each action, then read the result before choosing the next one. On two decision-making benchmarks, ALFWorld and WebShop, ReAct beat imitation and reinforcement learning methods by 34 and 10 percentage points of absolute success rate. Most agent frameworks still follow that reason, act and observe cycle.

Memory works in two layers. Short-term memory is the context window: everything in the current loop, including tool results, is resent to the model on each call. Long-term memory is a database or document store the agent can search, such as past tickets or a product catalog. Most deployed agents do not retrain themselves. They improve when a developer edits the instructions, adds examples or fixes a tool.

The line between workflows and agents

Anthropic's engineering team draws the most useful line in its guide Building effective agents. Workflows are "systems where LLMs and tools are orchestrated through predefined code paths." Agents are systems where the model decides its own process and which tools to call, and keeps control over how the task gets done. A lead-routing flow where the model only classifies each email, and fixed code does the rest, is a workflow. A system where the model decides which records to look up and what to do next is an agent.

The same guide advises "finding the simplest solution possible, and only increasing complexity when needed," and adds that this "might mean not building agentic systems at all." Its reasoning is cost and risk: agents trade latency and money for better results on open-ended tasks, and their autonomy brings "the potential for compounding errors."

Marketing has stretched the word well past that definition. In June 2025, Gartner estimated that only about 130 of the thousands of agentic AI vendors are real, and called the relabeling of chatbots and assistants "agent washing." Before you buy, ask a vendor two direct questions: which steps does the model choose, and which actions can it take without a person approving them. A product with fixed steps and no autonomous actions is a workflow tool, and should be priced like one.

Workflow tools now offer both styles. Platforms covered in the guide to n8n and how it compares with Zapier let you place a single AI step inside a fixed flow, or drop in an agent node that picks its own tools. Starting with the first and graduating to the second is the lower-risk route.

Where AI agents help a small business today

OpenAI's guide says agents fit tasks with complex decisions, rules that are hard to maintain, or heavy reliance on unstructured data such as emails, PDFs and call notes. In a 5 to 200 person business, agents earn their cost on tasks like these:

• Lead triage, where the agent reads each inbound inquiry, looks up the company, scores fit, updates the record in your CRM and drafts a reply for a salesperson to send.
• Order support on tickets like "where is my order": check the carrier, reissue the shipment or refund under a set limit, and escalate the rest with a summary.
• Invoice matching, pulling PDFs from an accounts payable inbox, matching them to purchase orders and flagging mismatches before anyone pays.
• Scheduling across a shared calendar, including moves and confirmations.
• Weekly reporting that gathers competitor prices or ad metrics from four or five sources onto one page.
• Shopping on a buyer's behalf, a retail shift covered in the guide to agentic commerce.

A pattern runs through the list. Each task comes up dozens of times a month, takes a person 5 to 30 minutes, touches two or more systems and has a clear test for done. Tasks that happen twice a month, or where a mistake costs thousands of dollars, are poor first candidates.

How reliable are AI agents in 2026

Reliability is improving fast, from a modest base. In TheAgentCompany benchmark, a research team led by Frank F. Xu and Graham Neubig built a simulated small software company with internal websites and data, then asked agents to do the kind of work its staff would do. In the latest version of the paper, revised in September 2025, the best agent completed 30 percent of tasks autonomously.

The trend line is steeper than that snapshot suggests. Researchers at METR measured how long a task, timed by skilled humans, an AI can finish with a 50 percent success rate. Their paper found a time horizon of about 50 minutes for frontier models in early 2025, such as Claude 3.7 Sonnet. That horizon "has been doubling approximately every seven months since 2019." A seven-month doubling means tasks four times as long come within reach every 14 months.

Why do so many AI agent projects stall

Compounding error explains why the gap between demo and production is large. Assume an agent gets each step right 95 percent of the time. A 10-step task then succeeds about 60 percent of the time (0.95 to the 10th power), and a 20-step task about 36 percent. Raise per-step accuracy to 98 percent and the 10-step task succeeds 82 percent of the time. Shorter tasks, better tools and a human check at the risky step move these numbers more than a bigger model does.

Gartner's June 2025 forecast puts the business risk in plain numbers: over 40 percent of agentic AI projects will be canceled by the end of 2027 "due to escalating costs, unclear business value or inadequate risk controls." The same release predicts that at least 15 percent of day-to-day work decisions will be made autonomously by agentic AI by 2028, up from 0 percent in 2024. Both can be true. The projects that survive pick narrow tasks and measure them.

What an AI agent costs to run per task

Model fees are billed per token, roughly three quarters of a word. As of October 2026, Anthropic's API pricing lists Claude Sonnet 5.5 at $2 per million input tokens and $10 per million output tokens. Claude Haiku 5.5 costs $0.10 and $0.50 for prompts up to 100,000 tokens. Batch processing halves both rates. OpenAI and Google price their models on the same per-token basis.

Each agent task costs more than a chatbot reply because it makes several model calls, and each call resends the growing history. The worked example below assumes a chatbot answer of one call, and a six-step support agent averaging 8,000 input and 500 output tokens per call.

On the same model, the agent costs about 13 times the chatbot per task. Moving routine steps to a smaller model cuts the bill by a factor of 20. And at 1,000 tasks a month, model fees are small next to the cost of building, testing and maintaining the integrations. The breakdown of what it costs to build an AI chatbot covers those build costs, and most of its line items apply to agents as well.

The hidden cost is review time. If a person spends 2 minutes checking each of 150 escalated tasks a month, that is 5 hours of staff time. At a loaded cost of $35 an hour, those 5 hours cost $175, more than the $126 token bill for 1,000 agent tasks on Sonnet 5.5. Budget for review from the first day.

The risks of AI agents and how to control them

The OWASP Top 10 for LLM applications names the main agent risk Excessive Agency: the weakness that "enables damaging actions to be performed" when a model produces an unexpected or manipulated output. OWASP traces it to three causes, too much functionality, too many permissions and too much autonomy, and each has a direct fix.

• Give the fewest tools. A reply-drafting agent does not need a delete-contact tool. Remove anything the task does not require.
• Scope every credential. Use read-only database access where possible, and a separate account for the agent so its actions show up in audit logs under its own name.
• Require approval for high-impact actions. OpenAI's guide lists actions that are "sensitive, irreversible, or have high stakes," such as canceling orders, authorizing large refunds or making payments, as triggers for human oversight.
• Set hard limits. Cap steps per task, spend per day and retries per tool, and escalate to a person once a limit is reached.
• Log everything. Keep each tool call, its inputs and its result, so you can answer what the agent did and why.
• Treat outside text as untrusted. Emails, web pages and uploaded files can carry hidden instructions, a technique called prompt injection. An agent reading your inbox should never hold more access than the task needs.

Data handling matters too. Check where the model provider stores prompts, whether your plan excludes your data from training, and which customer fields the agent can see. A support agent rarely needs full card numbers or dates of birth, so leave them out of what it can read.

Do you need an AI agent, a chatbot or a simple automation

Most small businesses should start one level below where the sales pitch points. The table maps common tasks to the lightest tool that does the job.

The recommendation is direct. Build the workflow with one AI step first and measure it for a month. Move to an agent only where exceptions eat most of the staff time. A fixed flow that handles 80 percent of cases cheaply and sends the other 20 percent to a person beats an agent that handles 95 percent and fails in ways nobody notices.

How to start your first AI agent pilot

A pilot should prove value on one task within about six weeks. These steps keep it small enough to measure:

1. Pick one task that repeats at least 100 times a month and takes 5 to 20 minutes each time.
2. Write down every step a person takes today and every system it touches, including the exceptions.
3. Measure the baseline: minutes per task, error rate and how many times it waits on someone.
4. Connect the agent with read-only access and have it draft its actions for a person to approve.
5. Set limits before launch: maximum steps, a daily spending cap and the actions that always need approval.
6. Run in this shadow mode on at least 100 real tasks and compare its drafts with what staff did on the same tasks.
7. Switch on autonomous actions one at a time, starting with the lowest-risk one.
8. Name an owner who reviews the logs weekly and adjusts instructions and tools.

What savings should an AI agent pilot show

Say a 20-person property management firm in Arizona handles 200 maintenance requests a month at 12 minutes each, or 40 hours of staff time.

If the agent fully handles 70 percent and a person spends 2 minutes reviewing each one, the firm saves about 23 hours a month. At a loaded staff cost of $35 an hour, that is roughly $800 a month, against a token bill under $30 on a mid-size model. Numbers like these justify the build cost, or show it is not worth it.

Start by listing the five tasks your team repeats most and timing each for a week. That list shows whether you need a chatbot, a workflow or an agent. The AI development team at Techno Mantra builds agents and automations on OpenAI, Claude or Gemini, using n8n, Zapier or custom code. Scoped permissions and approval steps ship in the first release, and you own all code and accounts. Send your task list through the contact page or to info@technomantraa.com, and every inquiry gets a reply within one business day.