Chatbot metrics: 15 measures of real performance and return
A chatbot’s performance is not measured by its subscriber count, but by business outcomes: how many requests are resolved without an agent, whether customers get answers faster, and how many leads or sales come through the bot. Measurement needs four levels: reach, scenario funnel, answer quality, and business impact. This guide covers 15 metrics with formulas, role-specific dashboards, and common analytics mistakes.
Four levels of chatbot metrics
Build chatbot metrics as a pyramid: reach shows whether people use the bot, the funnel shows whether they complete scenarios, quality shows whether answers help, and business impact shows whether the bot pays off.
| Level | Question | Audience |
|---|---|---|
| Reach | How many people arrive, and from where? | Marketing |
| Scenario funnel | Where do users abandon a flow? | Product, engineering |
| Quality | Does the bot resolve the issue? | Support, content |
| Business impact | What did the company save or earn? | Leadership |
Many reports stop at the first level: “the bot has 20,000 subscribers” says nothing about its impact.
15 chatbot metrics and how to calculate them
You do not need to track everything at once: choose one or two metrics from each level that match the bot’s goal.
| # | Metric | Formula | Level |
|---|---|---|---|
| 1 | First-time users by source | Unique users with a first recorded interaction and a source tag | Reach |
| 2 | Monthly active users | Unique users with at least one action | Reach |
| 3 | Unsubscribes and blocks | Unsubscribes in period ÷ users who received a message in the same period | Reach |
| 4 | Scenario completion | Reached the end ÷ started the scenario | Funnel |
| 5 | Highest-drop-off step | Step with the largest exit share | Funnel |
| 6 | Scenario completion time | Median time from start to outcome | Funnel |
| 7 | Share of requests confirmed resolved without an agent | Requests confirmed resolved without an agent and with no same-issue repeat in the observation window ÷ all requests in the same cohort whose full observation window has elapsed | Quality |
| 8 | Human handoff rate | Requests handed to an agent ÷ all requests in the same period | Quality |
| 9 | AI “I don’t know” rate | Queries not answered from an approved source ÷ all queries | Quality |
| 10 | Answer rating | “Helpful” ratings ÷ all rated answers | Quality |
| 11 | Repeat requests on the same issue | Requests repeated within 7 days ÷ requests with a full 7-day observation window | Quality |
| 12 | First response time | Median time from request to first reply | Business impact |
| 13 | Bot-sourced leads and deals | Deals attributed to “bot” ÷ leads from the bot | Business impact |
| 14 | Estimated hours freed | Confirmed requests resolved without an agent × handling-time difference from baseline | Business impact |
| 15 | Cost per request | Total bot costs for period ÷ requests in the same period | Business impact |
For some loyalty programs, add NPS: it measures willingness to recommend the company, but does not replace whether a specific request was resolved. Ask at an appropriate point, not after every message.
Traffic sources: deep links and QR codes
To identify which campaign or location brings users in, give each source its own tagged link to the bot. Without that, source attribution is guesswork.
Examples from our projects:
- Veres: each advertising campaign gets a separate deep link. The click records the source, and users enter a segment with its own message sequence. See the Veres case study.
- UA Made: each store gets deep links and QR codes automatically. Charts show registrations by platform, store, and period, as well as bonus-card clicks. See the UA Made case study.
How to measure business impact
Measure team time and revenue separately; both require a baseline and an attribution check.
- Time freed. Confirmed requests resolved without an agent × the difference between manual handling time before launch and current review/oversight time × the actual internal hourly cost. This is a scenario estimate, not an automatic cost reduction.
- Revenue. Track CRM deals tagged with “bot,” separating new customers from repeat purchases. Compare with a control or earlier cohort: a bot tag alone does not prove the bot caused the deal.
- Costs. Development (amortized over its useful life), server, AI API, messaging (for Viber), and support.
To measure bot-attributed revenue, each CRM deal needs the “bot” source and a specific scenario or campaign tag. See chatbot and CRM integration.
Dashboards for different roles
One dashboard rarely serves everyone: marketing needs sources, support needs answer quality, and leadership needs economics.
- Marketing: new users by source, active users, campaign response, unsubscribes.
- Support and content: automated request share, human handoffs, “I don’t know” answers, ratings, repeat requests.
- Sales: bot leads, deal conversion, first response time.
- Leadership: hours freed, cost per request, monthly bot-attributed revenue.
Learning bots can also track learning outcomes: administrators of the Astra Dent training bot can see each employee’s quiz results.
Common chatbot analytics mistakes
The most common mistake is having no baseline: without measurements from before launch, the bot’s effect cannot be demonstrated.
- There is no “before” data for response time, request volume, or conversion.
- Everyone arrives through one link, so the source is unknown.
- Bot-sourced deals are not tagged in the CRM.
- “Automated requests” include cases where the customer returned with the same question.
- Nobody reviews AI answers.
Our guides to inquiry processing and reducing contact center costs explain how to establish a baseline before launch.
Conclusion
A chatbot’s performance becomes visible when metrics cover four levels: reach, funnel, answer quality, and business impact. Tag sources through deep links and QR codes, pass a “bot” source into the CRM, record a pre-launch baseline, and create separate dashboards for marketing, sales, support, and leadership. Subscriber count alone says nothing about return.
Frequently asked questions
The share of automated requests, completion of key scenarios, first response time, and the number of bot-sourced CRM deals. Choose the rest based on the bot’s goal.
Create a separate deep link or QR code for each campaign, location, or placement. A visit through that link records the user’s source.
Estimate incremental gross profit and the verified value of time freed, then subtract total costs for the same period. ROI = net impact ÷ total costs × 100%. Compare with a baseline; do not count every deal tagged “bot” as incremental revenue.
Track the “I don’t know” share, “helpful” ratings, human handoffs, and repeat requests on the same issue; regularly review low-rated answers as well.
Sometimes, if the goal is loyalty. To assess a specific answer, ask separately whether the issue was resolved; NPS does not replace that measure.
No. Check whether the issue was resolved, whether the customer returned with the same question, and whether the team spent extra time reviewing the interaction. Only then estimate hours freed.
Sources
Chatbot analytics implementation consultation
In 15 minutes with a BotLabs implementation specialist, discuss your chatbot goal and available data. Choose a key metric, how to calculate it, and the missing events needed to compare results with a baseline.
One metric.
Data for comparison.
15 minutes with an implementation specialist: a metric, a baseline, and the events to track.
