PIPELINEGuide
AI chatbot: measure success, don't estimate it
Without a previously defined metric, you can't say afterwards whether an AI chatbot was worth it. Which numbers work, which mislead, and when to collect them.
The success of an AI chatbot can only be measured if the baseline value was recorded beforehand — two weeks of tracking is enough. Useful metrics are processing time per case, share of cases that complete without intervention, and external response time. Team satisfaction in the first two weeks is not a useful benchmark.
01 — Measurement
How you measure success
- Define the one number beforehand. Without a baseline value, nothing can be proven afterward. Two weeks of tracking is enough for a reliable reference point.
- Processing time per case. The most direct metric: How long does the affected process take today, how long afterward.
- Percentage of cases that complete without intervention. This ratio typically rises significantly in the first weeks when refinements are made — it's the best early indicator.
- External response time. How quickly does someone who inquires receive an initial binding answer.
- What you should not measure: Team satisfaction in the first two weeks. Every transition feels worse initially than it actually is.
02 — Calculation
How to calculate it
Four numbers are enough: How often does the process occur? How long does it take? What does an hour cost internally? And what does the solution cost — one-time plus operation over three years?
An example: A process that occurs twice a week and takes 40 minutes each time ties up around 69 hours per year. At an internal hourly rate of 45 euros, that's roughly 3,100 euros per year locked in there. If a solution reduces this effort by two-thirds, it often pays for itself — depending on the actual quote — within one to two years. We'll give you the exact figure for your case after the initial meeting.
This calculation is deliberately conservative: it only accounts for the time saved, not the errors avoided or the orders that come through due to faster response. In practice, the benefit is usually higher — but you shouldn't count on it before you've measured it.
03 — Context
What experience shows
Around two-thirds of all initial inquiries at small businesses are repeat questions that a well-maintained knowledge base covers completely.
04 — Questions
Frequently asked questions
Does this even pay off for us?
The calculation is simple: requests per week times processing time times hourly rate. With twenty repeat questions per week and seven minutes each, that's a good two hours of work — every week. We calculate this with your real numbers before the quote.
How much work does this create for us?
At the start, about two to three hours for gathering content and one alignment round. After that, nothing regular — unless you want to add new topics, which works through a simple document.
Do we have to label the chatbot as AI?
Yes. Since August 2026, the transparency requirements of Article 50 of the EU AI Act apply: Anyone communicating directly with an AI system must be able to recognize it. We build the labeling in as standard, it costs nothing extra.
Does this also work on a WordPress site?
Yes. The integration is a single code snippet in the footer and works the same on WordPress, Shopify, Webflow, Wix, TYPO3, or a custom-built site.
What happens to our customers' data?
The conversations run through servers in the EU, no training is done with your data, and we write the appropriate section for your privacy policy. On request, chat logs are automatically deleted after a set period.
Doesn't an AI chatbot just make up answers?
A free-running language model does. That's why we build the bot so it only answers from a verified knowledge base and provides the source. If it finds nothing, it says so and redirects — that's the most important part of the configuration.
Can we turn the bot off again?
Anytime, with one click. The snippet remains, the bot is gone. There's no minimum term that binds you.
05 — Next
Related pages
- AI chatbot — Overview
- AI chatbot — Integration
- AI chatbot — how it works
- AI chatbot — data protection
- AI chatbot — AI disclosure
- AI chatbot — Ongoing operation
- AI chatbot — Implementation
- AI chatbot — Viability
- WhatsApp chatbot
- AI phone assistant
- Customer support automation
- All guides
AI chatbot by region
Next step
Initial assessment, no obligation.
Briefly describe what you need. You'll get an honest answer about what's realistic and what it will roughly cost.