9 July 2026
AI for handling email queries — a knowledge base, sources and human handoff
AI for handling email queries isn’t an “autonomous inbox” that runs everything on its own, or a cheap chatbot — it’s a process built on a knowledge base: an incoming email is classified, and a clear rule decides when the system answers automatically from your materials (with a source) and when it hands the matter to a person (human handoff). The point is to lift repetitive replies off the team — the ones your knowledge base can answer in full — so people spend their time only on the hard emails the base can’t answer.
This post is about building it sensibly: what can be automated, when the agent should escalate, how to secure the data and how to measure answer quality. Email is a different channel from a website chat: questions are longer, often multi-part, and the answer is sometimes a document rather than a single sentence. If you want the hotel-guest variant in chat, see the AI hotel concierge — here we focus on inbound mail in sales, quoting and support.
Why email differs from a website chatbot
A website chatbot catches short, real-time questions. The inbox is different: a request for quote (RFQ) with five points, a change to an order, a complaint with an attachment, a question about group availability. The answer has to be complete, consistent with the earlier thread, and often needs data from several sources at once.
So the good pattern isn’t “a bot that guesses” but an assistant that classifies, answers what it can fully ground and knows when to step back. Routine questions it answers automatically when your knowledge base holds a complete, sourced answer; the hard ones — no answer in the base, price, contract, an exception — it hands to a person early, with context.
The map: query type → source → automation → escalation condition
Instead of “AI will run the whole inbox” — four query types, each with a different data source, a different level of automation and a different condition for handing to a human.
| Query type | Data source | Possible automation | Escalation condition |
|---|---|---|---|
| Product/service question (FAQ) | Product cards, terms, price list, FAQ | A complete, sourced answer sent automatically | No source / low confidence → human |
| Request for quote (RFQ) | Price list, availability, discount rules | A draft quote with line items and terms to verify | Non-standard price, negotiation → sales rep |
| Order / booking change | Order/booking system (read) | A proposed change, confirmed by a human | Any write to the system → human |
| Complaint / sensitive matter | Customer history, procedures | Classification and a first summary for the team | Always → human (tone, risk, exception) |
Note the rule that returns in every row: a complete, grounded answer can go out automatically; everything else — a draft to verify, a write to your systems, a judgment call — stays with a person. It’s the same boundary we set out for agents in general in production-grade AI agents and on the AI agents page.
When the agent answers on its own, and when it escalates
The most important design decision isn’t “how well the model writes” but when we don’t let it. Three signals that should force escalation to a human:
- Low confidence or no source. If RAG finds no coverage in your materials, the correct response is “handing this to the team”, not guessing. A correct refusal is a quality trait, not a failure.
- High stakes. A non-standard price, contract terms, a complaint, an exception to a procedure — cases where a mistake costs the relationship or money.
- A write to a system. An order, booking or payment change is irreversible or hard to undo. The AI proposes, a human confirms.
The handoff has to be clean: the human receives context (the classification, the sources found, the draft), not a raw email from scratch. A well-designed handoff shortens handling time on the cases a person still has to own.
How to secure inbox data
Email is one of the most sensitive data sets in a company: personal data, commercial terms, attachments. Three principles we start from:
- Data stays with you. We build the system in your own AWS account; content doesn’t go to public models or their training. Architecture for compliance matters more than a policy on paper.
- Permissions and scope. The knowledge base the AI answers from covers only what it may cite — not the whole company inbox. It’s the same permissions problem we break down in data readiness for RAG.
- Guardrails with tests. Filters that police what the bot won’t write only make sense with a test set of real queries. Guardrails without tests are decoration — we develop this in guardrails are not an AI policy.
How to measure answer quality
“It works” without measurement is a hunch. In email handling, four things matter: relevance (does the answer resolve the matter), groundedness (can you check where it came from), correct escalation (do hard cases reach a human rather than get force-closed) and consistency (do two customers with the same question get the same answer). You build a test set of real queries — question, expected answer, source — before deployment; how to do it is in evaluating RAG.
Where to start
Not with “AI will run all the mail”, but with one narrow query type on a clean source — usually repetitive product or terms questions, starting in draft-for-approval mode and switching to automatic sending once quality is proven on a real test set. A PoC in 30 days like that will show real quality on your correspondence and a real cost before you widen the scope. The same pattern — RAG with citation and quality measurement over sensitive data, on AWS — we run on our own healthcare product, mojApteczka: proof that the pipeline works where a mistake costs more.
FAQ
Where do I start automating email handling?
With one narrow query type on a clean source — usually repetitive product or terms questions, starting in draft-for-approval mode, then switching to automatic sending once quality is proven. A PoC on that slice shows quality and cost before you widen the scope to other kinds of case.
When can AI answer on its own, and when must it escalate?
On its own — when your knowledge base holds a complete, grounded answer and the stakes are low (e.g. a routine product FAQ); the reply goes out automatically. It escalates — when a source or confidence is missing, when the case is high-stakes (non-standard price, contract, complaint), or when it would require a write to a business system.
How does this differ from a website chatbot?
A chatbot catches short, real-time questions. Email means longer, multi-part queries (RFQs, complaints, group inquiries) that need a complete, consistent answer, often from several sources — so the hard ones end with a person, not an automatic reply.
How do you secure personal data from correspondence?
We build the system in your own AWS account, content doesn’t go to public models, and the knowledge base covers only data the AI may cite. Add per-role permissions, a clear legal basis and a retention period — architecture for compliance, not a policy on paper.
How do you measure whether the answers are good?
On four dimensions: relevance, groundedness, correct escalation of hard cases, and consistency across customers. You build a test set of real queries (question, expected answer, source) and measure quality on it before deployment and after changes.
What next
How we build a knowledge base with source citation — on AWS, your data staying with you — is on the RAG for business page; how an agent acts in your systems within set boundaries — on the AI agents page. The guest-service variant in a hotel (chat, multilingual) is on AI for hotels. If you don’t know where to start, start with an audit: we’ll find which query types in your inbox can be safely automated and which must stay with a human.