How to Build a Chatbot That Actually Knows Your Business

ChatGPT can't answer "What's our refund policy?" or "When is the Smith account due?" because it was never trained on your data. It knows the internet. It doesn't know your business.

RAG changes that. And it's simpler than you think.

What is RAG? (The librarian analogy)

RAG stands for Retrieval Augmented Generation. Fancy name, simple idea.

Think of it as hiring a librarian for your AI:

• Someone asks a question
• The librarian (RAG) goes to YOUR bookshelves — your documents, policies, procedures
• Finds the most relevant pages
• Hands them to the AI
• The AI reads those specific pages and answers the question

The AI doesn't memorise your entire business. It looks things up — like a librarian, not a god.

What you can feed it

• Company policies and procedures
• FAQ documents
• Product catalogs and price lists
• Client onboarding guides
• Training materials
• Email templates and standard responses
• Basically any text your team references regularly

How it actually works (without the jargon)

Step 1: Your documents get chopped into small chunks — paragraphs, not whole files.

Step 2: Each chunk gets converted into a mathematical fingerprint (called an embedding).

Step 3: These fingerprints go into a database.

Step 4: When someone asks a question, the system finds the chunks with the most similar fingerprints.

Step 5: Those chunks get sent to the AI along with the question.

Step 6: The AI writes an answer based ONLY on what it found.

This is why RAG chatbots don't hallucinate as much — they're answering from your actual documents, not making things up.

What it costs

• Database: Free to £20/month (Supabase free tier handles most SME needs)
• AI model: £10-50/month depending on usage
• Workflow tool: Free to £20/month (n8n self-hosted is free)
• Total: £0-90/month in running costs

The setup takes 1-3 days with the right expertise.

What it can't do

• It can't reason about data it hasn't been fed
• It's only as good as your documents (garbage in, garbage out)
• It won't replace complex human judgment
• It needs updating when your policies change

Who's using this today

• Law firms: answering client queries from case files
• Accountancy practices: staff looking up HMRC guidance
• Recruitment agencies: candidates asking about job details
• Property management: tenants checking policies

These aren't futuristic experiments. They're working systems deployed right now in UK service businesses.