AI chatbot for your website: designing an assistant grounded in your own content

Putting a language model behind a chat window takes an hour. Making it answer only with the right information, refuse the rest, resist manipulation and cost a predictable amount is the real work. This article describes how I approach it, starting from the example closest to hand: the chat on this site.

Grounded, not general-purpose

A general-purpose chatbot on a business website is a risk: it makes up prices, promises services that do not exist, answers on topics that have nothing to do with you. The useful assistant is grounded: it answers only from a defined knowledge base and, outside it, politely refuses and points to a person.

The expected behaviour is written before the code:

• covered question: short answer, with the next step (a page, a contact);
• partly covered question: what is known, plus a pointer to ask a person;
• out-of-scope question: a polite refusal and a reminder of what the assistant can do;
• attempt to bypass instructions: no change in behaviour.

Full context or RAG

There are two architectures.

• Knowledge base in the context: the whole reference text is passed to the model with every request. With a few dozen pages it is the best choice: no indexing, no retrieval errors, and with prompt caching the cost of the repeated part drops dramatically.
• RAG: relevant passages are retrieved and only those are passed. Needed when the content is too large for the context or changes constantly. It adds a point of failure, retrieval, which must be measured. I cover it in detail in AI assistant for internal documents.

For a presentation website, nine times out of ten the first route is enough.

Writing the knowledge base

This is the part that decides quality, and it is not generated automatically from the site's pages. Pages are written to persuade; the knowledge base must be written to answer: facts, conditions, limits, what you do not do. Every statement must be true and verifiable, and when a service changes the knowledge base is updated along with the site.

Security: prompt injection and surface

Users will try to make the model ignore its instructions. Effective defences are architectural, not just textual:

• separate instructions from content: rules live in the system prompt, and user input cannot overwrite them by construction;
• least privilege: if the chatbot can only answer from a public knowledge base, even a successful attack reveals nothing confidential. If it must read personal data or perform actions (book, edit), every operation is authorised and validated on the server, never left to the model;
• API key on the server only: the browser talks to its own backend, never directly to the model provider;
• rate limits per user and per IP, to avoid abuse and runaway costs;
• maximum length for questions and answers.

Cost and performance

Cost depends on the model, input and output tokens, and traffic. The main levers: caching the fixed part of the prompt, short answers by default, a model sized to the task (the most powerful one is often unnecessary) and streamed answers, which improve perceived speed at no extra cost.

Transparency, privacy, accessibility

• Users must know they are talking to an AI system: the EU AI Act sets transparency obligations on this point.
• Conversations may contain personal data: privacy notice, log retention periods, providers whose terms exclude training on submitted data.
• The chat must work by keyboard and with screen readers, with the message area announced correctly.

Before going live

A set of test questions (covered, partial, out of scope, manipulation attempts) run after every change to the knowledge base or instructions. It is the least visible part and the one that saves you from embarrassment.

Measuring whether it works

A site assistant is assessed with concrete indicators, not the impression that "it answers well":

• covered answer rate: how many questions find an answer in the knowledge base;
• refusals and handovers to a person: how many, and on which topics;
• recurring topics: what people really ask, often different from what the site highlights;
• enquiries generated: how many conversations end with a request or a click to the right page;
• explicit feedback, with a simple thumbs up or down.

Conversation logs, anonymised and kept for a limited time, are the best source of improvement: a frequent unanswered question points to information missing from the knowledge base, and often from the site too. In that sense a well-run assistant also becomes a tool for finding gaps in the content.

My take

The chat on this site is built this way: a single hand-written knowledge file, read by the server, passed to an Anthropic model with precise instructions. It answers only from what is written there, refuses everything else, has a rate limit and, if the API key is missing, falls back to canned answers instead of breaking. It does not know everything, and that is fine: I prefer an assistant that says "I don't know, write to me" to one that improvises.

I am wary of chatbots promising to "know everything about your business" after reading the site automatically. The value lies in a carefully written knowledge base and in the limits, not in the power of the model.

Why work with me

I have been programming since 1999, delivered more than 300 projects and work as an external developer with about 30 agencies, from Rome and remotely. I design grounded AI assistants with a curated knowledge base, server-side protections, controlled costs and testing before launch.

If you want an assistant on your site, get in touch: in a call of about 20 minutes we work out what it must know and what it must refuse. The service is in AI and automation.