Where enterprise assistants actually break
In the stalled projects we get called into, the language model is almost never the problem. The problem is the question the team started from. They asked which model to use, when the real question was where the answer currently lives, who owns it, and whether the version they are about to index is still valid. A model knows nothing about your leave policy or your supplier contract terms. Any assistant relying on its internal knowledge will produce something well written and wrong, which is a worse outcome than producing nothing.
The second failure is that the pilot always looks excellent. A handful of clean documents get loaded, the team asks questions it already knows the answers to, and everyone leaves the room convinced. Then real users arrive, phrasing questions nobody anticipated, against messy repositories where the same policy exists twice with different numbers. The gap between demo and production is wider in retrieval systems than in almost any other kind of software, because the defects live in the content rather than in the code.
The pattern comes from a 2020 research paper, and the idea is deliberately simple: rather than answering from memory, the system retrieves relevant passages from your own sources and the model answers strictly from those. Simple as a concept, demanding in practice, because every hard part moves into the first step — finding the right passage.
A decision test before you spend anything
Run the use case through four questions. Is the answer written down anywhere? If the knowledge lives in three people's heads, you have a documentation problem, and an assistant will only make the emptiness visible faster. Does the content change? Retrieval handles moving knowledge well; fine-tuning is a reasonable way to fix tone, format or domain vocabulary, and a poor way to carry facts that get revised every quarter.
Does the user need to verify the answer? If someone will base a client commitment, a pricing decision or a regulatory response on it, a citation is a requirement rather than a nice touch, and any system that returns prose without an openable source is unfit for that use. Finally, is the request knowledge or action? "What is the travel allowance cap?" is retrieval. "Submit my travel request" is an integration with write access to a system of record. Bundling both into one scope is a reliable way to make a project slip.
There is a fourth honest answer we sometimes give: build nothing yet. If your policy library is uncatalogued and no document has a named owner, cleaning that up and improving your existing search may deliver a faster result at lower cost — and makes the assistant project, when it comes, dramatically shorter.
The first deliverable is a source inventory, not a vector database
Before any pipeline is written, produce a table. One row per source: name, named owner, format, last updated, whether it is the authoritative version or a draft, and who is entitled to read it. It is unglamorous work, and it surfaces within days what usually emerges months in — that the same procedure exists in three places and nobody can say which one governs.
The inventory forces decisions no amount of tuning can substitute for. What gets excluded permanently: drafts, dead project folders, the archive from the old ERP? Which source wins when two conflict? Which documents need work before they can be indexed at all — scanned PDFs that contain images rather than text, for instance, which require optical character recognition and should be priced as a separate line item rather than discovered halfway through.
Our position, formed from doing this repeatedly: the highest-return improvement happens before a line of retrieval code is written. Removing one superseded policy from the index improves answers more than swapping in a larger model. Declining to fund this stage starts the project with a debt that later surfaces as wrong answers whose cause is genuinely hard to trace.
The pipeline, and where quality is really won
Six stages sit between a question and an answer, and each can ruin the result on its own. Ingestion and parsing extract text and structure from documents; a table flattened into a text blob loses its meaning entirely, and an approval matrix built from columns becomes a meaningless line. Chunking splits documents into retrievable passages. The working rule is to follow document structure — clause, article, subheading — rather than a fixed character count, and to prefix each chunk with a header naming document, section and date so it still makes sense out of context.
Retrieval is where the important decision sits. Semantic search alone is unreliable on part numbers, clause references, product codes and account identifiers. Keyword search alone fails the moment a user phrases a question differently from the document. Combining both is not engineering indulgence; in our experience it is the difference between a system people keep using and one they abandon within a fortnight. Re-ranking then filters what was retrieved, passing the model fewer and better passages, because flooding the context with marginally relevant text lowers answer accuracy rather than raising it.
Grounded generation and citation close the loop. The instruction to the model states explicitly that it answers only from the supplied passages and says so plainly when the answer is not there. And the citation must open the passage itself, not the file. A link to a forty-page PDF means, in practice, that nobody will ever check anything. Taken together, most of the quality in these systems comes from parsing, chunking and retrieval — not from moving to a bigger model.
Bilingual retrieval is a design constraint, not a translation task
In Saudi organisations the normal case is an employee asking in Arabic about a document written in English, or the reverse, mixing an Arabic term and a Latin-script product name inside a single sentence. This is not an edge case to handle later. Treated as an afterthought it produces two separate indexes and a system that answers half the questions asked of it.
The details that get missed are specific. Arabic morphology, optional diacritics and inconsistent hamza spelling make keyword matching alone unreliable. Borrowed business terms appear in both a native and a transliterated form. System and product names sit in Latin script inside Arabic sentences. And the one that causes the most rework: Arabic PDFs whose text layer extracts as disconnected letterforms or reversed character order. Test a sample of your real documents in week one, not during QA — discovering this late means redesigning the ingestion pipeline rather than patching it.
In practice we build a single index covering both languages with hybrid retrieval, query expansion generating alternative Arabic and English phrasings before searching, and an organisational glossary linking approved terms to their variants. The answer is written in the language of the question, while the quoted passage stays in the language of the source document, untranslated. Translating a contractual clause inside the answer turns the citation from evidence into a new source of ambiguity.
Permissions and the data path
The sharpest risk in these systems is not a wrong answer. It is a correct passage reaching someone who was never entitled to read it. An index consolidates content from many repositories into one place by design, and if entitlements are not applied at retrieval time you have built an efficient leak. The rule we work to: read permissions from your existing identity provider and apply them to results before anything reaches the model. Do not let a parallel permission model be built inside the assistant, because a parallel model drifts from the original within months and nobody notices until it matters.
Then the data path itself. What is sent to the model provider on each request, where the index is stored, and how long question logs are retained. Logs are the commonly overlooked one: query text routinely contains a client name, a contract number or personal data, and gets stored by default because nobody decided otherwise. Set retention and access rules before launch, not after an audit asks.
Any processing of personal or client data here is subject to applicable regulation and to your own security and privacy policies; that assessment belongs with qualified legal counsel and your security team before go-live, not deferred as an implementation detail. Published frameworks exist to structure the risk work — the NIST AI Risk Management Framework is one commonly used reference, and it makes a workable agenda for the internal conversation even where the regulatory context differs.
Proving it works before real users see it
An impression after a demo is not a measurement. What you need is an evaluation set: real questions harvested from internal support channels, staff emails and service tickets, each paired with the document and passage that constitutes a correct answer. Build it before development starts. Built afterwards, it quietly becomes a test written to fit the system that already exists.
Include three categories. Direct questions with one answer in one document. Compound questions requiring two documents to be combined, which is where weak retrieval reveals itself. And questions your content genuinely does not answer — the most valuable category, because it tests refusal behaviour. A system that invents an answer for an unsourced question in testing will do exactly that in production, in front of someone who believes it.
Judge each result on three things: was the right source retrieved at all, is the answer grounded in the retrieved passages without additions, and is it phrased usefully for the person reading it. That judgement belongs to your subject matter expert, not the engineering team — an engineer has no way of knowing that the governing clause is the amended annex rather than the original. Then re-run the whole set on every change: a model version update, a chunking adjustment, a new source. It is the only safety net that catches an improvement in one place breaking another.
What it costs to keep running
An assistant is a service you operate, not a project you hand over. Sync and re-indexing run continuously as documents change. Model versions move and behaviour moves with them — reason enough to re-run the evaluation set on a schedule, not only when something visibly breaks. Running cost scales with usage rather than project size, which surprises teams that budgeted from the build phase.
The most valuable output after launch is often not the assistant itself but the gap report: the list of questions asked for which no source existed. That list turns documentation from a permanently deferred task into a backlog ordered by what people actually ask. In our experience it is the artefact that convinces a sceptical executive the project was worth funding.
Before signing with any supplier, ask four direct questions. How are permissions applied and where are they read from? What does a citation look like — does it open the passage or the file? What is the evaluation set and who judges it? And what happens when there is no answer? A supplier without a specific answer to all four has not run one of these in real production.
Key takeaways
- Confirm the answer is actually written down, in a document with a named owner and a current version — if it is not, you have a documentation project, not an AI project.
- Build an evaluation set from real questions before development starts, and include questions your content cannot answer so refusal behaviour is tested.
- Require citations that open the source passage itself; a link to a whole file guarantees nobody will verify anything.
- Apply entitlements at retrieval time from your existing identity provider, and refuse any parallel permission model inside the assistant.
- Start with one department and one document type until quality is stable, then expand — early breadth hides the cause of every problem.
Frequently asked questions
Do we need to fine-tune a model on our documents?
For enterprise knowledge, usually not. Fine-tuning is effective for tone, output format and specialised vocabulary, and a poor mechanism for carrying facts that change, since every policy revision implies retraining. More decisively, a fine-tuned model cannot point to where an answer came from, which rules it out wherever the reader needs to verify before acting.
Can this run inside our own infrastructure without sending documents to an external provider?
In principle yes, and the trade-off deserves to be stated plainly: full self-hosting gives you maximum data control at the cost of higher infrastructure and operations spend and a narrower model selection. Middle options exist, such as deploying a model inside your own cloud tenancy. The choice follows your data policy and content sensitivity, and should be made during the inventory stage rather than after the build has started.
What happens if the assistant gives an employee a wrong answer?
Protection is layered: generation constrained to retrieved passages, a visible citation so the reader can check, explicit refusal when no source supports an answer, and an in-interface reporting channel whose cases are logged and reviewed. The system should also be positioned internally as a faster route to the authoritative document, not as the authority itself.
Where do we start if our content is disorganised?
With one narrow scope: a single department, or a single document type whose owner and current version are known. Cleaning a bounded set and proving answer quality there is far quicker than tidying everything before you begin, and it gives leadership a concrete result to justify expansion. Starting across every repository at once tends to produce a system that is mediocre everywhere and trusted nowhere.
Sources
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — arXiv · https://arxiv.org/abs/2005.11401
- AI Risk Management Framework — National Institute of Standards and Technology (NIST) · https://www.nist.gov/itl/ai-risk-management-framework