Where programmes actually stall
The failure pattern is consistent and it is rarely technical. A team runs a generative tool against one task, produces something impressive for a leadership meeting, and six months later nothing is in production. A demo needs one good example. Production needs a maintained data source, permissions that respect who may see what, a defined human review path, specified behaviour on edge cases, and a monthly cost you understood before signing rather than after.
The measurement changes as well. A pilot is judged by its best output. A production system is judged by its worst acceptable one. When we build an internal assistant, the question that decides whether it survives is not how good the best answer looks. It is what the system does when the answer is not in the source material: does it say so and hand off to a person, or does it produce something fluent and wrong? That is a design decision you build and test for. No platform gives it to you by default.
The practical consequence is that the opening question is not which tool to buy. It is which process is expensive, repetitive and verifiable. Tooling is reversible at moderate cost. Choosing the wrong process burns something harder to replace than budget, which is the organisation's willingness to try again.
A five-part filter for the first use case
The highest-leverage decision in an AI programme is which use case goes first, not which vendor signs the contract. We apply five criteria with clients and ask that at least four hold before committing to a build.
Frequency: the task recurs daily or weekly at a volume worth automating, not once a quarter. Verifiability: a competent employee can judge whether an output is right in a minute or two, without an investigation. Blast radius: a first mistake does not reach a customer or drive a financial decision unreviewed. Data availability: the inputs already sit in a system you run today and do not require a full data programme first. Ownership: one named person inside the business has a materially better day if this works, and will spend time each week reviewing it.
The last criterion is skipped most often and matters most. Initiatives run purely out of IT, without an operational owner who feels the pain being solved, tend to end as a tool used only by the people who built it. If a candidate fails two or more criteria, it belongs in the second wave. That is sequencing, not rejection, and saying so early protects the case rather than killing it.
Four patterns that survive contact with production
Internal knowledge retrieval. A search-and-answer layer over your own material: policies, standard operating procedures, product manuals, the contract archive. The value is not conversation, it is removing the recurring questions that always land on the same three people who know everything. The condition that separates a working deployment from an abandoned one is citation. Every answer points to the document and the section, or nobody trusts it past the first week.
Document extraction. Pulling structured fields out of invoices, purchase orders, receipts and contracts, then pushing them into the accounting or ERP system. This pattern is easy to measure because a field is either correct or it is not, and it works at modest data volumes. Most of the return comes from the dull high-frequency documents, not the complicated rare ones — which is the opposite of where teams instinctively start.
Triage and drafting. Classifying inbound customer messages by type and urgency, summarising prior conversation history for the agent, and preparing a draft reply the agent approves or edits. The difference between this and an open chatbot is that a human stays in the loop, so handling time improves without giving up control of what goes out under your company name.
Forecasting and anomaly detection. Demand planning for stock, flagging outliers in expenses or claims, ranking leads by likelihood to close. Worth noting plainly: a large share of what gets requested under the AI heading is solved better by a straightforward statistical model or an explicit business rule — cheaper to run, easier to maintain, and far easier to defend to an internal auditor.
The pattern we see fail repeatedly is replacing whole professional judgement — pricing, credit assessment, hiring decisions — with model output and no review. The constraint is not capability. It is that accountability for the mistake stays with the company and cannot be delegated to a system.
Arabic is an engineering requirement, not a localisation task
Most platforms are built and benchmarked in English first. In an Arabic-language operation, gaps appear that no vendor demo will show you. Customer conversations arrive in dialect rather than standard Arabic. Document archives are frequently scanned images rather than searchable text. A single document mixes Arabic and English within the same line, which is normal in contracts, technical specifications and variation orders.
Then there are smaller details that break systems in practice: the same personal name spelled several ways, inconsistent hamza and taa marbuta, Hijri and Gregorian dates sharing one field, and national ID or commercial registration numbers written in varying formats. An extraction pipeline that does not handle these cases explicitly will produce data that looks clean and is not. Silent wrongness is more expensive than visible failure.
The rule we apply before any commitment: build an Arabic evaluation set from your own documents, by hand if necessary. One to two hundred real cases with the correct answer recorded for each is enough to separate two vendors that look identical in a presentation. Do not accept an evaluation built on translated English examples, because translation removes precisely the failure modes the set exists to expose.
Data readiness comes before platform selection
The usual question is whether there is enough data. The more useful question is who owns correcting it. A capable system sitting on data nobody is responsible for maintaining produces confident errors, which are worse than having no system at all, because confident errors get believed and acted upon.
Here is a check you can run this week at no cost. Does every important field have one agreed source of truth, or do three systems hold three conflicting versions? Do you know when each source was last updated? Are permissions defined at the document and row level, or does access to a system mean access to everything inside it? Is the document archive text or images? Is there a written retention and deletion policy?
Permissions in particular are the item discovered late and paid for dearly. An assistant that indexes everything on the shared drive will answer a warehouse employee's question about management salaries unless access rules are enforced in the retrieval layer itself, not merely hidden in the interface. Sorting permissions before the build is far cheaper than remediating after an incident, and considerably easier to explain afterwards.
Governance you should have in writing before rollout
Processing personal data in the Kingdom falls under the Personal Data Protection Law, with the Saudi Data and AI Authority (SDAIA) designated as the supervising authority. SDAIA has also published AI Ethics Principles covering areas including privacy and security, transparency, and accountability. Treat both as a starting point for the internal conversation only: any actual arrangement touching personal data or a regulated activity is subject to review by a qualified legal adviser before you rely on it.
Operationally, four documents are enough to begin and none of them takes months. First, an acceptable use policy naming what must not be pasted into public tools — customer records, contracts, source code, employee files. Second, a list of cases requiring human review before an output reaches a customer or informs a decision. Third, a log of who used which tool for what purpose. Fourth, explicit contract terms with the vendor covering where data is processed and stored, whether your inputs train their models, how long anything is retained, and how deletion works when the relationship ends.
One observation from doing this repeatedly: write these before adoption spreads, not after. Staff are already using public tools whether or not a policy exists. The absence of a policy does not remove the exposure — it removes your visibility into it, and with it any ability to manage it.
Baselines, unit cost and a stop rule
Record the baseline before launch. How long does the task take today? How many cases are reworked? Who does it, and at what cost of their time? Without that number captured beforehand, you cannot demonstrate impact afterwards, and the discussion collapses into competing impressions at the next budget meeting.
Three indicators are enough to judge the first ninety days. Cycle time from intake to completion. The share of outputs accepted without material edits. And operating cost per completed task, including both model consumption and the human review minutes. The third is what kills projects after scale-up, and it is almost never calculated during the pilot, because volumes are small and the cost is invisible.
Add a fourth indicator that is not technical at all: how many people used the tool in week four compared with week one? A sharp drop is a direct signal that the tool solves a problem nobody actually had, however well it scored in testing. Adoption curves are more honest than accuracy scores.
Finally, write the stop rule before you start. Which number, if unmet within a defined window, ends the effort? Initiatives without a stop rule do not conclude; they fade, consume attention and spend the team's appetite for the next attempt. Ending a pilot on schedule is sound management, not an admission of failure.
Key takeaways
- Select the first use case on verifiability and a named operational owner, not on how new the technology is.
- Build an Arabic evaluation set from your own documents before committing to any vendor.
- Resolve permissions and data ownership before building an internal knowledge assistant.
- Write the acceptable use policy and human review boundaries before tools spread through the company.
- Capture a baseline and a stop rule at launch, and measure cost per task rather than cost per month.
Frequently asked questions
Where do we start if our data is not organised?
Start with a use case that does not depend on organising everything. Document extraction is a good example: the input is a file, the output is fields, and no unified warehouse is required. A knowledge assistant over company documents does need a baseline of structure and permissions, because answer quality equals source quality. Tidy one bounded domain rather than waiting for a full data programme.
Should we use an off-the-shelf model or build our own?
Start off-the-shelf in most cases. Building your own is justified when you hold substantial labelled domain data, when a constraint prevents data leaving your environment, or when usage volume makes running cost the deciding factor. Otherwise, investment in source preparation and review design returns more than model training.
Where is our data processed, and does it leave the Kingdom?
That depends on the vendor and deployment configuration, and it belongs in the contract rather than in a sales conversation: processing and storage location, whether your inputs are used to train the vendor’s models, retention period, and the deletion mechanism. If the data is personal or sector-regulated, have the arrangement reviewed by a qualified legal adviser before you rely on it.
How long before we see a first operational result?
For a single scoped use case with available data and a partly dedicated owner, expect weeks rather than months to reach genuine use with human review in place. Timelines stretch when documents are scanned images, when permissions are unresolved, or when the case spans several departments with no single owner.
Sources
- Personal Data Protection Law — Bureau of Experts at the Council of Ministers, Saudi Arabia · https://laws.boe.gov.sa/BoeLaws/Laws/LawDetails/b7cfae89-828e-4994-b167-adaa00e37188/1
- AI Ethics Principles, issued by SDAIA — Digital Government Authority, Saudi Arabia · https://dga.gov.sa/en/AI-Ethics-Principles
- Laws and Regulations — Saudi Data & AI Authority (SDAIA) · https://sdaia.gov.sa/en/SDAIA/about/Pages/RegulationsAndPolicies.aspx