What a custom AI system actually is

A custom AI system is software built around a particular business task, its information and its working rules. It does not necessarily involve training a new model. In this article, we mean systems using language models to help find information, prepare work or carry out controlled steps.

The useful buying question is not "Can we have our own AI?" It is "Which job is difficult now, and what would a dependable improvement look like?"

Compare existing software first

Business software and AI assistants may already offer document uploads, connectors, search or workflow features. Check what your current subscriptions can do before commissioning a separate product.

A custom build becomes more plausible when the work crosses several systems, requires specific permissions or needs a process existing software cannot reasonably support. A simple reminder or calculation may need ordinary automation, without a language model.

Our recommendation is to write one example of the desired input and output. "Find the current delivery policy and show its source" is easier to assess than "Give everyone an intelligent assistant."

Retrieval brings relevant information into the answer

One possible design searches an approved document collection and supplies relevant material to a model. This approach is commonly called retrieval-augmented generation, or RAG. The original RAG research paper describes combining generation with retrieved information; commercial implementations vary.

Retrieval is not a guarantee of accuracy. The document may be old, the search may select the wrong passage or the generated answer may misinterpret it.

For a fictional maintenance business, a useful assistant might find a current equipment procedure and link directly to the source. It should distinguish that procedure from an obsolete revision. If the information is missing or the user lacks permission, it should stop short of inventing an answer.

Before adding more documents, decide who owns them, which version is current and who should be allowed to read each one.

Reading and acting need different controls

An assistant that summarises a record is different from one that changes it or sends a message. The latter needs clear limits, permission checks and an approval path appropriate to the consequences.

Model Context Protocol, or MCP, is one standard for connecting AI applications with data and tools. Its architecture documentation describes clients and servers exchanging capabilities such as resources and tools. Using the standard does not, by itself, make a business integration safe or correct.

For an initial pilot, we recommend allowing the system to prepare work for review. Test changes to live records separately, including duplicate requests, failures and recovery. Do not give an assistant broader access than the person or workflow it serves needs.

Buy an evaluation, not only a demonstration

A supplier's polished example is useful, but your test set should include the awkward cases too:

• A straightforward question with a known answer.
• Conflicting versions of a document.
• Information that is absent.
• A request involving another user's restricted information.
• An instruction hidden inside an untrusted document.
• A disconnected service or a repeated action.

Agree the expected behaviour before testing. Record correct answers, unsupported answers, missed information, review time and operational failures. The number of examples depends on the task; no fixed count proves a system is reliable.

NIST's AI Risk Management Framework provides a broader framework for identifying, measuring and managing AI risks. The checks above are our practical starting point for a small-business pilot, not a certification.

Include the work after launch in the price

Ask for separate estimates for discovery, integration, document preparation, testing, rollout and support. Running costs may combine subscriptions, model usage, hosting, monitoring and human review. Check the supplier's actual charging model rather than assuming every AI service is billed in the same way.

Also agree who owns the code and accounts, how data can be exported, who maintains integrations and how access is removed when someone leaves. Operational records should be proportionate: access controls, retention and redaction matter more than keeping every prompt forever.

Choose one outcome worth measuring

A good initial objective might be faster preparation of a checked quotation, fewer incomplete internal requests or less time locating an approved procedure. Establish the current time and error rate before the pilot, then measure the same work afterwards.

Time released is not automatically a payroll saving. It may create capacity, reduce overtime or improve responsiveness; whether a cost actually disappears is a separate business decision.

Read our automation guide for a worked approach. If you need implementation, our sister agency Suffolk Web builds business AI systems. That relationship is disclosed on our about page.