Adding AI to your existing product
There's a version of this conversation that goes badly: an ambitious plan to "add AI", six months of work, a chat box nobody opens. There's another that goes well, and it starts from a much smaller question — where in this product do users currently do something boring, repetitive and language-shaped?
Choosing what to build
The features that earn their keep tend to fall into four buckets:
- Summarising. Long threads, documents, call transcripts, tickets — anywhere a user has to read a lot to get a little.
- Drafting. The reply, the description, the report, the summary email. Users are usually happy to edit; they resent starting from a blank page.
- Extracting structure. Turning a messy invoice, email or form into clean fields. Unglamorous, enormously valuable, and easy to verify.
- Answering questions about their own data. Not general knowledge — their documents, their records, with citations.
The test is simple: if the feature disappeared tomorrow, would a user have to do that work by hand? If yes, build it. If it only exists to say the product has AI, don't.
How it actually plugs in
The common shape is unremarkable engineering. Your backend gathers the relevant context — the document, the record, the last few messages — sends it to a model API with clear instructions, and does something structured with the response. For anything involving your own content, retrieval matters more than the model: fetch the relevant passages first and give them to the model, so it answers from your data rather than its training. Keep the model call behind your own server, never in the browser, so your keys stay yours and you can log, cache and rate-limit.
Costs behave differently
Unlike most features, AI has a per-use cost that scales with usage rather than a fixed hosting bill. Before launch, work out roughly what a typical user costs per month and compare it with what they pay you. Then reduce it the boring way: send only the context that's needed rather than everything available, cache repeated requests, use a smaller model for simple steps and reserve the expensive one for hard ones, and put a sensible ceiling on heavy users.
Design for the wrong answer, because you will get one. The question is only whether it costs the user thirty seconds or their trust.
Designing for imperfection
Every AI feature should assume the output is sometimes wrong. In practice that means output arrives as an editable draft, not a completed action. It means showing where an answer came from so the user can check it. It means a visible way to say "that's not right", both as a signal to you and as an escape for them. And it means no irreversible action — sending, deleting, charging, publishing — without an explicit human confirmation.
Ship one and watch
Pick the single feature with the clearest value, put it behind a flag, and release it to a slice of users. Then look at the numbers that matter: how many people used it more than once, how often they kept the output versus rewriting it, and what it cost you. That evidence, not a roadmap written in advance, should decide what comes second.
- Target tedious user work — summarising, drafting, extracting, answering from their data.
- Use a general model via API with retrieval over your own content; don't train your own.
- Keep model calls server-side so you control keys, logging, caching and limits.
- Model costs scale with usage — measure cost per user before you launch.
- Present output as a draft, cite sources, and require confirmation for anything irreversible.
Frequently asked questions
Where should we add AI to an existing product first?
Look for the tasks your users find tedious rather than the ones that would look impressive in a demo. Summarising long content, drafting text they'd otherwise write from scratch, extracting structure from messy input, and answering questions about their own data are the reliable starting points.
Do we need to train our own model?
Almost never. Calling a general-purpose model through an API, grounded in your own data, covers the vast majority of product features and takes days rather than months. Training your own only makes sense with a genuinely unusual task and a large, high-quality dataset.
How do we handle AI getting things wrong?
Assume it will. Design the feature so mistakes are cheap: present output as a draft the user edits rather than a final action, cite the source so answers can be checked, and never let a model take an irreversible action without human confirmation.
ZIVARA adds AI features that earn their place in products people already rely on. Let's talk. Related: integrating AI without the hype and what is RAG.