Someone always wants an AI feature.
Summarize this. Autocomplete that. Add a chatbot to the page that already has three chatbots. Put a magic wand on the settings screen because a competitor’s marketing site has one.
Prefer AI. you ship AI. Also say no a lot. Saying no is how the yeses stay useful.
The real cost is not the API bill
Tokens are cheap compared to:
- support burden when the feature is confidently wrong
- eng time babysitting prompts instead of fixing the core product
- trust debt when users learn to ignore your “smart” suggestions
- compliance review for anything that touches regulated data
If you only price the model call, you will green-light a lot of glitter.
five questions before we build
Ask these out loud. If we cannot answer them, we do not start.
1. What job gets faster or safer? If the answer is “it feels modern,” that is not a job.
2. What happens when it is wrong? Wrong code suggestion in a sandbox is fine. Wrong clinical-sounding advice in a patient chat is not. Design the blast radius before the demo.
3. Who owns the output? If nobody reviews it, you built an unsupervised intern with production credentials.
4. Can we measure success in two weeks? Time saved, error rate down, conversion up, tickets down. Pick one. “Engagement” is where features go to pretend they matter.
5. What is the non-AI alternative? Sometimes a template, a checklist, or a better form wins. AI should beat the boring option, not just exist beside it.
Patterns you usually approve
Say yes when the work is narrow, repetitive, and checkable:
- draft a reply the human sends
- classify and route a ticket
- extract fields into a schema
- summarize a call the team already recorded
- generate a first-pass FAQ from approved source docs
Notice the theme. AI proposes. Systems constrain. Humans keep the keys on anything that moves money, care, or reputation.
Patterns you usually reject
- open-ended “ask us anything” bots with no retrieval and no escalation
- features that invent policy when the KB is empty
- rewriting the whole app because a model suggested a cleaner architecture overnight
- AI that emails customers without a human gate on day one
- anything whose main KPI is “we can put AI on the homepage”
Also: features that duplicate a tool the team already ignores. Automating a broken process just gets you a faster mess.
The demo trap
Demos are intoxicating. A model on a happy-path script looks like magic. Production has accents, rage, partial info, and a user who pastes an entire PDF into a phone note.
Force a “mean demo” before approval:
- missing fields
- contradictory inputs
- out-of-scope questions
- timeout and tool failure
If the feature only works when the user cooperates perfectly, it is not a feature. It is a stage play.
A lightweight no that does not make you the villain
People hear “no” as “you are wrong.” you try to say:
“Not now. Here is the prerequisite. When the KB is clean / when we have evals / when support can review outputs, we revisit.”
Put the prerequisite in writing. Then the next conversation is about readiness, not about who is anti-innovation.
Kill criteria after launch
Shipping is not the finish line. I set kill criteria up front:
- if precision on the eval set drops below X, turn it off
- if humans override more than Y percent, rethink the design
- if nobody used it in 30 days, delete it without a eulogy
Deleting unused AI is a gift to your future self. Dead features still scare auditors and confuse users.
The operator’s punchline
AI is a power tool. Power tools in the wrong room make expensive holes.
Bottom line
Your job is not to sprinkle models on every surface. Your job is to pick the few places where judgment is scarce, volume is high, and failure modes are understood. Say no everywhere else. Protect the yeses. That is how you stay the person who ships systems, not the person who collects demos.
