Last spring someone tried to sell me an AI agent that would “replace your tier-one help desk by Q3.” I asked how it handles a user who swears their password is correct when it isn’t. The demo account didn’t have that user. Funny how demos work.
Some tools are useful. Many demos are theater. After nearly three decades helping businesses make technology decisions, our AI consulting starts with a different goal: keep clients from expensive mistakes rather than accelerating a signature.
Start with a problem, not a product
Before you sit through a demo, write the business problem in one paragraph: who’s affected, how you measure success today, and what better would look like in ninety days in numbers a skeptic would accept. If the vendor can’t map their product to that paragraph without inventing metrics, you’re looking at a solution hunting a budget. Product-first evaluation is how organizations get sold capabilities they never operationalize. Problem-first is how you avoid funding a story. I’ll admit writing the paragraph is annoying. It’s also the cheapest work on the whole project.
Demand an exit before you fall in love
Lock-in hides in data formats, proprietary workflows, multi-year terms, and services only the original vendor can perform. Ask in writing how you export data and prompts in a form you can actually use elsewhere, what happens to custom automations if you leave, which fees are mandatory after year one, and who owns fine-tunes, configs, and conversation history. Vague answers mean the contract is designed around retention. Independent advice exists to surface that before legal becomes a cleanup crew. I’ve watched companies stay with mediocre tools for years because leaving felt harder than complaining. Don’t design that trap for yourself on day one.
Pilot with success criteria and a kill switch
A pilot without criteria is a long sales cycle. Define a time box, a budget cap, a single owner, and two or three measurable outcomes. If it fails, stop. “Give it another quarter” is how pilots become permanent without ever becoming good. Also separate the tool from the process, because an agent that drafts tickets still needs policy, supervision, and escalation. Rushing automation into production without governance is one of the most reliable ways to create bad technology decisions in the AI era. We once ran a pilot where the only “success” metric was executive enthusiasm. We killed it. The tool wasn’t evil. The measurement was.
Hype language is a tell
Be careful with “autonomous,” “replace your team,” and “no training required.” Real environments are messy: permissions, exceptions, compliance, half-documented workflows. Tools that ignore the mess fail after the pilot, usually after the discount period ends. Ask who supports the edge cases and what the tool does when it’s wrong. If the answer is a shrug and a smile, keep your wallet closed.
What we actually do in AI consulting
We help clients evaluate, select, implement, and govern—use-case fit, vendor-neutral comparison, permission design, human review, and contract language that doesn’t paint you into a corner. It’s not glamorous. It works. Green Orb’s role isn’t to reject AI. It’s to help you use it with your eyes open so you capture upside without funding someone else’s quota. If you want a second set of independent eyes on a shortlist, that’s the work. Bring the paragraph. We’ll take it from there.
Last note: involve the people who will live with the tool, not only the people who enjoy demos. The quiet admin who knows every exception will find the failure modes faster than the strategy deck. Listen to them early. They’re usually right, and they’re rarely invited.