A CMO asked me a simple question last year. "We've spent twelve months on AI projects. Did any of them work?"
Nobody in the room could answer. Not because the projects had failed, but because nobody had written down what "worked" would mean before they started. There were demos, a few happy anecdotes and a lot of Slack messages saying "this is so cool". There were no numbers.
This is the norm, not the exception. Here's how I set projects up so the question has an answer.
Measure three things, in this order
1. Time. How many hours a week did this task take before, and how many does it take now? This is the easiest to measure and the most honest. Ask the people who do the work to estimate it before you start, then again a month after launch.
2. Quality. Did the output stay as good, or get better? Pick one or two quality signals that already exist. Error rates, approval rounds, sentiment scores, the number of leads sales rejects. If time goes down but quality drops with it, you haven't saved anything. You've moved the cost somewhere else.
3. Revenue. Did it move a number the business cares about? Conversion rate, pipeline created, cost per acquisition, return on ad spend. This is the hardest to attribute and the one leadership cares about most, so be careful and be conservative.
Most teams start with revenue, can't prove it, and give up. Start with time. It funds the next project.
Take a baseline before you build anything
The single most useful thing you can do is write down the "before" numbers before you change anything. It takes an hour. Once the new system is live, the old way of working is gone and you can't measure it any more.
My baseline doc is one page:
- What the task is and who does it.
- Hours per week, estimated by the people doing it.
- The quality signal we'll watch and its current value.
- The business metric we hope to move and its current value.
- What we'd consider a success at 30 and 90 days.
Count the hidden costs
AI projects have running costs that don't show up in the demo. Model usage, enrichment credits, tool subscriptions, and, most of all, the time someone spends maintaining it. Prompts need updating. Fields get renamed. Edge cases appear.
I add those up monthly and subtract them from the time saved. Most good projects still come out well ahead. A few don't, and it's much better to find out in month two than month twelve.
Report it the boring way
When I report on an AI project, I avoid the word "AI" in the headline. Leadership doesn't care how it works. They care what changed.
"The content team got back 10 hours a week, approval rounds dropped from three to one and sentiment scores went up" is a sentence a CFO can repeat in a board meeting. "We deployed an agentic workflow leveraging LLMs" is not.
When to switch something off
Every project gets a 90-day review with one question: would we build this again knowing what we know now? If the honest answer is no, turn it off. Teams are oddly reluctant to do this with AI work, as if switching it off means admitting the future didn't arrive. It doesn't. It means you're spending your attention where it pays.
The teams I see getting the most out of AI aren't the ones with the most projects. They're the ones who can tell you, in one sentence each, what their three projects actually did.