Most bad AI purchases come from evaluating the demo rather than the task. Here is a process that takes a week and prevents a year of paying for something nobody uses.
Short answer
Pick one real, recurring task. Define what “good enough” looks like before you start. Run two or three candidates on that task using their free tiers, and count the rework. Buy the one with the least rework, not the best demo.
1. Start from the task, not the tool
Write down one task, precisely, with how often it happens and how long it takes. “We spend about four hours a week producing the weekly progress report” is a brief. “We should use more AI” is not.
2. Define good enough before you look at anything
What would the output have to be for you to use it without rewriting? Write that down first, or every demo will look impressive.
3. Test on your own material
Vendor demos use material chosen to flatter the tool. Run your actual documents, your actual data, your actual meeting audio. The gap between a demo and your material is where most disappointment lives.
4. Count the rework, including the review
The honest measure is time before versus time after, including the time you spend checking output. A tool that halves drafting time but doubles review time has saved you nothing.
5. Check what happens to your data
Before the first upload: retention, training use, and where the data is stored. This takes ten minutes on the vendor’s terms page and it is much harder to undo later.
6. Decide, and set a review date
Subscribe, or do not. Either way, put a date in the calendar three months out to check whether it is still being used. Unused subscriptions are the most common AI cost in a small business.
Frequently asked questions
How long should an evaluation take?
A week of real use beats a month of reading reviews, including this one.
Should I evaluate more than three tools?
Rarely. Beyond three the comparison cost exceeds the difference between them.