The 3-Question Test for Using AI Effectively
The Wall Street Journal published a report last week that AI isn’t actually helping us work less — it’s the opposite.
In fact, according to the story, it’s making us do more distracting work, and less deep work:
“Meanwhile, the amount of time AI users devoted to focused, uninterrupted work—the kind of concentration often required for figuring out complex problems, writing formulas, creating and strategizing—fell 9%, compared with nearly no change for nonusers. ”
The article goes on to explain why that might be the case and how it could get better, but it has me thinking about something different.
We all say AI is helping us do more, and making us more productive, but how many of us are really thinking about the goal?
Understanding Why You’re Using AI
I was talking to a friend recently, and he mentioned that his favorite LLM is super good at writing marketing copy now that he’s trained it. So I asked: “Good how? good as in it’s converting customers, or good as in you just don’t have to do it.”
I didn’t really get an answer, but the point was taken.
I have another friend who struggles with shiny object syndrome (let’s be honest…many of us do), and he loves that AI is helping him create more software projects than ever before.
But he’s still struggling with the core issue he always has: he likes building, but moves on when the marketing gets hard.
Someone else told me AI has 10x’d their productivity without really explaining what that means.
It’s easy to feel like AI helps us because it does stuff we don’t want to do, or don’t feel like doing. But is it really accomplishing our goal? Because if not, you’re not actually saving time; you’re wasting it.
When I’ve used AI to write sales or marketing copy, those pages never converted. Ever.
Whenever someone says, “AI helps me accomplish more,” it’s a red flag to me. It seems like they’re not really measuring efficacy. And as Ryan Holiday said in his book, Discipline is Destiny, “The person who puts efficiency over efficacy is not very efficient.”
How to Measure If AI Actually Helps You
So how do we make sure AI is actually helping accomplish our goals, and not just creating more waste?
There’s a 3 question test:
- Does this task actually need to be performed? If it doesn’t, then neither you, nor AI should do it.
- If it needs to be performed, ask, “what is the goal of the task?” Without knowing the goal, you cannot possibly know if AI is actually helping you with it.
- Then finally, “how can I measure if and how AI is helping me with this task?”
Let’s look at a real world example, then something from our work.
Last year, Yankees Second Baseman Jazz Chisholm Jr. was struggling. He has a smooth swing, but when he was put under the white hot lights of New York sports, he fell into a trap of trying too hard — of doing too much.
He’d swing too hard to hit a home run and strike out. He’d rush throws in the field by trying to make them too fast, and make an error.
A few months into the season he said he adopted a “70%” mindset. Instead of giving 100% effort and giving up control, he’d give 70% effort and regain that control. Both his fielding and hitting improved.
He accomplished his goal by doing less, better. This is often the best approach for solopreneurs like us, too.
Now let’s look at a few examples we are likely to encounter.
Writing a Sales Page
- Does this task need to be performed? Yes
- What is the goal of the task? To get people to book a sales call
- How can I measure this goal? Am I getting at least 3 people to book sales calls each week?
Now, if you choose to let AI write the sales page, or at least help you, your metric isn’t “I didn’t have to write the sales page.” It’s “I am booking 3 calls per week.”
Posting on Social Media Every Day
- Does this task need to be performed? I would argue no, but some people will say yes.
- What is the goal of this task? Get people to sign up for my mailing list.
- How can I measure this goal? Am I getting 30 newsletter signups per week?
Like the sales page, you can have AI write the posts, or even just comments…but unless you’re getting more newsletter signups, it’s not working. And in this instance, you want to make sure it’s not having a negative effect of losing followers or engagement.
Prepping a Podcast Episode for Publish
- Does this task need to be performed? Yes, assuming your podcast is something people listen to and helps your business.
- What is the goal of this task? Have a fully transcribed episode with assets created and added to Notion for me to review.
- How can I measure this goal? Once I’m done recording, will the assets show up in Notion without me having to do anything?
This is a very real use case for me, and one I’m trying to get working with Claude Cowork (an AI automation tool I’m testing) once I’m done traveling.
The flow is:
- Record with Logic Pro and export the episode into a specific folder
- Drag and Drop the episode into MacWhisper to get a transcript
- Copy the transcript into a Claude Project to get suggested titles, show notes, and copy*
- Paste all of that into Notion
*I have a really specific prompt for this based on how I format my episodes, and my show notes.
Using Ai, so We Can Do Deep Work
This is something David Sparks calls “donkey work,” and it’s incredibly time consuming. If that’s something AI can take off my plate so I don’t have to do it, it would free up 30-45 minutes per episode. That would allow me to record an extra episode, write another newsletter, or take 2-3 sales calls.
And I think, like David, this is the real power of AI — do the “donkey work” so that you can focus on the real, important, creative work.
That deep work that saw a 9% drop in people who say they use AI to help them work better.
We just need to remember: more is not better. Faster is not better.
Better is better.
