How to Use AI to Dramatically Improve Your Quality
Want to improve AI output quality so your content, reports, and deliverables don't read like everyone else's? Wondering how to use AI personas and feedback loops to consistently produce better work? In this article, you'll discover how to build...
BigThink
Want to improve AI output quality so your content, reports, and deliverables don't read like everyone else's? Wondering how to use AI personas and feedback loops to consistently produce better work?
In this article, you'll discover how to build AI personas and feedback loops that improve the quality of your deliverables before anyone else sees them.
This article was co-created by Austin Marchese and Michael Stelzner. For more about Austin, scroll to the end of this article.
Why Quality AI Content Is the New Differentiator
As AI gets better, the cost to produce something approaches zero. Anyone can generate 100 short-form content ideas instantly, but that output carries no real value on its own. AI, by its nature, produces an average output, and there's no demand for average.
The impact is significant for marketers, creators, and business owners using AI. As the quantity of AI-generated content rises, demand for quality rises in parallel because quality becomes scarcer. Everyone using these tools the same way produces the same average result.
For example, when ChatGPT first introduced image generation, everyone thought the results were remarkable. A few months later, those images all looked the same, and audiences could immediately spot AI-generated work.
The same thing happens with reports, content, and any other deliverable that goes out without human refinement. When the recipient can tell it was generated by AI, it erodes trust and perceived value. The goal isn't to use AI to do more. The goal is to use AI to improve quality across every output that matters.
The people who maintain their own thinking while using AI as a quality tool are the ones whose work stands apart.
#1: Set Up the AI Quality Control System With a Project, Skills, and a Knowledge Base
The technical setup has three layers, each building on the one before it.
Start With a Claude Project
The most accessible entry point is a Claude project. All of the data about the audience — their preferences, past feedback, and communication samples — gets uploaded directly into the project's context. Conversations within the project can then reference that data. Austin recommends it as the starting point for anyone new to the approach. He considers it roughly 80% as effective as the more advanced setups.
Develop Your Knowledge Base
For users working in Claude Cowork or Claude Code, the system gains an additional advantage: access to local files. Instead of manually uploading context into a project, all persona data lives in folders on the user's computer. A folder called “internal focus group” might contain subfolders for each persona, each holding raw transcripts, processed notes, and distilled preferences.
Austin references a knowledge base structure popularized by Andrej Karpathy that organizes information into two layers. A “raw” folder contains unprocessed data, such as call transcripts and message exports. A “wiki” folder holds AI-processed summaries and extracted learnings. When the focus group skill runs, it reads from the wiki for speed and falls back to the raw data when it needs a specific quote or detail.
To setup this structure, Austin recommends a direct prompt:
I want to make my system into an LLM knowledge base.Tell me how to do it.
The AI will examine the existing project environment and provide specific directions for organizing the files. Because the term “LLM knowledge base” is well-established in AI training data, the AI already knows the pattern and can tailor it to whatever system the user is working in.
This local file approach also creates what Austin calls the difference between “renting intelligence” and “owning intelligence.” When all the context, data, and system instructions live on the user's machine rather than inside a platform, the user owns the entire system as intellectual property. If a user switches from Claude to ChatGPT to an open-source model, the context travels with them.
Marketers Are Learning AI Alone
Sound familiar? 85% of marketers learn AI by experimenting on their own, just 7% get company training, and more than half spend their own money on tools.
Our third annual AI Marketing Industry Report surveys 681 marketers and distills everything into 32 clear charts — what's working, what platforms marketers rely on, and what to learn next.
Add Skills for Repeatable Workflows
A skill, in AI tooling terms, is a reusable prompt: a saved set of instructions that executes the same task the same way every time it's called. Instead of typing out instructions every time, a skill packages the entire workflow into a single command. For example, you could create a skill called “internal-focus-group” that takes an output, runs it past the audience personas, and returns structured feedback.
Rather than writing the skill prompt manually, Austin recommends having Claude or ChatGPT conduct an interview to define the skill's behavior. A starting prompt might be:
Interview me to create an internal focus group skill where I want to take an output, have an audience set review it, and provide me with feedback.Ask me any questions to help develop this skill, and identify things I might not be thinking of.
The AI builds the prompt based on the conversation, which consistently produces better results than writing it by hand.
Pro Tip: Austin says he never writes a prompt or skill by hand. Instead, he speaks to AI using voice input, either Claude's built-in voice feature or a tool like Wispr Flow. Voice captures nuance and detail that most people skip when typing, and the AI turns that spoken input into a structured skill.
#2: Identify Where Quality Matters
The first step is identifying which tasks are force multipliers, where going from good to great creates a meaningful difference. Not every task requires a high-quality output.
First, Austin applies the 80/20 rule to find these tasks. He finds the 20% of tasks where quality improvements produce 80% of the impact and directs all quality effort to those tasks first.
For his own work, that turned out to be YouTube video packaging: titles and thumbnails.
For a corporate professional, it might be the weekly report sent to a manager.
For a consultant, it could be the deliverables shared with clients.
The key is being deliberate about where quality effort goes rather than trying to optimize everything at once.
Once the high-impact task is clear, the next step is figuring out who sits on the receiving end. Improving quality requires feedback, and feedback requires understanding the person evaluating the output.
Which AI Moves Actually Matter?
That's what every marketer is wrestling with now. New AI strategies, new tools, new takes every week — but no clarity. Most marketers and business owners are trying to figure out AI alone.
The AI Business Society is your trusted guide. Get expert-led training you can put to work immediately. Plus a community of marketers sharing what's actually working.
#3: Build an AI Persona From Real Data
The core of this quality system is replacing slow human-to-human feedback cycles with fast human-to-AI-clone feedback cycles.
In a traditional workflow, someone creates a report, submits it to a manager, waits for feedback, revises it, resubmits it, and repeats. Each cycle takes time and costs perceived value. Every round of corrections signals that the first attempt wasn't good-enough. The AI persona approach introduces a layer of quality assurance before the work ever reaches the real recipient.
Austin explains the process: create a persona of the recipient inside the AI system, run the output past that persona, iterate as many times as needed, and only then share the final version with the actual person. The goal is to catch 80%, 90%, or even 100% of the feedback before the real human sees the work. This is how AI output quality improves: not by prompting better, but by building a feedback system that catches gaps before they reach the audience.
This isn't a fundamentally new behavior. In a pre-AI world, someone creating a report would review it themselves before sending it, looking for gaps, reconsidering phrasing, and anticipating questions. The AI persona approach augments the existing review process rather than replacing it. The workflow stays the same; the review just gets sharper.
The data behind the persona is what makes it effective. When selecting a person to clone, pick someone with whom there's either direct access or a robust historical data set.
For his YouTube video packaging system, Austin incorporated insights from an actual viewer: text-message conversations, direct feedback on video ideas, and transcripts or notes from in-person discussions.
For a manager persona, the data sources include email threads, Slack messages, call transcripts, and past feedback.
For creators and marketers, social engagement data adds another layer: comments, direct messages, and engagement patterns all serve as inputs for building a more accurate persona.
The more specific the data, the more precise the AI persona's responses.
#4: Create an Internal AI Focus Group
An internal AI focus group is a set of AI-cloned personas, each representing a different audience segment, that reviews work and provides individualized feedback before it ships.
For Austin’s YouTube video packaging quality system, the Darren persona represents one audience segment. A Johnny persona represents another. Each clone reflects a different archetype: founders, day-job builders, technical users, price-sensitive buyers, and risk-averse decision-makers.
The structure creates flexibility. A YouTube video concept gets reviewed by four personas. A client report gets reviewed by three. Each focus group can be configured for the specific output being evaluated, and the feedback from each persona is individualized.
Austin recommends formatting the focus group output as a structured chart that rates the work on a scale of 0 to 10 across all personas. This creates a consistent, comparable benchmark across iterations and content types.
The results speak for themselves. Austin implemented this system in April, and since then, his YouTube subscribers have grown 10x. He can point to the exact moment on his growth chart where the internal focus group changed the trajectory.
For those who want to go further, Austin also suggests creating a “board of advisors,” AI personas built from publicly available data on thought leaders like Seth Godin or Alex Hormozi. These serve a different purpose than the internal focus group: they provide a strategic perspective rather than audience-specific feedback. Everything covered about building personas extends to this use case as well.
#5: Iterate Until the AI Persona Matches Reality
The iterative cycle of testing, comparing, and refining is where the system produces its real value. Building the focus group is the setup. Running it repeatedly and correcting its mistakes makes it accurate.
Austin's process with Darren illustrates this clearly. He would generate a YouTube title idea, run it through the AI Darren persona for feedback, then text the real Darren to ask what he thought. If the AI persona's feedback matched the real person's feedback, the system was calibrated correctly. If it didn't, Austin would screenshot the real conversation, feed it back into the system, and update the persona data so the AI wouldn't make the same mistake again.
After about five or six iterations, the AI persona was accurate enough that Austin stopped texting Darren entirely. The real Darren eventually asked why Austin had stopped reaching out. The system had replaced the human feedback loop.
One practical detail affects how updating works. In Claude Projects, AI can update a skill or its memory, but it can't update the system instructions directly. Austin recommends structuring each persona as a separate skill within the project. That way, after a correction, the prompt is simply “update the Darren skill,” and the AI modifies that specific persona's instructions without touching anything else. This keeps each persona independently refinable.
The update process itself is simple. After a conversation reveals a gap between the AI persona and reality, a prompt like “based on this conversation, update my project so it doesn't make the same mistake again” triggers the AI to revise the underlying data. Each correction hardens the system, making it more accurate with every cycle.
Other Notes From This Episode
Connect with Michael Stelzner @Stelzner on Facebook and @Mike_Stelzner on X. Watch this interview and other exclusive content from Social Media Examiner on YouTube.Listen to the Podcast Now
This article is sourced from the AI Explored podcast. Listen or subscribe below.
Where to subscribe: Apple Podcasts | Spotify | YouTube Music | YouTube | Amazon Music | RSS
✋🏽 If you enjoyed this episode of the AI Explored podcast, please head over to Apple Podcasts, leave a rating, write a review, and subscribe.
Stay Up-to-Date: Get New Marketing Articles Delivered to You!
Don't miss out on upcoming social media marketing insights and strategies! Sign up to receive notifications when we publish new articles on Social Media Examiner. Our expertly crafted content will help you stay ahead of the curve and drive results for your business. Click the link below to sign up now and receive our annual report!
Discover Proven Marketing Strategies and Tips
Want to go even deeper with your marketing? Check out the Social Media Marketing Podcast! Publishing weekly since 2012, the Social Media Marketing Podcast helps you navigate the constantly changing marketing jungle, with expert interviews from marketing pros.
But don’t let the name fool you. This show is about a lot more than just social media marketing. With over 650 episodes and millions of downloads each year, this show has been a trusted source for marketers for well over a decade.
