What KPIs Should You Actually Hold Your Marketing Agency Accountable To?
Mike Camarda
After seeing plenty of brand leaders walk into their first meeting carrying a stack of monthly reports from their previous agency, you realize it's more of a trend than a coincidence. Beautiful decks. Rising charts. Impressions in the millions. And when asked the one question that matters, "did any of this move revenue?", the room usually goes quiet.
That silence is the whole problem.
Marketing agencies are easy to hire and hard to hold accountable. The good ones make themselves accountable on day one. The rest bring up metrics that look great in a slide but never show up in your bank account. If you're paying an agency and you can't answer, with confidence, whether they're profitable for your business, you're not working with a partner.
Here's the honest breakdown of what KPIs actually matter, why they matter, and how to structure the conversation before you sign another retainer.
Why do so many agency reports focus on the wrong metrics?
Because vanity metrics are safe. They're easy to grow, easy to visualize, and almost impossible to argue with. Impressions went up. Followers went up. Engagement rate stayed strong. On paper, everything is working.
Unfortunately, a recent study shows that 36% of CFOs list the use of vanity metrics by marketing leaders as a top concern. That's exactly why marketing budgets get cut when the economy tightens. If your agency can't tie its work to revenue, someone in your finance meeting is quietly making that case for them.
Vanity metrics exist because they're simple and easy to understand reporting. Real reporting takes actual attribution work. Guess which one most agencies default to?
What KPIs actually prove a marketing agency is worth the retainer?
The list is shorter than you'd think. Ignore the 40-metric dashboard. The KPIs that matter are the ones that tie marketing efforts directly to business outcomes. That typically includes acquisition efficiency, customer quality, return on ad spend, retention, and other business-specific metrics that align with your growth goals.
How should you measure acquisition performance with an agency?
Every business measures acquisition success differently. The important question isn't whether you're measuring CAC a particular way. It's whether the agency is consistently delivering customers at or below the efficiency targets your business has established.
A good agency should understand your business's economics and optimize toward the metrics that actually drive profitable growth. That may include:
Customer Acquisition Cost (CAC)
Return on Ad Spend (ROAS)
Cost per Subscription
Cost per Purchase
Cost per Qualified Lead
Trial-to-Subscription Rate
Incremental Cost per Acquisition
The right KPI depends on the business, not the agency
What role does customer quality play in agency accountability?
Driving more customers only matters if they're the right customers.
Depending on the business, quality may be measured through customer lifetime value (LTV), repeat purchase rate, subscription retention, activation rate, or another downstream metric.
LTV can provide valuable insight into the long-term value of the customers your marketing efforts are bringing in, helping brands understand whether acquisition strategies are driving sustainable growth rather than just short-term volume.
A strong agency should understand which downstream metrics matter most to your business and optimize acquisition efforts accordingly, rather than simply chasing the lowest possible acquisition cost.
How should you evaluate ROAS with an agency?
Return on Ad Spend (ROAS) answers a simple question: For every dollar spent on marketing, how much revenue was generated?
For many businesses, it's one of the clearest ways to evaluate the efficiency of marketing spend and its impact on revenue. While ROAS isn't the primary KPI for every company, it's an important metric for evaluating the efficiency of your marketing spend.
A good agency should understand the ROAS targets that matter to your business, monitor performance closely, explain what's driving changes over time, and proactively recommend ways to improve results.
Like any metric, ROAS shouldn't be viewed in isolation. When considered alongside customer acquisition cost, customer quality, retention, and other business-specific KPIs, it provides a more complete picture of marketing performance.
How should you evaluate agency performance when conditions change?
Marketing performance is influenced by more than your agency's work. Product pricing, website conversion rates, seasonality, inventory, competitive activity, and broader economic conditions can all impact acquisition results.
That doesn't mean agencies shouldn't be held accountable. It means they should be evaluated on how they respond.
A strong agency communicates what's changing, explains how those changes are affecting performance, and adjusts strategy accordingly. They don't use external factors as excuses, but they don't ignore them either.
The best agencies go beyond sending reports. They explain what happened, why it happened, what they're testing next, and how those changes support your business goals. That's what strategic accountability looks like.
Why is retention a KPI you should hold your agency accountable to?
Acquisition metrics only tell part of the story. Driving customers at an efficient cost is important, but it only creates long-term value if those customers stay engaged, continue purchasing, or remain subscribed.
That's why retention is another important measure of marketing quality.
Your customer retention. If your agency is only measured on acquisition, they'll happily bring in low-quality customers who churn in 60 days. A strong partner cares about the quality of the customer they deliver, not just the volume. Ask them how their targeting decisions affect retention downstream. If they don't know, they're not looking.
The agency's own client retention. This one gets missed. According to the AgencyAnalytics 2024 Benchmarks Report, agencies typically retain clients for 2 to 5 years, with 26.7% keeping them longer than five. That's the baseline. If an agency can't tell you their average client tenure or their churn rate, ask why. This tells you everything about how deeply an agency integrates with its clients versus how quickly they get swapped out.
How often should your agency report on these KPIs?
Weekly for tactical performance, such as spend, ROAS, and acquisition metrics by campaign. Monthly for broader business KPIs like customer quality, retention trends, and overall acquisition efficiency. Quarterly for a strategic business review that evaluates whether your marketing strategy is still aligned with your goals.
Reporting shouldn't stop at the numbers. A good agency explains what changed, why it changed, what's being tested next, and how those decisions support your business goals.
The role of the agency isn't to decide which metric matters most. It's to understand your business objectives, align on the right KPIs, and consistently optimize performance against them.
What are the red flags in agency reporting?
A few patterns to be aware of if you notice your agency doing:
Reports that lead with impressions, reach, or engagement rate as the headline metric
No mention of CAC, LTV, or profitability anywhere in the deck
Rising traffic paired with flat revenue and no real explanation
No comparison to a baseline or the previous period
Attribution windows that conveniently favor the agency's channels
If you're seeing celebration without a clear connection to business outcomes, ask the harder question.
How do you set up the KPI conversation with a new agency?
Before you sign, get four things on paper:
What KPIs will we be measured on, and who decides them? If they hand you the list, that's a problem. It should be co-created.
What does success look like at 90 days, 6 months, and 12 months? Vague answers here become vague answers later.
How is attribution being handled across paid, organic, and offline? This is where a lot of "great" campaigns turn out to be double-counted.
What happens if we're not hitting our KPIs? A good agency has an answer. A weak one changes the subject.
These are questions that make the difference between conversations down the road being about how do we extend rather than having to go through the vetting process yet another time.
The bottom line
The best agency relationships don't succeed because every month is perfect. They succeed because both sides agree on what success looks like, measure the right KPIs, communicate openly, and continuously optimize toward shared business goals.
If your current agency is showing you activity instead of achievement, that's your signal to reset the conversation.
At Klay Media, that's exactly how we approach every client partnership. We align on the KPIs that matter most to your business, communicate transparently, and continuously optimize toward long-term, sustainable growth. If you're rethinking how you evaluate your marketing agency, these are the conversations worth having before you sign your next agreement.