The Evaluation Mistake Most Businesses Make

The typical agency selection process involves issuing an RFP, receiving proposals, sitting through presentations, and choosing the agency that most impressed you in the room. The problem: none of those steps measure the things that actually predict results. Great presenters aren't necessarily great marketers. A polished proposal with impressive-looking case studies doesn't tell you whether those results are reproducible for your specific situation.

The agencies that win most pitches are the agencies with the best pitch — not the best outcomes. And once you've signed a 12-month contract and paid a significant retainer, the agency has significantly less incentive to perform than it did to win the business. Understanding this dynamic is the first step to selecting against it.

The selection process that works inverts the usual logic: spend less time evaluating presentations and more time evaluating evidence. Require specifics where agencies offer generalities. Test their thinking before you buy their execution. And structure the contract to align incentives with your outcomes, not their revenue.

The Right Questions to Ask During Agency Pitches

Most pitch meetings are dominated by questions from the agency to you and a lot of slides from the agency about themselves. Flip this dynamic. The questions that reveal real capability: "Walk me through a campaign that didn't perform as expected and what you did about it." How an agency handles failure is more revealing than how they describe their successes. "Who specifically will work on my account day-to-day, and what's their experience?" Many agencies pitch senior talent and deliver junior execution. "What does your reporting look like, and how do you connect it to business outcomes?" Agencies that report impressions and clicks without connecting to revenue either don't have the data or don't want you to see it.

Also ask: "What information do you need from us to do this well?" Good agencies have clear input requirements. If the agency claims they can produce results without access to your analytics, your CRM data, and meaningful time from your team, they're describing a service built for efficiency, not performance.

Green Flags vs Red Flags in Agency Proposals

Green flags: Specific, numeric commitments tied to clearly defined timeframes (not "we aim to improve your rankings" but "we'll increase your top-20 keyword rankings by 40% in 6 months, measured in Semrush"). Clear account team structure with named individuals and their relevant experience. Honest acknowledgment of what the agency does not do well. A defined onboarding process that includes audit, strategy, and feedback loops before execution begins. Transparent reporting with client access to raw data, not just summarised reports.

Red flags: Guaranteed results without conditions (no ethical agency can guarantee specific Google rankings). Heavy use of proprietary metrics that only the agency can measure. Long-term contracts with no performance exit clauses. Vague deliverables ("comprehensive SEO strategy" rather than "technical audit, keyword mapping for 200 priority pages, 8 pieces of optimised content per month"). Resistance to sharing full analytics access.

How to Evaluate Case Studies Properly

Case studies are the primary evidence agencies offer — and the primary place where selection processes break down. The questions to ask about every case study presented: Was the client in the same or comparable industry, with similar competitive intensity and starting conditions? What was the time frame, and what were the conditions at the start of the engagement? What specifically did the agency do — not the outcome, the tactical actions? Can you speak to the client directly?

Reference checks with actual clients are the single most valuable step in the selection process. Not email references that the agency curated, but references you source independently — through LinkedIn, through your network, through industry communities where the agency's clients might be active. Ask those clients: what did the agency do well, what would you change, and would you re-sign if your contract ended today?

Contract Terms and Performance Guarantees to Require

The contract terms you negotiate before signing significantly affect your leverage throughout the relationship. Key terms to include: a 90-day performance review with defined success criteria and a break clause if targets are missed; clear definition of deliverables in the statement of work (not just "SEO services" but specific outputs per month); data ownership language confirming that all account access, data, and creative assets belong to you, not the agency; and a knowledge transfer requirement if the relationship ends to prevent your own data from being held hostage.

Performance-based fee structures — where a portion of the fee is tied to achieving defined outcomes — align incentives better than pure retainer models. Many agencies resist them because they carry risk. An agency confident in its capabilities should be willing to have skin in the game. Even a 20% performance component creates meaningfully different incentive dynamics than a flat retainer.

Ready to put these insights into action? Lumo’s team builds and manages AI Consulting strategies for growth-stage businesses.

Explore AI Consulting →