Keyword Research When Search Volume Data Is Dying

Search volume numbers from keyword tools are increasingly unreliable because they can't capture conversational AI search queries, and long-tail, natural-language searches now make up a growing share of real intent. The practical replacement isn't a new tool, it's a shift to topic and entity-based research, mining real questions from forums and customer conversations, and prioritizing by business relevance instead of a volume number that was always an estimate anyway.

Why Keyword Volume Data Was Already Shaky, and Is Getting Worse

Keyword volume numbers from tools like Ahrefs or Semrush were always modeled estimates, not ground truth, pulled from clickstream panels and sampled data, then extrapolated. That was a manageable limitation when most search behavior funneled through a handful of predictable query patterns. Two things have broken that assumption:

1. AI search interfaces (AI Overviews, AI Mode, ChatGPT, Perplexity) capture a growing share of query volume that never shows up in traditional keyword tools, because those tools are built on search engine clickstream and API data, not on conversational AI platforms.
2. Query patterns themselves have shifted toward longer, more natural, conversational phrasing, "best CRM for a 10-person agency that already uses HubSpot" instead of "best crm software", and long-tail conversational queries are exactly the ones volume tools underrepresent, since each individual phrasing gets vanishingly small measured volume even though the underlying intent is common.

The practical effect: a keyword showing "10 searches/month" in a tool might represent an intent that's actually searched hundreds of times across dozens of phrasings, you're just not seeing the aggregate because volume tools count exact-match strings, not the topic underneath them.

The Replacement Methodology: Topic and Entity-First Research

Instead of starting with a keyword list, start with the topic and the entities (people, products, concepts, named things) connected to it, then work outward to the specific phrasings people use. This mirrors, not coincidentally, how modern search engines and LLMs themselves model relevance, through entities and their relationships, not exact keyword strings. Google's documentation on how Search works has described entity understanding as core to ranking for years now; the keyword-research process should catch up to that same model.

Step 1: Map the topic, not the keyword

For any content area, list the core entities involved, for an edtech brand, that might mean specific technologies (Python, React), job roles (data analyst, backend developer), and outcomes (placement, salary hike, career switch). Your content plan should cover the entity relationships, not chase individual keyword variants of the same entity.

Step 2: Mine real questions from where people actually ask them

• Reddit and niche forums, search your topic directly on Reddit and read actual threads; the phrasing people use in a genuine question is far closer to real AI-search query patterns than anything a keyword tool will surface.
• People Also Ask and related searches on the actual SERP, still one of the best free, real-time signals of adjacent intent, even as its role shifts.
• Customer support tickets, sales call transcripts, community Slack/Discord questions, this is first-party data no competitor has access to, and it reflects exactly the language your actual buyers use.
• Quora and product review sites for anything comparison or decision-stage related.

Step 3: Prioritize by business relevance, not volume

With volume less trustworthy as a standalone signal, weight these instead:

• Proximity to a buying decision, does answering this question move someone toward a real business outcome (signup, application, purchase)?
• Competitive gap, is this a question your competitors are answering poorly or not at all, regardless of its estimated volume?
• Repeat frequency in first-party sources, if the same question shows up repeatedly in sales calls or support tickets, that's a stronger real-world demand signal than a keyword tool's estimate.
• Durability, is this a question people will keep asking for years, or a fad phrasing tied to a moment?

Where Traditional Keyword Tools Still Help

This isn't an argument to abandon Ahrefs or Semrush, they're still useful for:

• Directional comparison between two known topics (is "X" more searched than "Y," even if the absolute numbers are soft).
• Competitive gap analysis, what keywords competitors rank for that you don't, which is a relative signal, not an absolute-volume one.
• SERP feature tracking, whether a query currently triggers an AI Overview, featured snippet, or other SERP element, which increasingly matters more than the raw volume number for deciding whether a page is worth the effort.

Use them as one directional input among several, not the sole prioritization mechanism.

Building an Entity Map: A Worked Example

Take an edtech brand teaching data analytics. A traditional keyword-first process would start with "data analytics course," pull its estimated volume, and branch into modifiers (city, price, duration). An entity-first process starts differently: map out the core entities, the tools taught (Excel, SQL, Python, Power BI), the roles the course leads to (data analyst, business analyst, reporting analyst), the outcomes candidates care about (salary range, placement rate, career switch feasibility), and the comparison entities (bootcamp vs. degree, self-taught vs. structured course).

From that entity map, the content plan writes itself: pieces on each tool's role in the analyst job market, pieces comparing career paths, pieces addressing the specific anxieties around career switching. Crucially, many of these topics will have thin or nonexistent volume data in a keyword tool, because the actual demand is distributed across dozens of conversational phrasings ("is it too late to switch to data analytics at 30," "do I need a degree for data analyst jobs") that a volume tool undercounts individually but that collectively represent a large, durable audience.

Signals That Matter More Than Volume in an AI Search World

A few signals worth tracking that traditional keyword research mostly ignored:

• Whether a query currently triggers an AI Overview or AI Mode response, and how comprehensive that response is, a shallow AI answer still leaves room for a click to a more detailed resource; a comprehensive one leaves much less.
• Question complexity, genuinely multi-step or personalized questions ("which is better for someone with my background") are harder for AI answers to fully resolve and more likely to still drive a click to expert content.
• Brand and product mentions inside AI-generated answers, track whether your brand shows up as a cited source in AI Overviews or chatbot answers for your core topics; this is becoming a real, if hard-to-measure, proxy for topical authority that traditional rank tracking doesn't capture at all.
• Direct traffic and branded search growth, if entity-first content is working, you should see a rise in people searching your brand name directly after encountering your content elsewhere, even if the original discovery query never shows meaningful "volume."

A Practical Weekly Research Habit

Rather than a big quarterly keyword research project, build a lightweight ongoing habit:

1. Weekly: Scan 2-3 relevant subreddits or forums for new question patterns in your space.
2. Weekly: Pull any recurring questions from that week's sales calls or support tickets (a quick tag in your CRM or helpdesk works fine).
3. Monthly: Cross-reference accumulated questions against current SERP behavior, which ones trigger AI Overviews, which ones still show a clean organic opportunity.
4. Quarterly: Roll findings into the content calendar, prioritized by business relevance score, not raw volume.

FAQ

Are keyword volume numbers completely useless now? No, they're still a useful directional signal for comparing topics against each other, but they should be one input among several rather than the primary prioritization metric.

What's the best free source for real search intent data? People Also Ask boxes, Reddit threads, and your own first-party customer conversations (support tickets, sales calls) are the strongest free signals available right now.

How do I prioritize content topics without reliable volume data? Score topics on proximity to a buying decision, competitive gap, how often the question recurs in first-party sources, and durability, then rank by combined score rather than by volume alone.

Do AI Overviews mean certain keywords aren't worth targeting anymore? Not necessarily worth abandoning, but worth re-evaluating, check whether the specific query currently triggers an AI Overview and, if so, whether ranking for it still drives meaningful clicks before investing heavily.

Should small teams still use paid keyword research tools? Yes, for competitive gap analysis and directional comparisons, but pair them with free first-party and forum-based research rather than relying on volume numbers alone.

If your keyword research process still starts and ends with a volume spreadsheet, it's worth rethinking the input mix, happy to talk through what's worked for the edtech and startup teams I work with at younusfardeen.com.