How to Extract AI Overview Query Fan-Outs Using Screaming Frog + Gemini

Published: July 29, 2025• Updated: July 29, 2025

How to Extract AI Overview Query Fan-Outs Using Screaming Frog + Gemini featured cover image

As Google’s AI Overviews and AI Mode continue to shape the search experience, SEOs are looking for better ways to understand how these LLM-driven systems interpret content and generate related queries.

One opportunity: query fan-outs. These are the semantically related queries that AI Overviews and AI Mode surface to support or expand a response—both of which are powered by Google’s Gemini models.

We wanted a way to extract these query fan-outs for content our website to better underestand how Google will suggest related topics to what we’ve written about. So we built a workflow that grabs the H1 from a page, sends it to Google’s Grounding API that is powered by the same LLMs that power AI Overviews and AI Mode (gemini-2.5-flash and gemini-2.5-pro as time of this writing), and returns both the model’s response and the list of related queries it generated.

Here’s how you can do the same using Screaming Frog and the Gemini API.

Step 1: Get Your Gemini API Key

Start by visiting Google AI Studio. Sign in with your Google account and click “Create API Key.”

This key gives you access to Google’s Gemini models, including the ability to enable Google Search grounding. Once you have it, copy the key and keep it handy—you’ll plug it directly into a script inside Screaming Frog.

Step 2: Enable JavaScript Rendering in Screaming Frog

Head into Screaming Frog and open the configuration panel:

Go to Config → Spider → Rendering
Set the rendering mode to JavaScript

This ensures Screaming Frog loads and executes page scripts, which is important for evaluating real H1 content in the DOM.

Step 3: Create a Custom JavaScript Snippet

Now we’ll write a script that:

Extracts the H1 from the page
Sends it as a prompt to Gemini
Extracts both the AI-generated response and the grounding-based query fan-outs

Go to:

Config → Custom → Custom JavaScript
Click Add to create a new snippet
Set the Type to “Extraction”
Open the Javascript Snippet Editor.

Paste in this script, replacing 'your_api_key_here' with your actual key:

return (async () => {
  const h1Text = document.querySelector('h1')?.textContent.trim() || document.title;
  const GEMINI_API_KEY = 'your_api_key_here';
  const apiUrl = `https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash:generateContent?key=${GEMINI_API_KEY}`;

  try {
    const resp = await fetch(apiUrl, {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({
        contents: [{ parts: [{ text: h1Text }] }],
        tools: [{ google_search: {} }],
        generationConfig: { temperature: 1.0 }
      })
    });

    if (!resp.ok) {
      const txt = await resp.text();
      throw new Error(txt);
    }

    const json = await resp.json();
    const candidate = json.candidates?.[0] || {};
    const fanouts = candidate.groundingMetadata?.webSearchQueries || [];
    const response = candidate.content?.parts?.[0]?.text?.trim() || '[No response]';
    const fanoutText = fanouts.length > 0 ? fanouts.join(', ') : '[No fan-outs returned]';

    return seoSpider.data(
      `H1: ${h1Text}\n\nQuery Fan-out: ${fanoutText}\n\nResponse: ${response}`
    );
  } catch (err) {
    return seoSpider.error(err.message);
  }
})();

Step 4: Crawl Your Site

Once your snippet is saved:

Start your crawl
Screaming Frog will extract the H1 from each page
It sends that H1 to Gemini
Gemini generates a response and (if grounding is triggered) a list of related queries

You’ll see these appear in the Custom JavaScript column of your crawl results.

Warning: Depending on the size of your site, this process can be costly. Each page sends an API call to Gemini, which may count against your quota or billable usage. We recommend starting with a small set of key pages before scaling up your crawl.

Example Output

If the H1 is:

Generative Engine Optimization

Your result might look like:

H1: Generative Engine Optimization

Query Fan-out: generative SEO, LLM search optimization, AI content tuning

Response: Generative Engine Optimization (GEO) is the practice of crafting content to align with how AI systems like Gemini understand and retrieve information.

If Gemini doesn’t trigger grounding, you’ll still get a response but the fan-out will return as [No fan-outs returned].\

Why This Matters

If you’re building content to earn visibility in AI Overviews or other generative search features, it’s not enough to match exact keywords. You need to cover the full semantic space LLMs might explore. By running your own pages through this workflow, you can:

Discover what queries Gemini thinks are topically related
Analyze how the content on your page matches with those topics
Find coverage gaps across key search clusters
Improve your content to align with suggested query fan-outs

This setup gives you direct visibility into how Google’s AI Overviews and AI mode through Gemini might interpret and expand on user queries related to the content on your site. It’s lightweight, repeatable, and leverages tools many SEOs already have in their stack.