AI visibility tracker — track AI citations for ~$5–10/month

You'll help the user answer one question on a schedule: when people ask AI assistants the questions my pages answer, do the assistants cite me, and if not, who do they cite?

The stack, chosen for cost and data ownership:

• Elmo (github.com/elmohq/elmo, MIT): open-source AI-visibility platform. Dashboard, scheduler, and a Postgres database holding every answer and every cited URL. Runs locally in Docker.
• Bright Data pay-as-you-go: scrapes the real consumer surfaces (ChatGPT, Google AI Overviews, Google AI Mode) at about $1.50 per 1,000 answers.
• The user's own GA4 + Search Console data to pick the questions. This is the part that matters most.

Work through the phases in order. Tell the user what each phase is for before doing it. Phases 3 and 5 spend money, and phase 5 exposes a local service; get the user's go-ahead before each.

Phase 0 — Check prerequisites

Confirm, don't assume:

• Docker Desktop (or Docker Engine + Compose v2) installed and running: docker info.
• Python 3.9+ and Node 18+ (Node only for the optional Elmo CLI).
• Access to the site's GA4 and Search Console, as CSV exports or BigQuery.
• A card for Bright Data. Its minimum top-up is $10.

If the machine won't be on most of the time, say so now. Elmo only runs while Docker runs. A small VPS (~$6–12/mo) is the alternative, but it needs real hardening, because it's a new internet-facing server.

Phase 1 — Pick the questions (the part the tools get wrong)

Commercial tools start from ~50 "brand" prompts you invent. Start from evidence instead. Build these sets:

Why not a random or rotating sample: until you know where you show up, a site-wide citation rate is mostly zeros. On the site this was built on, only 5 of the 52 AI-referred pages were among its money pages. AI demand and Google revenue can be different maps, so check before assuming.

How pages become prompts, and what to tell the user up front. The data picks the pages. Each page then gets one prompt, and its wording comes from one of two places:

• Best: the page's top Search Console query. The Search Console interface only shows queries one page at a time, so getting them for every page needs the Search Console BigQuery bulk export (or the Search Console API). The bulk export only collects data from the day it's switched on, so if the user plans to keep doing this, tell them to turn it on now.
• Fallback: without that, you write the prompt from the page's URL and title. This works well when URLs are readable (/chipotle/gluten-free/) and poorly when they aren't (/p?id=4821). With unreadable URLs, fetch each page's title and headings, and expect to edit more of the list by hand.

Say which path they're on before building the panel, so the prompt list isn't a surprise.

Steps:

1. Get AI-referred landing pages. Read references/data-sources.md for the GA4 UI export or the BigQuery query. ChatGPT tags its links utm_source=chatgpt.com, so GA4 sees them.
2. Get page performance from Search Console (Pages tab export) and, if available, revenue or conversions per page (GA4 or the ad network). Search Console has no conversion data.
   • Optional: export the pages from Search Console's Generative AI report for the Visible set. It has no query data, so the next export is what turns those pages into prompts.
   • Optional but recommended: each page's top query (BigQuery bulk export or the Search Console API). This is how a page becomes a prompt.
3. Run scripts/build_panel.py (--ai-impressions for the Generative AI export, --queries for top queries) to merge them into panel.csv with a set column and an empty question column.
4. Write one question per page yourself (the model): the natural question a person would type into ChatGPT for which this page is the best answer. Use the page's top Search Console query when the export has it, and fetch the page title when it doesn't. Keep the same template across pages of the same type so results compare, e.g. "What are the gluten-free options at {brand}?".
5. Show the user the list and wait for approval before loading it.

Size it to the budget (Phase 2 math). ~100–130 questions is a good start.

Phase 2 — Do the cost math with the user

answers per month = questions × engines × RUNS_PER_PROMPT × (720 / DEFAULT_DELAY_HOURS)
cost ≈ answers / 1000 × $1.50

Worked example: 112 questions × 3 engines × 2 runs × 7.5 firings (every 96h) ≈ 5,000 answers ≈ $7.50/month.

• Always set RUNS_PER_PROMPT explicitly. Elmo's default is 5, which multiplies the bill by 5.
• More runs beat more questions at first. AI answers are probabilistic. With one sample per question per week (~4 a month), a page cited 10% of the time shows zero citations about two-thirds of the time. Aim for ~15 samples per question per engine per month (2 runs every 4 days).
• Frequency buys precision, not speed. Changes to your pages show up only after the engines recrawl, which takes weeks.
• Engines: pick from GA4 referrals. ChatGPT usually dominates. Add Google AI Overviews and Google AI Mode, because Google's AI surfaces don't show up as referrers at all. Skip engines that send nothing.

Phase 3 — Bright Data (spends money: confirm first)

1. The user creates a Bright Data account, adds a payment method and tops up. $10 minimum; ~$40 lasts months at the example rate.
2. They create an API key and put it in an environment file you read. Never paste keys in chat.
3. "Customer is not active" on the first request right after a top-up is normal. Wait a few minutes and retry.
4. Google AI Overviews needs a SERP zone named sdk_serp: run scripts/make_serp_zone.py. (ChatGPT and AI Mode use Bright Data's ready-made scrapers and need no zone.)
5. Run scripts/smoke_test.py: one ChatGPT answer and one AI Overview, under a cent. Show the user what came back, and whether their domain appears.

Phase 4 — Install Elmo

Use scripts/write_elmo_config.py instead of the interactive elmo init wizard. It writes the same two files (~/.elmo/.env, ~/.elmo/elmo.yaml) with these differences on purpose:

• Dashboard and Postgres bound to 127.0.0.1, so they're unreachable from the network. The wizard's default publishes Postgres (password postgres) on all interfaces.
• Telemetry off. The panel settings (engines, runs, cadence) are set up front.
• restart: unless-stopped, so it comes back whenever Docker starts.

Then docker compose -f ~/.elmo/elmo.yaml up -d and check: docker compose -f ~/.elmo/elmo.yaml logs worker should say SCRAPE_TARGETS validated and worker is ready.

Have the user open http://localhost:1515, register (their own password, never set by you), and create the brand with their domain. No model API key (OpenAI/Anthropic) is needed; that's only for Elmo's onboarding wizard, which you skip by loading questions through the API.

Phase 5 — Load the panel (starts spending: confirm first)

scripts/load_panel.py (dry run by default, then --apply). It creates one Elmo prompt per row, tagged with its set, and sets brand aliases for mention detection. It's idempotent. Then mark the brand onboarded so the full dashboard menu appears (the command is in the script's docstring). The first firing starts immediately.

After the frontier questions have run once: load_panel.py --disable-frontier.

Phase 6 — First readout (after the first firing, ~1 hour)

Run scripts/readout.py. It reports, per engine and per set:

• % of answers that searched the web. An answer from memory can't cite anyone. Read citation rates among answers that searched.
• % citing the user's domain, and right-page rate: was it the page the question targeted, or the homepage?
• Top cited domains: who wins instead. Add the real ones to Elmo as competitors so share of voice means something.
• Ranked but not cited (AI Overviews): the user's organic position on the same results page vs whether the Overview cites them. This gap is usually the most useful finding.
• Hidden AI Mode citations: Google AI Mode hides many sources behind google.com/goto redirects, which Elmo stores as google.com, so its dashboard undercounts AI Mode. The script recovers the ones whose title or text names the domain.

Frame it for the user as questions, not verdicts: the first firing is one sample, and the Proven set is biased toward pages where they're already strong.

Gotchas (read references/gotchas.md before Phase 3)

Fake "AI bot" traffic in server logs · test scripts impersonating AI crawlers · affiliate links in provider docs · Bright Data being weaker on AI Overviews than other providers (DataForSEO is the fallback) · terms-of-service gray area of scraping consumer AI apps.