Monitoring · 7 min read

How to read an AI visibility notification.

Not every change is a change. What the cited, mentioned and absent transitions really mean, why a single flip is usually noise, and which notifications are worth interrupting your morning for.

· By the Askwords team

The hard part of monitoring AI answers is not detecting change. It is refusing to report most of it. These systems move on their own, constantly, and an inbox that reflects that faithfully is worthless within a week.

So the useful question when a notification arrives is never “did something change?” It is “did something change for a reason?” Everything below is about telling those apart quickly.

01 · The noise problem

Ask the same question twice and you get different brands

Ask an identical question twice on the same day, and the set of brands named typically overlaps by only about a third to a half. Nothing changed on the web in between. That is simply how sampling, retrieval and ranking interact.

Naive monitoring, run daily and mailed out on any difference, produces daily email that is mostly nothing. People unsubscribe, and then they miss the one message that mattered. The noise wastes attention, and then it destroys the channel.

A change is only worth telling you about if the system tried hard not to. Confirmation is the whole point.
02 · Confirmed vs pending

Two scans, on different days

A change earns a notification only once it has held across two scans on different calendar days. Seen once, it is pending: visible in your dashboard, never in your inbox. Seen twice, on two days, it is a confirmed change and worth telling you about.

Two details do a lot of work here, and both decide whether the second scan is a real second opinion:

A carried result is not evidence

Weekly answer engines reuse their last result on days they are not asked. Nothing was measured, so agreement with yesterday is guaranteed and meaningless. Only fresh observations count towards confirmation.

Two runs in one day are one opinion

A manual re-check right after a scheduled one is the same day’s conditions twice over. Confirmation needs two different days, not two different clicks.

The practical consequence: if you make a change on Monday, the earliest an improvement can be confirmed is Wednesday, and realistically it is weeks, because pages have to be re-crawled first.

03 · The severity ladder

Five transitions, two severities

Only moves between cited, mentioned and absent are worth sending. A move across two steps is severity 2 and worth your attention now; a one-step move is severity 1 and worth your attention this week.

Cited → absent

Severity 2

The strongest position to the weakest, in one step. Something changed on your side or a competitor’s page displaced you outright.

Do this: Investigate today. Check the page still resolves, still renders and is still crawlable, then read the new sources.

Absent → cited

Severity 2

The one you were working towards. Usually four to six weeks after a page change, which is the lag worth remembering.

Do this: Note what you shipped, and when. This is the only reliable way to learn what actually moves your category.

Cited → mentioned

Severity 1

You are still in the answer, but a different page is now the evidence. Often a competitor published something better on that exact question.

Do this: Read the new citation. It is a direct brief for what your page is missing.

Mentioned → absent

Severity 1

You dropped out of the shortlist. On a competitive category question this is normal churn; on your own brand question it is not.

Do this: Check whether the whole slate changed or only you. If only you, look for a recent change on your side.

Absent → mentioned

Severity 1

Early progress. The model now associates you with the question, but has not chosen your page as evidence.

Do this: Nothing urgent. Keep going. This is the step before a citation, not a plateau.

Which answer engine moved matters as much as the move

The same transition means different things on different engines. Losing a citation on Google AI Overviews touches far more buyers than losing one on Perplexity. Read the answer engine name before you read the severity. A severity 1 on a big answer engine can outrank a severity 2 on a small one.
04 · Reading the why

The sources that appeared and disappeared

A good notification tells you more than that you moved. It tells you which sources the answer cited before and which it cites now, and that difference is usually the whole explanation.

Your URL vanished from the sources

Look at your own site first: a deleted page, a changed URL, a new noindex, a redirect, a rendering regression. This one is nearly always self-inflicted and nearly always quick to fix.

A competitor’s new URL appeared

You were displaced by something they published. Open it. It is a specification for what the model now considers the better answer.

A third-party page swapped for another

A roundup or review site was replaced. Check whether you are on the new one. This is a mentions job, not a content job.

The sources barely changed but you dropped

The likeliest single cause of an isolated drop is ordinary model variation. Wait for confirmation before acting.

Citation churn with no move between those three is context, not news. It is worth reading when you are already investigating; it is not worth an email on its own.

05 · Ten-minute triage

What to do when one lands

  1. 1

    Read the question, not the score

    Is this a question you care about commercially? A drop on a peripheral phrasing is not the same event as a drop on your main category question.

  2. 2

    Check the answer engine

    Weight your reaction by reach. Google AI Overviews and ChatGPT carry most of the audience.

  3. 3

    Open the answer text

    Read what the model actually said. Sometimes you are absent from the list but described favorably in the prose, a very different situation from silence.

  4. 4

    Compare the source lists

    Old versus new. This normally names the cause within a minute.

  5. 5

    Ask what changed on your side

    Deploys, redirects, CDN rules, robots.txt edits, a CMS migration. Correlate the date before blaming the model.

  6. 6

    Decide: act, watch, or ignore

    Most notifications end at watch. That is a healthy ratio, not a failure of the system.

06 · What to ignore

The notifications that should not change your week

Knowing what to dismiss is most of the skill. These are the patterns that look alarming and are not:

  • A single unconfirmed flipOne observation. Wait for the second day.
  • Movement on one small answer engine onlyIf the big answer engines held steady, the buyer impact is minimal.
  • A reshuffle among competitors below youTheir order changing while your position holds is not your event.
  • Wording changes, same resultAnswers are regenerated every time. Different sentences, same outcome, no story.
  • A drop the same week you migratedAlmost certainly you. Fix the migration and re-measure before drawing conclusions about the model.

Save your urgency for the case that deserves it: a confirmed cited → absent, on a commercially important question, on an answer engine with real reach. That is the one worth dropping your morning for. Everything else is a note for the weekly review.

The takeaway

A quiet inbox is the feature.

Confirmation across days, real moves only, severity weighted by answer engine: the point of all of it is that when something does land, you read it. Notifications you trust are worth more than notifications that are complete.
Check three buyer questions free

Sources and further reading

Askwords measures a selected set of buyer questions against fresh answers from each engine. AI answers vary by model, time, context and personalization, so a check is evidence of the answers observed, not a universal judgement about a brand.