Skip to main content
NewNMC’s advertising guidelines for doctors are in force from 6 Oct 2026. See what to change, and get a free compliance auditNew NMC advertising rules for doctors: what to change
SEO

Monitoring Structured Data at Scale: Catching Schema Breaks Across 5,000 Doctor Pages

Design a scheduled crawl that extracts, normalises and diffs structured data across thousands of doctor pages, so template changes that strip markup are caught before Search Console shows it.

Monitoring Structured Data at Scale: Catching Schema Breaks Across 5,000 Doctor Pages

This post is for the technical SEO owner or engineering lead at a hospital group or doctor-directory style site, where thousands of doctor profiles share a handful of templates. Structured data on such a site rarely fails loudly. A template change drops one property, a plugin update renames a node, a consent script stops injecting JSON-LD, and nobody notices until a report weeks later shows fewer enhancements or a lost feature.

The question is how to catch those breaks early. The answer is a scheduled crawl that extracts the structured data from every sampled page, stores a normalised snapshot, and diffs it against the last good snapshot by template. We describe the design, the checks, the alert rules and the failure modes. The scale in the title is 5,000 doctor pages; the same design works for 500 or 50,000. There are no client results here, and the example is hypothetical.

Our guides on healthcare schema markup, the implementation guide and advanced medical schema explain what to mark up. This piece assumes the markup exists and asks how you keep it alive.

Why Search Console alone is not enough

Google Search Console is where you will eventually see enhancement and error reports, and you should use it. But it is a lagging, sampled view. It reports what Google processed, after it recrawled the pages, for the features it supports. It does not warn you before a release goes out, and it says nothing about types Google does not turn into features. Google's structured data introduction points to testing tools and Search Console reports for this job, which is sensible, but those are checks on one URL at a time or on Google's own timetable.

Your own monitor does three things Search Console cannot.

  1. It runs on your schedule, including against staging before release.
  2. It checks all the nodes you care about, including medical types with no Google feature.
  3. It diffs against your last known good state, so the alert reads "the Physician hospitalAffiliation disappeared from this template" and not "errors went up".

Design the extraction

You need a crawler that fetches pages, finds structured data and parses it. Three decisions shape reliability.

Raw or rendered. If your JSON-LD is in the server HTML, a plain HTTP fetch is cheaper and more stable. If it is injected by a tag manager or a client-side script, you need a headless browser to see it. Google's JavaScript SEO basics confirm that JSON-LD can be generated by JavaScript, with a warning that errors are easy to introduce. Run raw first; for pages where it finds nothing, fall back to a rendered fetch and record which mode produced the data. A shift from "raw" to "rendered only" is itself a finding.

Formats. Extract JSON-LD, and also Microdata and RDFa if any legacy plugin still emits them. A page can carry two copies of the same entity in different formats.

Politeness. Doctor pages sit on the same server patients use to book. Crawl at a low concurrency, observe response time, back off on slow responses, and run outside peak hours. Four parallel requests is usually enough for 5,000 pages overnight.

Normalise into a snapshot

Raw JSON-LD is noisy. Two renderings of the same page can differ in key order, whitespace or array order. Normalise before storing.

  • Parse every JSON-LD block. Record parse failures as findings; a block that fails to parse is invisible to every consumer.
  • Flatten @graph into a list of nodes, each with @type, @id and its properties.
  • Sort keys and normalise arrays of types.
  • Store, per URL, a compact record: template, fetch mode, status, final URL, list of node types, list of @id values, a list of property paths present per node, and a hash of each node.
  • Keep the full normalised JSON for a rolling window so a diff can be inspected.

A snapshot per run, per URL, is a few kilobytes; 5,000 pages fit comfortably in a database table or a folder of files.

What to check on each page

Write checks as rules that can pass or fail. Group them in three levels.

Structural checks

  • The page contains at least one parsable JSON-LD block.
  • Every @id referenced is defined somewhere on the page or is a known site-level identifier.
  • No @id appears twice on one page with different values.
  • No empty strings, null values or placeholder text such as "undefined" or "Lorem".

Template contract checks. For each template, define the nodes and properties it must emit. For a doctor template that might be: a Physician node with name, url, medicalSpecialty, hospitalAffiliation and image; a WebPage node that is about the Physician; and the site-level organisation reference. The contract is a short file kept with the template code.

Consistency checks. Compare markup to visible text: the doctor's name in markup equals the h1; the affiliation named in markup appears on the page; the specialty matches the visible specialty. Google's general guidelines say not to mark up content that is not visible, so a mismatch is also a policy risk.

If you use stable @id identifiers across pages, add a graph-level check: every campus @id the doctors reference has a full definition on its own page.

Diff against the last good snapshot

Checks tell you what is wrong now. Diffs tell you what changed. Compare each URL's snapshot with its last passing snapshot and classify the change.

ChangeLikely causeSeverity
Node type disappearsTemplate or plugin changeCritical
Property disappearsField renamed or removedHigh
Property value changes type (text to object)Template refactorHigh
Fetch mode changes from raw to renderedScript moved to tag managerMedium
New property appearsTemplate extendedInfo
Value changesContent editInfo, unless in a protected field
Parse failureBroken JSON, bad escapingCritical

Aggregate by template. A single page that changed is an edit. A template where hundreds of sampled pages lost the same property is a release regression. Report the share of affected pages as a plain count, "412 of 500 sampled pages in the doctor template".

Want this applied to your practice? Get a free seo audit.

Book my free audit

Alert rules that people will not ignore

An alert that fires daily for trivial reasons gets muted. Set rules like these.

  • Page now: critical, template-wide. Alert the owner of the template if more than a small fixed number of pages in one template, say ten, show the same critical change.
  • Next working day: high-severity, template-wide.
  • Weekly digest: medium and info, with counts per template and the five most common changes.
  • Silence individual-page edits unless the page is on a priority list of high-traffic profiles.

Choose the numbers for your site and write them down. The goal is that every alert names a template, a change and a probable cause, so the recipient can act in a few minutes.

A hypothetical incident

A group has 5,000 doctor profiles on one template. A release replaces the template's header component. The release is tested visually and goes live. The next night's crawl finds that doctor pages no longer carry hospitalAffiliation, because the field was read from a variable the new component does not set. Visible content is unchanged, so no editor notices.

Search Console would show nothing for weeks, since Physician has no enhancement report to flag the loss. The monitor flags "property removed" on the doctor template, with 4,998 of 5,000 sampled pages affected, and the diff shows the exact path. The developer fixes the variable the same day. Without the monitor, the doctor-to-hospital links in the graph would have been missing until someone audited by hand. The numbers are invented for the illustration.

Run it before release, not only after

Point the same crawler at a staging environment with a sample of 20 pages per template. Make the checks a gate in the release pipeline: a critical failure blocks the deployment. Then run the full crawl on production after each release and nightly or weekly according to how often content changes. Doing both means a regression is caught before it ships or, at worst, the next night.

Failure modes of the monitor itself

  • False alarms from rendering. A slow page returns without its script; the monitor reports lost markup. Retry failed pages once with a longer wait before alerting.
  • Bot protection. A challenge page produces no JSON-LD. Recognise block pages and exclude them.
  • Sampling bias. A random sample can miss a rare template. Sample every template, and keep a fixed set of canary URLs.
  • Alert fatigue. Review thresholds monthly.
  • Stale contracts. A template evolves legitimately and the contract is not updated. Make updating the contract part of the template's change process.
  • Privacy. Do not store personal data you do not need. Doctor profiles are public pages, but your snapshots should hold markup, not form submissions or session data.

For the validation tools themselves, the Schema Markup Validator and the Rich Results Test are good for spot checks. Our technical SEO audit and schema markup service pages describe how we set up this kind of scheduled check.

Where to start

Do not build everything at once. This week, pick the doctor template and write its contract in ten lines. Crawl 100 pages, extract the JSON-LD and check them against the contract by hand or with a short script. Store the result as your first snapshot. Next week, rerun and diff. Once the diff is useful, add the other templates, then the staging gate, then the alerts. A small monitor that runs every night is worth more than a large one that never gets finished.

Filed understructured data monitoringschema monitoring at scaleJSON-LD diff crawlschema markup regression testingmonitor schema markupstructured data validation hospital

About the author

Co-Founder & CTO · Gurugram, India

Arush builds the systems the rest of the company runs on — the patient-acquisition pipelines, the AI chatbots, the analytics that tie a single click to a booked appointment. If a campaign reports a number, it's because something his team built is tracking it.

Free guide · 15 pages

The Patient Acquisition Blueprint

A 90-day plan from first search to booked appointment.

The Pioneers Brief — one essay, three benchmarks, one teardown, every Thursday. Subscribe free →

Free seo audit

Your next chapter starts with a conversation.

Three quick questions. We come prepared with a plan for your specialty and city.

  • A 30-minute call with a healthcare specialist
  • Your top three growth opportunities, in writing
  • Honest advice — even if that means not hiring us
  • No obligation and no hard sell
You’ll speak with a senior strategist
Consultations are hosted by Arush Thapar’s team.
Step 1 of 3Takes 30 seconds

What do you want to improve first?

Details stay privateNo obligationReply within 1 working day

Client portfolio · Healthcare websites

Work relevant to monitoring structured data at scale catching schema breaks.

Explore client work →