Skip to main content
NewNMC’s advertising guidelines for doctors are in force from 6 Oct 2026. See what to change, and get a free compliance auditNew NMC advertising rules for doctors: what to change
AI & Automation

Validating a Lead Score: Lift Charts, Calibration and the Sales Feedback Loop

How to check that a patient lead score predicts arrival and revenue: decile lift tables, calibration, leakage checks and a counsellor feedback loop, worked on synthetic data.

Validating a Lead Score: Lift Charts, Calibration and the Sales Feedback Loop

Most hospital lead scores are never checked. A vendor or an agency builds one, the CRM shows a number from 0 to 100, counsellors call the high ones first, and everyone assumes it works. Six months later nobody can say whether a lead scored 80 is actually more likely to arrive for a consultation than a lead scored 40.

This article is for marketing leads and analysts at hospitals and clinic groups who already have a score, from a model or from hand-written rules, and want to know whether it earns its place. It covers four checks: whether the score ranks leads correctly (lift), whether its numbers mean what they say (calibration), whether it predicts the outcome that matters (arrival and revenue, not form fills), and how to retrain it from counsellor feedback. Every figure below is synthetic, generated to show the arithmetic, and is not a benchmark.

Prerequisites: a CRM export with a lead ID, the score at the time of first contact, the date of the enquiry and the eventual outcome. If your system overwrites the score as the lead progresses, fix that first, because you need the score as it stood when a counsellor decided whom to call.

Decide what the score is supposed to predict

A score without a target is decoration. Choose one outcome, define it precisely and fix the window.

  • Arrival: the lead attended a first consultation within 30 days of enquiry.
  • Conversion: the lead started treatment or a package within 90 days.
  • Revenue: billed value within 90 days.

Arrival is the most useful starting point. It happens soon enough to measure, it is less affected by clinical decisions than treatment, and it is the point at which marketing spend has done its job. A score that predicts form completion, which is easy to measure, may predict nothing about who turns up.

Write the definition in the data dictionary: target_arrived_30d equals 1 if an attended visit is recorded within 30 days of enquiry_at, otherwise 0. Exclude leads whose window has not yet closed. Including recent leads makes everything look worse than it is.

Check 1: the lift chart

Lift tells you whether ranking by score puts the people who arrive nearer the top.

Method.

  1. Take all leads with a closed outcome window.
  2. Sort them by score, highest first.
  3. Split into ten equal groups (deciles) of leads.
  4. For each decile, calculate the arrival rate: arrivals divided by leads.
  5. Calculate lift as the decile's arrival rate divided by the overall arrival rate.

Synthetic example. Suppose 2,000 leads, of whom 500 arrived, an overall arrival rate of 0.25.

DecileLeadsArrivalsArrival rateLift
1 (top scores)200880.441.76
2200720.361.44
3200620.311.24
4200540.271.08
5200480.240.96
6200400.200.80
7200360.180.72
8200380.190.76
9200340.170.68
10200280.140.56

Read this table in three ways. First, the top decile has a lift of 1.76, so the score is doing something. Second, the arrival rates fall almost steadily, which is what you want. The slight rise from decile 7 to 8 is likely noise at this sample size. Third, even the bottom decile arrives at 0.14, so low-scoring leads still convert. This matters for policy: do not stop contacting them, just contact them differently.

Plot cumulative arrivals captured against cumulative leads contacted. If calling only the top three deciles in this example captures 222 of the 500 arrivals, you can say that a team with capacity for 30 per cent of leads would reach 44 per cent of those who would have turned up, compared with 30 per cent under random calling. Convert to counts when you present it, so nobody mistakes it for a promise.

Sample size. With under a few hundred closed leads, deciles will be jagged. Use quintiles, or pool several months. Compute a confidence interval for each band, or at minimum show the counts, so a reader can see how thin the data is.

Check 2: calibration

Lift cares only about order. Calibration cares about the numbers. If your CRM tells a counsellor "probability of arrival: 0.70", then roughly seven in ten such leads should arrive. A score that ranks well but is poorly calibrated misleads anyone who forecasts bookings, sets staffing or decides how much to spend per lead.

Method. Bin leads by predicted probability, for example 0 to 0.1, 0.1 to 0.2 and so on. For each bin, compare the mean predicted probability to the observed arrival rate. Plot predicted on the x-axis and observed on the y-axis. A well-calibrated model lies close to the diagonal.

The scikit-learn documentation on probability calibration explains reliability diagrams and shows how different model types distort probabilities: logistic regression is usually reasonably calibrated, while random forests tend to avoid extreme values and naive Bayes pushes toward 0 and 1. It also notes the Brier score mixes calibration with discrimination, so a lower Brier score does not by itself prove better calibration. Use the reliability diagram as your primary check.

Synthetic example.

Predicted binLeadsMean predictedObserved arrivalReading
0.0 to 0.27000.100.12Slightly under-confident
0.2 to 0.46000.290.24Slightly over-confident
0.4 to 0.64000.500.34Over-confident
0.6 to 0.82200.690.45Over-confident
0.8 to 1.0800.880.46Strongly over-confident

(The bins add up to the same 2,000 leads and 500 arrivals as the lift table.)

This score ranks well but promises too much at the top. A forecast built on its face values would overstate arrivals. The remedy is calibration: fit a mapping from the raw score to observed rates on data the model was not trained on. The same page describes two options. Sigmoid (Platt) scaling is a simple monotone transform that works better with small samples, and isotonic regression is more flexible but needs more data and can alter the ranking by creating ties. For a hospital with a few thousand leads, start with the sigmoid, check the diagram again and move to isotonic only once you have more history.

If the score is a hand-built points system, treat the points as a ranking and calibrate by mapping each score band to its observed rate. That is all calibration is, and it makes forecasts honest.

Want this applied to your practice? Get a free ai & automation audit.

Book my free audit

Check 3: does it predict what you are paid for?

Arrival is a proxy. Check it against the downstream outcomes.

  • Repeat the lift table with target_converted_90d and revenue_90d. A score may predict arrival well and conversion poorly, which suggests it is rewarding cheap curiosity, such as people who click a lot but never commit.
  • Break the table out by source and specialty. A score tuned on dental enquiries can mislead for oncology. If lift disappears within a segment, build or tune a separate score.
  • Look for leakage. If a feature is recorded after the counsellor's first call, such as "booked appointment" or "call duration", the score is peeking at the answer. Strip any feature unavailable at the moment you would use the score.
  • Check stability over time. Compute lift per month. A score whose top-decile lift swings widely is fragile, perhaps because campaigns changed the lead mix.

For a privacy check, remember that outcome and symptom fields are personal data. The DPDP Rules 2025 start phasing in obligations, with most due from May 2027, per this summary of the notification. Keep the validation dataset pseudonymised, limit who can see it and take advice on retention.

Check 4: close the loop with counsellor feedback

Models learn from outcomes, but counsellors know things the data does not: a caller who sounded unwell, a patient whose family was already booked elsewhere, a lead who was clearly a student doing research. Capture this in a structured way.

Add three fields to the CRM:

  • counsellor_rating: hot, warm, cold, set after the first conversation
  • reason_code: a short list such as wrongspecialty, priceshopping, notdecisionmaker, insuranceblocked, outofarea, noresponse
  • override_flag: yes if the counsellor ignored the score's suggested priority

Then run a monthly review.

  1. Disagreement table. Cross the score band with the counsellor rating. Look at cells where the score is high and the rating cold, and the reverse.
  2. Who was right? For each disagreement cell, compute the actual arrival rate. If counsellors outperform the score in some cell, find out why and consider adding the reason as a feature or a rule.
  3. Reason-code audit. If "outofarea" is frequent among high scorers, add distance as a feature. If "insurance_blocked" is, add a pre-check question to the form.

Retraining

When retraining, respect time. Train on older leads and test on the most recent ones, never a random split, because a random split lets the model see the future. Keep a versioned record: score_version, training window, features, the lift and calibration tables on the holdout, and the date of approval. Run the new version alongside the old for a few weeks and compare on the same leads before switching. A change of score definition mid-year breaks trend reports, so mark the cut-over date in your dashboards.

Retrain on a schedule, quarterly for most practices, and also when something changes materially, such as a new campaign channel, a new specialty line, or a price change.

What the checks cannot tell you

A score can rank well and still be bad policy. Ranking by predicted arrival will favour patients similar to those who arrived before, which may under-serve groups the practice would like to reach. Review the lift table by region and language to see whether lower-scoring groups are being deprioritised unfairly. And never let a score decide anything clinical. It sets the order of callbacks for sales and service, nothing more.

Where to start

  1. Define the target and window in writing.
  2. Export score-at-time-of-contact, enquiry date, source, specialty and outcome for every closed lead.
  3. Produce the decile lift table and a reliability diagram.
  4. Calibrate if the diagram is off the diagonal; check for leakage if lift looks too good.
  5. Add counsellor rating and reason code, and review disagreements monthly.
  6. Document the version, then retrain on a schedule.

If you want enquiries, owners and outcomes in one place, Pioneer CRM offers a walkthrough for that. For the basics, read the lead scoring entry, our piece on AI lead scoring for healthcare practices and the AI lead scoring service page.

Filed underlead score validation lift calibrationlead scoring model validationcalibration curve lead scorehospital lead scoring accuracyretrain lead score counsellor feedback

About the author

Co-Founder & CTO · Gurugram, India

Arush builds the systems the rest of the company runs on — the patient-acquisition pipelines, the AI chatbots, the analytics that tie a single click to a booked appointment. If a campaign reports a number, it's because something his team built is tracking it.

Free guide · 15 pages

The Patient Acquisition Blueprint

A 90-day plan from first search to booked appointment.

The Pioneers Brief — one essay, three benchmarks, one teardown, every Thursday. Subscribe free →

Free ai & automation audit

Your next chapter starts with a conversation.

Three quick questions. We come prepared with a plan for your specialty and city.

  • A 30-minute call with a healthcare specialist
  • Your top three growth opportunities, in writing
  • Honest advice — even if that means not hiring us
  • No obligation and no hard sell
You’ll speak with a senior strategist
Consultations are hosted by Arush Thapar’s team.
Step 1 of 3Takes 30 seconds

What do you want to improve first?

Details stay privateNo obligationReply within 1 working day

Client portfolio · Healthcare websites

Work relevant to validating a lead score lift charts calibration and the sales.

Explore client work →