Vaccine effectiveness vs efficacy: how Australia measures whether a vaccine actually works

You will see two numbers quoted about a vaccine, and they are not the same thing. Efficacy is the percentage reduction in disease measured in a randomised controlled trial, under tightly controlled conditions. Effectiveness is the percentage reduction measured in the real world, in actual community use, with everything that real life brings to bear: imperfect storage, missed doses, mismatched strains, variable timing, immunocompromise, co-administered medications, age extremes. Both numbers are useful, but it is effectiveness that tells you what to expect this season.

Why the two numbers diverge

A vaccine trial recruits a defined population, often healthy adults, randomises them to vaccine or placebo, controls cold-chain meticulously, and measures specific lab-confirmed outcomes. The headline efficacy figure is typically the relative reduction in those outcomes. When that vaccine then rolls out to a community of millions, results look different. Older adults respond less well, immunocompromised people respond less well, the strain mix in the community may not match the trial, doses occasionally get warm in transit, and people sometimes turn up unvaccinated when records say otherwise. Effectiveness is what you actually get.

How Australia measures effectiveness

Test-negative case-control studies

The workhorse of influenza vaccine effectiveness research. People presenting to sentinel GP networks or hospitals with an influenza-like illness are tested for influenza. Those who test positive are cases; those who test negative are controls. Researchers then compare vaccination rates between the two groups. Because both groups self-selected to seek care, the design controls for healthcare-seeking behaviour, a major source of confounding in observational studies. The Australian Sentinel Practices Research Network (ASPREN) and FluCAN publish season-by-season estimates using this design.

Surveillance-based screening method

Used when test results are not centralised. Population vaccination coverage is compared to the proportion of disease cases who were vaccinated. If the vaccine works, vaccinated people will be under-represented among cases. The screening method is cheaper and faster than test-negative studies but more vulnerable to confounding.

Linked-data cohort studies

Australia’s linked health data, AIR plus Medicare plus hospital admissions plus death records, allows researchers to track vaccinated and unvaccinated cohorts over time. NCIRS uses linked data to evaluate effectiveness against severe outcomes like hospitalisation, where laboratory testing of every case is not feasible.

How to read an effectiveness number

• Outcome. Effectiveness against lab-confirmed disease is always higher than effectiveness against severe disease, which is higher than against hospitalisation, which is higher than against death. 40% effectiveness against medically attended influenza can correspond to 80% effectiveness against influenza-related ICU admission.
• Population. Effectiveness in children is usually higher than in older adults whose immune systems respond less briskly. Effectiveness in the immunocompromised is lower again. Headline numbers usually average across populations.
• Confidence interval. An estimate of 50% with a 95% CI of 35–65% is far more useful than a point estimate alone. A wide CI (5–80%) means we cannot say with confidence what the true effectiveness is.

Why influenza estimates vary year to year

Influenza is the canonical worked example. Effectiveness against medically attended influenza in working-age adults has ranged from around 10% in poorly matched seasons to over 60% in well-matched ones. The single biggest driver is the strain mix: WHO predicts which strains will circulate in the Southern Hemisphere based on Northern Hemisphere data, and the prediction is sometimes wrong. A secondary driver is antigenic drift, the virus mutating between when manufacturers freeze the formulation and when the season peaks.

What good effectiveness looks like

• Measles (two-dose MMR), around 99% lifetime effectiveness against measles disease.
• HPV (one dose, adolescent), over 90% against persistent infection with covered HPV types over a decade-plus.
• Pneumococcal conjugate, around 70–80% against invasive pneumococcal disease caused by covered serotypes.
• Annual influenza, 40–60% in well-matched seasons against medically attended disease.
• Influenza in older adults, often closer to 30%, which is why enhanced formulations are NIP-funded for 65+.

What this means for you

A vaccine with 40% effectiveness against symptomatic disease still prevents tens of thousands of cases when given at scale, that is the population-level reason vaccination programs are cost-effective even when effectiveness against individual symptomatic disease is moderate. Effectiveness against severe outcomes is almost always substantially higher than against any symptoms at all.