Adversarial Design: Six Principles for Gaming-Resistant Systems

Module 8: Adversarial and Malicious Behavior Depth: Field Guide | Target: ~1,200 words

Thesis: Systems can be designed to resist gaming by separating measurement from incentive, using composite metrics, building audit trails, and red-teaming before deployment.

The Design Problem

Every metric in healthcare creates a target. Goodhart’s Law — “when a measure becomes a target, it ceases to be a good measure” — is not a curiosity. It is a design constraint. Campbell (1979) stated it more precisely: the more a quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures, and the more apt it will be to distort and corrupt the social processes it was intended to monitor. Bevan and Hood (2006) documented exactly this in the UK’s National Health Service, where performance targets for emergency department wait times produced systematic gaming — patients held in ambulances to stop the clock, trolleys reclassified as beds, definitions of “admission” stretched until they broke.

The question is not whether metrics will be gamed. They will. The question is whether the system was designed to make gaming difficult, detectable, and costly — or whether it was designed as if everyone would cooperate in good faith. Jerry Muller’s The Tyranny of Metrics (2018) argues that metric fixation — the belief that measurement and transparent reporting will inevitably improve performance — is itself a form of institutional naivete. The remedy is not abandoning measurement. It is designing measurement systems that anticipate adversarial behavior.

Six design principles, applied together, produce gaming-resistant systems.

The Six Principles

1. Separate Measurement from Incentive

The people measuring performance must not be the same people whose compensation depends on the results. This is the most basic structural control and the one most frequently violated. When a physician practice reports its own quality scores to a payer who adjusts reimbursement based on those scores, the measurement function and the incentive function are fused. The predictable result is not fraud in the criminal sense — it is motivated reasoning applied to gray-area coding decisions, documentation choices, and patient panel composition.

The mechanism is not dishonesty. It is what psychologists call motivated cognition (Kunda, 1990): when a decision-maker has a stake in the outcome, their judgment shifts unconsciously toward the favorable interpretation. Separating measurement from incentive removes the psychological pressure that makes motivated cognition inevitable. External chart audits, blinded data collection, and third-party measurement are structural implementations of this principle.

2. Use Composite Metrics

Single metrics are easy to game because they have a single optimization axis. A readmission rate target can be gamed by reclassifying readmissions as new admissions, extending index stays past the measurement window, or diverting returning patients to observation status. Each of these manipulations exploits the same vulnerability: a unidimensional measure with clear boundary conditions that can be worked around.

Composite metrics require gaming multiple dimensions simultaneously. A composite that weights readmission rate, total cost of care, patient-reported outcomes, and functional status at 90 days is harder to game because improving the appearance of one component without affecting the others is difficult. The gaming cost — the effort required to manipulate all components in a coordinated way — rises nonlinearly with the number of independent dimensions. This does not make gaming impossible. It makes gaming expensive enough that genuine performance improvement becomes the easier path.

3. Measure Outcomes, Not Just Processes

Process metrics measure whether a step was completed: was the screening performed, was the form signed, was the care plan documented. They are easy to game because completion can be documented without substance. A depression screening checkbox can be marked without a meaningful clinical assessment. A care coordination note can be templated and signed without a conversation occurring.

Outcome metrics — readmission rates, functional status changes, mortality, patient-reported health — are harder to game because they measure what happened to the patient, not what the clinician documented. You can fabricate a checklist entry. You cannot fabricate a patient not being readmitted. The design principle is not to abandon process metrics — they have legitimate quality improvement uses — but to ensure that the metrics attached to incentives are weighted toward outcomes that are independently verifiable.

4. Build Audit Trails

Gaming becomes rational when it is undetectable. Audit trails change the calculus by making manipulation discoverable after the fact, even if it is not prevented in real time. The relevant design choices are: timestamp all data entries with user identity. Log edits, deletions, and amendments. Record the original values alongside corrections. Make audit data immutable — stored in a system that the people being measured cannot alter.

Bevan and Hood (2006) found that UK NHS gaming persisted precisely because the measurement systems lacked the granular audit data needed to distinguish genuine improvement from definitional manipulation. When audit trails exist and are known to exist, the deterrence effect operates even before an audit is conducted. The behavioral mechanism is the same as that in tax compliance research: perceived detection probability drives compliance more effectively than penalty severity (Allingham and Sandmo, 1972).

5. Red-Team Before Deployment

Before any metric goes live with incentives attached, a team should be tasked with gaming it. The instruction is specific: “Here is the metric definition, the data source, the scoring methodology, and the incentive structure. You have two weeks. Find every way to improve the score without improving the outcome it is supposed to measure.”

If the red team succeeds — and they usually do on the first pass — the metric is redesigned before deployment, not after gaming is discovered in production. This is the adversarial testing principle from cybersecurity applied to incentive design. It is cheap to run (a small team, a bounded timeframe) relative to the cost of deploying a gameable metric at scale and discovering the gaming years later in evaluation data that can no longer distinguish real improvement from measurement artifact.

6. Sunset and Rotate Metrics

Campbell’s Law has a temporal dimension: gaming sophistication increases with metric tenure. The longer a metric is in place, the more thoroughly the optimization strategies are developed, shared, and institutionalized. Annual metric rotation — replacing a subset of the metric portfolio each year — prevents entrenchment. Rotation also maintains the signal value of metrics that have not yet been optimized against. The design constraint is that rotation must be announced in advance (to allow operational preparation) but the specific replacement metrics should not be telegraphed far enough ahead to allow pre-positioning.

Quick-Reference: Common Healthcare Metrics and Gaming Vulnerabilities

Healthcare Example: Redesigning a Value-Based Contract

A 340-bed regional health system enters a value-based care contract with its largest commercial payer. The original metric set: 30-day readmission rate, ED utilization per 1,000, and patient satisfaction (HCAHPS top-box). Within 18 months, the health system’s reported performance improves substantially — but the payer’s actuarial analysis shows no change in total cost of care. Investigation reveals the optimization strategies: readmissions reclassified as observation stays, ED visits shifted to urgent care sites excluded from the denominator, and patient satisfaction scores boosted through pre-survey coaching scripts.

The redesigned contract applies all six principles. Measurement is separated from incentive: a third-party clinical auditor conducts blinded chart reviews on a random sample of cases that triggered or avoided quality events. Composite scoring replaces single metrics: the new score weights readmissions (20%), total cost of care (30%), patient-reported outcomes at 90 days (25%), and care gap closure verified by claims data (25%). Outcomes dominate: three of four components measure what happened to the patient, not what was documented. Audit trails are built into the contract: all clinical documentation edits within 48 hours of a quality event are flagged for review. Red-teaming was performed pre-launch: the payer’s analytics team spent three weeks attempting to game the new metric set and identified two vulnerabilities — one in the episode grouper logic and one in the patient-reported outcome collection timing — both corrected before go-live. Metric rotation is contractual: 20% of the composite components are replaced annually, with replacement metrics disclosed 90 days before the new performance year.

After 24 months under the redesigned contract, total cost of care declines 6.2% — a real actuarial change, not a measurement artifact. The health system’s leadership reports that the redesigned metrics made genuine care improvement easier than gaming, which is precisely the design objective.

Warning Signs

• A single metric drives a large incentive. The ratio of incentive dollars to metric dimensions is the simplest predictor of gaming pressure. High dollars, few metrics, guaranteed gaming.
• The measured entity controls the measurement process. Self-reported quality data with no external validation is an invitation to motivated cognition, not a measurement system.
• Performance improves on the metric but not on the outcome it represents. Readmission rates improve but total cost does not change. Screening rates rise but disease detection does not. The metric and the outcome have decoupled — the definition of successful gaming.
• Metrics have been unchanged for three or more years. Institutional expertise in optimizing the metric has had time to develop, spread, and become standard operating procedure.
• No one has tried to game the metric before deployment. If the design team cannot describe how the metric could be gamed, they have not thought about it — not because it is ungameable.

Integration Points

Public Finance M3: Compliance and Control. Gaming resistance is a design requirement for every compliance system, not an afterthought. The compliance controls described in Public Finance Module 3 — audit sampling, exception monitoring, separation of duties — are structural implementations of the same principles described here. The connection is direct: separation of measurement from incentive is the metric-design equivalent of separation of duties in financial controls. Audit trails for metric integrity serve the same function as audit trails for financial integrity. A compliance system designed without adversarial testing will fail for the same reasons a metric system designed without red-teaming will fail — both assume cooperative behavior from actors with incentives to defect.

Public Finance M7: Policy and Incentives. Every policy metric described in Module 7 should be evaluated against these six principles before deployment. Policy incentives that attach funding consequences to single, self-reported, unaudited metrics with no rotation schedule are structurally designed to be gamed — regardless of the policy’s intent. The principles here are the engineering discipline that converts policy goals into measurement systems that actually track the outcomes they claim to measure.

Product Owner Lens

What is the human behavior problem? Measurement systems attached to incentives are systematically gamed, producing metrics that improve while the outcomes they represent do not.

What cognitive mechanism explains it? Goodhart’s Law and Campbell’s Law describe the macro pattern. The micro mechanism is motivated cognition (Kunda, 1990): when decision-makers have a stake in the measured outcome, their judgment shifts toward favorable interpretations of ambiguous cases — coding decisions, documentation, patient classification. This is not conscious fraud. It is the predictable consequence of asking humans to measure their own performance when money depends on the result.

What design lever improves it? The six principles: separate measurement from incentive, use composite metrics, measure outcomes, build audit trails, red-team before deployment, rotate metrics. These are structural interventions that change the gaming calculus rather than appealing to good faith.

What should software surface? Metric-outcome concordance tracking: does the metric move in the same direction as the underlying outcome it represents? Documentation edit patterns near quality events (timestamp, user, original vs. amended value). Composite score decomposition showing whether improvement is balanced across dimensions or concentrated in the most gameable component. Red-team finding logs linked to metric definitions, with resolution status.

What metric reveals degradation earliest? Metric-outcome discordance — when the metric improves but the underlying outcome (total cost, actual health status, independently measured utilization) does not. This discordance is the earliest detectable signal that gaming has displaced genuine improvement, and it is computable from claims data and clinical outcomes data that most health systems already possess. When a readmission metric improves by 15% but total cost of care is flat, the measurement system is being optimized. The patient’s health is not.