No Better Than a Coin Flip: Benchmarking Quantum Kernels on ICU Mortality

Can a quantum kernel beat logistic regression at predicting whether an ICU patient survives their hospital stay? I built the whole stack to find out: three quantum feature maps, a variational quantum classifier, a quantum neural network, and a matched set of classical baselines, all running from one command. The honest answer on this benchmark was no. The part worth writing about is not the no. It is how I made the no trustworthy, so it reads as a measurement rather than a bad day for the quantum models.

Null results have a reputation problem. They get filed as failures, or quietly reframed until a chart points the right way. I wanted the opposite: a benchmark where "the quantum models did not beat classical" is a specific, falsifiable statement with error bars around it, not a vibe. That discipline is the whole point of the project, and it is the reason this writeup leads with the finding instead of burying it.

One disclaimer before any numbers, because it changes how you should read all of them. The results below come from the synthetic-fallback run, not the real WiDS ICU dataset. The pipeline pulls the live Kaggle data when credentials are present and drops to a 5,000-row, schema-matched synthetic set when they are not. This reproduction used the synthetic set. So every number here is a statement about a controlled benchmark that runs from a fresh clone with no accounts, not a claim about real patients. I will keep saying that, because it matters.

The setup

The task is ICU mortality prediction, framed on the WiDS Datathon 2020 dataset (roughly 91,000 stays, 186 features). Mortality is about 8 percent of cases, so accuracy is a trap: a model that predicts "survives" for everyone scores in the low 90s and is useless. ROC-AUC is the honest headline metric, with balanced accuracy, PR-AUC, and F1 alongside it.

There are eight models in the comparison. Three classical baselines: logistic regression with L2, a 200-tree random forest, and an RBF SVM. Five quantum models: three quantum support vector classifiers built on different feature maps, a variational quantum classifier (VQC), and a quantum neural network (QNN). All eight see the same cohort, the same split, and the same scoring, so nothing about the comparison is rigged in either direction.

Encoding six features into six qubits

The classical baselines train on the full preprocessed feature set. The quantum models cannot, because the cost of the quantum kernel grows with both the number of qubits and the number of training points, so the data has to come down to something a CPU simulator can chew through. Here is the shared pipeline and where it forks.

Preprocessing is unglamorous and load-bearing: median imputation, one-hot encoding of the categoricals, a stratified 70/10/20 split that preserves the 8 percent positive rate in every fold, and a StandardScaler fit only on the training fold. Then SelectKBest with an ANOVA-F score picks the top six features. Six features means six qubits, run at two repetitions, on a class-balanced subsample of 200 points. The subsample is not laziness. The fidelity kernel is O(N^2) in the training set, and 200 points is already 20,000 circuit evaluations per kernel.

The three quantum SVC feature maps were chosen to be genuinely different from one another, not three settings of the same knob.

The kernel itself is a FidelityQuantumKernel using the ComputeUncompute method: it estimates the squared overlap between two encoded states and enforces the result to be positive semi-definite before handing it to the SVM. The VQC pairs a ZZFeatureMap with a RealAmplitudes ansatz trained by COBYLA on cross-entropy. The QNN wraps a PauliFeatureMap and RealAmplitudes as a SamplerQNN with a parity readout. Both run on the exact statevector simulator, so there is no shot noise hiding in these numbers. The whole thing reproduces from one command:

The result: classical above the line, quantum on it

Lead with the picture, because it is the argument.

The three classical bars sit to the right of the 0.5 chance line, whiskers and all. Every quantum bar sits on the line, with an interval that straddles it. The full scoreboard, with 95 percent bootstrap confidence intervals in brackets:

Why "indistinguishable from random" is the right claim

The temptation is to write "quantum lost" and move on. That is the wrong summary, and the confidence intervals are exactly why.

Every quantum model's 95 percent interval for ROC-AUC contains 0.5. The VQC scores 0.540, but its interval runs from 0.457 to 0.616, so on this benchmark I cannot distinguish it from a coin flip. The same is true for all five. The honest statement is not "the quantum models are worse than classical." It is "the quantum models are not distinguishable from chance here," and that is a claim the intervals let you test rather than assert.

The classical side is the control that makes the reading credible. Logistic regression lands at 0.817 with an interval of 0.787 to 0.845, comfortably clear of 0.5. And it is not a lucky split: 5-fold stratified cross-validation puts its mean ROC-AUC at 0.810 with a standard deviation of 0.013, right on top of the single-split estimate. Random forest and SVM-RBF tell the same story at 0.791 and 0.756. There is real, stable signal in this data. The classical models find it. The quantum models do not.

Feature-map choice turns out to be noise at this scale. The three QSVM maps land within about 0.01 ROC-AUC of one another, and all three intervals overlap 0.5. That is what you would expect from the kernel geometry: at 200 points and six qubits, the fidelity kernel matrix is close to the identity, with nearly orthogonal off-diagonal structure, so the SVM has almost nothing to separate on. Deciding between ZZ and Pauli and the custom ring here is like arguing about the paint color of a car with no engine.

The cost, and why it shaped the experiment

The other axis is time, and it is not close.

The classical models train in roughly 0.01 to 1.1 seconds. The quantum models take 37 to 174 seconds each on the statevector simulator, so somewhere between 100 and 10,000 times slower for a result that cannot clear chance. On real hardware, once shot noise and calibration enter, the gap widens.

That cost is also why the evaluation is deliberately asymmetric, and I want to flag the asymmetry rather than hide it. Every model gets bootstrap 95 percent confidence intervals from 1,000 seeded resamples of the test predictions. Only the classical models get 5-fold cross-validation, because a fidelity kernel is O(N^2) per fold and cross-validating the quantum models on a CPU simulator was not worth the hours. So the quantum models rest on a single split plus bootstrap, the classical models on both. That is a real limitation, disclosed on purpose.

Where a quantum kernel could still matter

None of this says quantum machine learning is a dead end. It says a quantum kernel did not help on this dataset, at this scale, on this hardware. Those qualifiers carry weight.

The theory is more careful than the hype. Schuld and Killoran (2019) framed quantum feature maps as embeddings into a feature Hilbert space, which is the lens that makes the whole kernel comparison well-posed. Liu, Arunachalam and Temme (2021) went further and identified specific data-encoding regimes where a quantum kernel is provably hard to approximate classically. Those regimes are constructed, not clinical. ICU mortality on 186 tabular features is not one of them, and there was never a strong prior reason to think it would be.

What does transfer is the engineering. Designing feature maps that are actually distinct, estimating fidelities, enforcing positive semi-definiteness, running everything through the modern primitive-based Qiskit API instead of deprecated account-based calls: that stack is the same stack a fault-tolerant machine will need. The result on this data is null. The scaffolding is reusable, and building it against a real, messy, imbalanced medical task rather than a toy dataset is what taught me where it strains.

What I would change

The limitations are the roadmap.

The biggest is the data. This run used the synthetic fallback, so the very next step is the real WiDS cohort with Kaggle credentials, to check whether the null result survives contact with real missingness and real class structure. The second is scale. At 200 points the quantum kernel had no room to express structure, so a targeted study that holds the feature map fixed and pushes N upward would separate "quantum has no signal here" from "quantum had no room here." The third is the single-split dependence on the quantum side, which real cross-validation would fix if the compute budget allowed.

To be exact about the scope: "no better than chance" is a claim about this synthetic benchmark, at 200 points and six qubits, on a noiseless simulator. It is not a claim that quantum machine learning fails in general, and it is not a claim that quantum is worse than classical. It is a bounded, reproducible measurement with intervals attached. This is a research and portfolio project, and none of it is validated for clinical use.

The repository leads with the same honest framing, and the live demo lets you poke at the classical side in the browser.

References: