A credible quantum machine learning experiment is more than a circuit that produces a score. It requires disciplined data preparation, fair comparisons, reproducible training, and evaluation that separates model behavior from simulator settings or hardware noise. This checklist provides a reusable workflow for testing a quantum model alongside classical baselines, from the first dataset split to the final comparison.
Overview
Quantum machine learning experiments typically combine a classical data pipeline with a parameterized quantum circuit. A classical process selects and scales features, encodes them into quantum states or circuit rotations, and passes measured outputs to a loss function or downstream classifier. An optimizer then updates circuit parameters, often through repeated calls to a simulator or quantum processor.
That hybrid structure creates additional ways for an experiment to become misleading. A small dataset can make a result highly sensitive to one train-test split. Feature encoding may use more qubits than the target environment supports. Shot noise can make optimization appear unstable. A quantum model may also seem competitive only because the classical comparison is weak or has been tuned less carefully.
Use the following workflow for every new dataset, ansatz, optimizer, simulator, or hardware-backed run. Record decisions as you make them rather than trying to reconstruct the experiment later.
Reusable experiment record
- Task: Define the prediction target, input type, and whether the task is classification, regression, ranking, or representation learning.
- Dataset version: Record the source, preprocessing code, row counts, feature names, labels, and any filtering rules.
- Quantum configuration: Record qubit count, feature encoding, ansatz, circuit depth, measurement scheme, shots, simulator or QPU, and transpilation or compilation settings where applicable.
- Training configuration: Record initialization method, optimizer, learning rate or step rule, batch size, number of iterations, stopping criteria, and random seeds.
- Evaluation configuration: Record the split strategy, metrics, confidence intervals or repeated-run summaries, and the exact models used for comparison.
Checklist by scenario
Scenario 1: Preparing a new dataset
- Define the unit of prediction. Ensure each row represents one independent example, or document the grouping when examples come from the same user, device, patient, transaction, or time series.
- Inspect labels before modeling. Check class balance, missing labels, duplicate records, unexpected categories, and target values that may have been generated from the features.
- Choose the split before fitting preprocessing. Create train, validation, and test partitions before calculating scaling parameters, imputations, feature-selection rules, or learned embeddings.
- Fit transformations on training data only. Apply the training-set mean and standard deviation, for example, to validation and test data. Do not let held-out examples influence preprocessing.
- Match features to the encoding budget. If the circuit accepts one feature per qubit, reduce, group, or project the input explicitly. State whether dimensionality reduction is classical and whether its parameters are fit only on training data.
- Scale for the encoding. Rotation-based encodings often work best when feature values are mapped to a documented range. Preserve the same transformation across training, validation, testing, and deployment simulations.
Scenario 2: Establishing a fair baseline
- Start with simple models. Use an interpretable baseline such as a majority classifier, mean predictor, logistic regression, linear regression, or a shallow tree where appropriate.
- Add a capable classical comparator. Include at least one model suited to the data type and task. A quantum model should not be judged against an unnecessarily weak baseline.
- Keep the data pipeline consistent. Give classical and quantum models the same training, validation, and test partitions. If the quantum model uses dimensionality reduction, provide a documented equivalent input pipeline for the comparison.
- Tune fairly. Allocate comparable validation effort and record the search space, number of trials, and selection rule. Avoid tuning the quantum model on the test set.
- Report resource context. Record training time, circuit evaluations, shots, memory needs, and hardware queue or execution constraints when they affect interpretation.
Scenario 3: Training a variational quantum model
- Specify the circuit. Document data encoding, trainable layers, entanglement pattern, depth, measurement observables, and the number of trainable parameters.
- Choose initialization deliberately. Compare at least one reproducible initialization strategy. Note whether parameters are sampled, set near zero, copied from a previous run, or initialized from a classical procedure.
- Track the full training history. Save training loss, validation loss, metric values, parameter norms where useful, optimizer steps, circuit evaluations, and failures or retries.
- Watch for stalled optimization. A flat loss curve may indicate a poor learning rate, an unsuitable initialization, excessive circuit depth, noisy gradients, or an encoding problem. Diagnose before increasing the iteration count.
- Separate development from final testing. Use validation results for model and hyperparameter choices. Evaluate the selected configuration on the test set once, or define a careful repeated evaluation plan in advance.
Scenario 4: Moving from simulation to hardware
- Freeze the model first. Establish a simulator result with fixed preprocessing, parameters, circuit structure, and evaluation code before changing execution backends.
- Record shots and noise assumptions. A statevector result, a finite-shot ideal simulation, a noise model, and a hardware run answer different questions. Label them separately.
- Use matched evaluation conditions. Where feasible, compare the same circuit and measurement procedure under the relevant backends. Note any changes required by qubit connectivity, supported gates, or compilation.
- Repeat noisy runs. Treat one hardware execution as an observation, not necessarily a stable estimate. Use repeated runs or batches when the cost and access model permit, and report variability.
What to double-check
Leakage and split integrity
Data leakage can enter through duplicate examples, temporal overlap, grouped records, target-derived features, normalization, feature selection, augmentation, or hyperparameter selection. For time-dependent data, use a time-aware split instead of randomly mixing future and past examples. For grouped data, keep related records in the same partition.
Metrics that fit the task
Accuracy can conceal poor minority-class performance. Consider precision, recall, F1, balanced accuracy, calibration, or area-under-curve measures for classification when appropriate. For regression, report a metric such as mean absolute error or root mean squared error alongside an interpretable error summary. Define the primary metric before reviewing test results.
Variance and repeatability
Repeat runs with different random seeds when initialization, minibatch order, sampling, or finite shots can change the outcome. Report the mean and spread, or show the individual results, rather than presenting only the best run. If the dataset is small, consider repeated splits or cross-validation, while keeping a final untouched test set when the project requires one.
Framework and API assumptions
Quantum development frameworks change in naming, import paths, autodiff behavior, optimizer interfaces, device support, and execution abstractions. Before sharing an experiment, pin the environment or record package versions and confirm that the documented gradient method, backend, and shot configuration still behave as expected. Whether you use PennyLane, Qiskit Machine Learning, a classical library, or another stack, verify the actual API in the installed version instead of relying on an older example.
For broader tooling decisions, compare the workflow requirements rather than choosing by reputation alone. The Qiskit, Cirq, and PennyLane SDK comparison can help frame differences in circuit construction, machine learning integration, and backend access. A separate quantum simulator benchmarking guide is useful when execution cost, scale, or noise modeling becomes part of the decision.
Common mistakes
- Using the test set as a tuning set. If test performance influences the choice of encoding, ansatz, optimizer, or preprocessing, the test set is no longer an unbiased final check.
- Comparing different data transformations without explanation. A model given more favorable preprocessing has not been fairly compared.
- Reporting the best seed. Best-case performance can hide unstable training and makes replication difficult.
- Confusing more circuit depth with a better model. Additional layers can increase parameters, execution cost, optimization difficulty, and sensitivity to noise.
- Ignoring class balance. A high accuracy score may simply reflect the dominant class. Always inspect class-specific results.
- Mixing simulator and hardware conclusions. A noiseless simulation can test an algorithmic idea, but it does not establish hardware performance.
- Changing several variables at once. If the encoding, optimizer, ansatz, shots, and dataset split all change together, the source of an improvement is unclear.
- Failing to save artifacts. Keep the split indices, preprocessing parameters, configuration, logs, environment details, and final parameters with the result.
When to revisit
Revisit this checklist before each new experiment cycle, especially when the dataset, target definition, feature count, encoding, ansatz, optimizer, or execution backend changes. It should also be reviewed when a framework releases a major update, when a simulator is replaced, or when a workflow moves from local testing to a cloud platform or QPU.
For a practical handoff, create a short experiment review with five decisions: what changed, what stayed fixed, which baseline was used, how uncertainty was measured, and what evidence would justify the next step. If the next step involves cloud execution, pair the model record with a reproducible job process covering submission, monitoring, retry behavior, and results capture; see the quantum cloud job management guide.
Finally, do not treat a small performance difference as a conclusion by itself. First verify split integrity, repeatability, metric choice, resource cost, and backend conditions. A well-controlled negative result can be more useful than an impressive but irreproducible score: it tells the team which assumption to change in the next quantum machine learning experiment.