Machine Learning Model Benchmark Matrix Markdown Table Template
A benchmark matrix supports a model-selection decision: which architecture gives the accuracy you need within your latency, memory, and cost budget. Readers are the reviewers of a paper, the team choosing what to deploy, or the next person who wonders why the edge build uses a smaller network. The table is only useful if every number in a column was produced under the same protocol — same test split, same preprocessing and input resolution, same hardware for timing.
Edit This Template
Edit cells, add rows, or sort — the Markdown updates livePaste cells from Excel or Google Sheets into the first header to replace the table · Tab / Enter / arrows move · Shift+F10 opens row & column actions
When to Use This Table
Precision, recall, and F1 need an averaging mode whenever there are more than two classes. In single-label multi-class classification, micro-averaged precision, recall, and F1 all equal plain accuracy, so reporting them adds nothing. Macro averaging takes the mean of per-class scores, and on a test set with the same number of examples per class, macro recall equals accuracy exactly. Those identities are a useful sanity check: if your columns violate them, something in the evaluation is wrong.
Numbers copied from different papers are rarely comparable. Results vary with training recipe, extra pre-training data, test-time resolution, and augmentation, and inference times depend on GPU, batch size, numeric precision, and runtime. Either re-run every model under one script, or add columns that disclose the differences, and cite the source and date of every figure you did not measure yourself.
Formatting This Table in Markdown
A machine learning model benchmark matrix table puts a few things in cells that Markdown treats as syntax. These are the ones to watch so it renders correctly on GitHub, GitLab, Obsidian and docs sites.
Bold the best value, and say which direction wins
Bold the highest accuracy and F1 but the lowest latency and parameter count, and mark the direction in the header with ↑ or ↓ (
Inference (ms) ↓). Bold tied values together, and wrap the bold around the code span, as the Swin row does, so every best cell looks alike.Report mean ± std in one consistent shape
Cells like
76.1 ± 0.3line up when right-aligned only if every row has the same decimals, so keep trailing zeros (0.750, not0.75) on both sides of the ±. If rows get cluttered, split the score and the standard deviation into two columns.Long model names squeeze the metric columns
Column width follows content and cannot be set in GFM, so a name like “Vision Transformer (ViT-Base, IN-21k pre-trained)” pushes six numeric columns toward a horizontal scroll. Keep names short, give pre-training and resolution their own columns, and abbreviate headers (
Params (M),Acc); see column width.
What Goes in Each Column
| Column | Alignment | What to put here |
|---|---|---|
| Model Architecture | left :--- | Architecture plus variant and checkpoint, e.g. “ResNet-50 (torchvision IMAGENET1K_V2)”, and the input resolution if it varies between rows. Bold the row you recommend, not necessarily the one with the top score. |
| Params (M) | right ---: | Total trainable parameters in millions to one decimal, right-aligned. Since the header already says (M), write 25.6 rather than 25.6M. Count from the checkpoint you evaluated, e.g. sum(p.numel() for p in model.parameters()). |
| Accuracy | right ---: | Top-1 accuracy on a named split. Choose one format for the whole table — percentages or 0–1 decimals — since mixing 76.1% with 0.752 invites misreading. With several seeds, report mean ± standard deviation. |
| Precision | right ---: | For binary tasks, TP / (TP + FP) for the positive class. For multi-class tasks, put the averaging mode in the header, e.g. “Precision (macro)”, and compute it with the same library call for every model. |
| Recall | right ---: | TP / (TP + FN), averaged the same way as precision. If your test set is class-balanced, macro recall should match accuracy to rounding; a mismatch means the averaging or the labels are off. |
| F1 Score | right ---: | Harmonic mean of precision and recall. Note that scikit-learn’s macro F1 is the mean of per-class F1 scores, which generally differs from the harmonic mean of macro precision and macro recall. State which one you report. |
| Inference (ms) | right ---: | Median latency per input with hardware, batch size, precision, and runtime stated in the header or a note (“A10G, batch 1, FP16, PyTorch 2.x”). Right-aligned. Say whether preprocessing is included; for edge models it often dominates. |
How to Fill It In
- 1Write the evaluation protocol above the table: dataset and split, preprocessing, resolution, metric averaging mode, hardware, and software versions. Every row must follow it, or carry a footnote explaining the deviation.
- 2Evaluate all models with one script. scikit-learn’s
precision_recall_fscore_support(y_true, y_pred, average="macro")andclassification_reportgive consistent definitions across models. - 3Time inference properly: run warm-up iterations, synchronise the GPU before reading the clock (
torch.cuda.synchronize()), time a few hundred iterations, and report the median. Wall-clock timing of asynchronous GPU calls without synchronisation measures only the launch. - 4Cross-check against each model’s published card or paper. If your accuracy is well below the reported figure, check resizing, cropping, and normalisation constants before blaming the model.
- 5Generate the table from your results dataframe (see Python Markdown tables), then bold the best value per column — highest for quality metrics, lowest for params and latency.
Ready-to-Paste Variations
Binary classifier comparison at a fixed threshold
For fraud, spam, or diagnostic models. Threshold-free AUC sits next to threshold-dependent precision and recall; replace the example rows with your own results.
| Model | Threshold | Precision | Recall | F1 | ROC AUC |
| :--- | ---: | ---: | ---: | ---: | ---: |
| Logistic regression (baseline) | 0.50 | 0.810 | 0.720 | 0.762 | 0.861 |
| Gradient-boosted trees | 0.42 | 0.860 | 0.790 | 0.824 | 0.912 |
| **Fine-tuned transformer** | 0.38 | **0.880** | **0.850** | **0.865** | **0.934** |Reproducibility columns for a paper or report
Discloses the factors that make published numbers incomparable, and reports seed variance so readers can tell real gaps from noise.
| Model | Pre-training data | Resolution | Top-1 (mean ± sd, 3 seeds) | Latency p50 (ms) | Source |
| :--- | :--- | ---: | ---: | ---: | :--- |
| Model A | ImageNet-1k only | 224 | 78.4 ± 0.2 | 9.1 | This work |
| Model B | ImageNet-21k | 224 | 81.0 ± 0.3 | 14.6 | This work |
| Model C | ImageNet-21k | 384 | 83.2 | — | Reported in paper |Compact README model zoo
For a repository that ships several checkpoints: size, quality, speed, and a download link per row.
| Checkpoint | Params (M) | Top-1 (%) | CPU latency (ms) | Weights |
| :--- | ---: | ---: | ---: | :--- |
| `small` | 5.1 | 74.9 | 6 | [download](https://example.com/small.pt) |
| `base` | 22.0 | 79.8 | 21 | [download](https://example.com/base.pt) |
| `large` | 86.4 | 82.6 | 73 | [download](https://example.com/large.pt) |Common Mistakes
Putting a model trained with extra pre-training data at higher resolution in the same column as one trained on the base dataset at 224 px, then declaring a winner.
Add pre-training and resolution columns, or re-train and re-evaluate under one protocol. The reproducibility variation shows the extra columns.
Reporting micro-averaged precision, recall, and F1 for a single-label multi-class task.
Those three numbers equal accuracy and tell the reader nothing new. Use macro or weighted averaging, name it in the header, and add per-class results if a minority class matters.
Publishing latency without hardware, batch size, or numeric precision.
Put the full timing setup in the header or a footnote. A latency figure without its setup cannot be reproduced or compared.
Bolding a 0.2-point winner from a single training run.
Run at least three seeds and report mean ± standard deviation. If the intervals overlap, say the models are tied rather than bolding one.
Related Markdown Table Templates
Browse all 50 templatesLarge Language Model (LLM) Capabilities
Compare foundation models by context window, input and output token pricing, multimodality, and benchmark scores, with the date each row was checked.
Classification Model Confusion Matrix
Binary classification confusion matrix comparing actual vs predicted positives and negatives, with row and column totals for deriving precision and recall.
API Latency Percentiles (p50 to p99.9)
Measure service response latency distributions across p50, p90, p95, p99, and p99.9 percentiles alongside error rates.
Machine Learning Model Benchmark Matrix: Frequently Asked Questions
Should I report accuracy or F1 score?
Report accuracy when classes are roughly balanced and errors cost about the same. When one class is rare, accuracy hides failure on that class, so report precision, recall, and F1 for it — or the macro average across classes — alongside accuracy.
What is the difference between macro, micro, and weighted averaging?
Macro averaging computes the metric per class and takes the unweighted mean, so rare classes count equally. Micro averaging pools all predictions first, which for single-label multi-class equals accuracy. Weighted averaging is the per-class mean weighted by each class’s support.
How many decimal places should benchmark scores have?
No more than the test set can support. With 10,000 test examples and accuracy near 80%, the standard error is about 0.4 percentage points, so a third decimal place in a percentage is false precision. One decimal for percentages or three for 0–1 scores is plenty.
Should inference time be measured on CPU or GPU?
Measure on the hardware you will deploy to, at the batch size you will serve. A model that wins on a data-centre GPU at batch 64 can lose on a phone CPU at batch 1, so label the column with the setup or add one column per target.