SciKnowOrg / Scientific Table Reconstruction
SCIENTIFIC TABLE RECONSTRUCTION BENCHMARK

From table images
to structured knowledge.

Measure how well models reconstruct the tables in scientific papers. One image in, one table out—across five research domains.

610table images
253scientific papers
5research domains
12reported measures
PUBLIC TEST SET · v1.0.1· Image-only inputs · Open gold annotations
COMPARE RECONSTRUCTION QUALITY

The leaderboard

How scoring works ↘

Relative Mapping Similarity · approximate matching with text and numeric tolerances.

#Model / methodMacro equal weight per tableMicro pooled mappingsEvaluation modeCoverage

Ready for the first result.

No reviewed model submissions yet.
Run your model on the test images and help set the benchmark.

See the submission guide →

Every submission covers the full test set. Empty and failed extractions stay in the denominator. Domain views are diagnostic; the official ranking uses all 610 tables.

A CLOSER LOOK

Real tables. Across science.

View all 610 tables ↗

Original table crops paired with gold rows of cell strings. Each example includes its domain, paper ID and table ID.

01 / MODEL INPUTTable image
Example scientific table crop
02 / TARGET OUTPUTGold reconstruction
FIVE FIELDS. THAT’S IT.domainpaper_idtable_idimagegold_rows
CLEAR RULES, REPRODUCIBLE SCORES

Two ways to measure
the same reconstruction.

RMS gives partial credit. Exact Mapping Match requires exact normalized mappings. Both use a frozen, versioned Tabulus evaluator adapted from DePlot.

Read the complete evaluation protocol ↗
What counts as a mapping?

The first row provides column headers and the first column provides row labels. Their combined key maps to each body value. Whitespace and case are normalized before comparing mappings. This measures associations, not raw CSV bytes.

How do macro and micro differ?

Macro averages each table’s precision, recall and F1 separately. Micro pools matched credit and gold/predicted mapping counts, then computes the three scores. Failed tables receive zero macro scores and keep their gold counts for micro recall and F1.

What does “exact” mean here?

Exact Mapping Match uses identical normalized keys and values. RMS uses approximate text matching (threshold 0.5) and relative numeric matching (threshold 0.1). Each selects the better of original and transposed predictions using per-table F1. Evaluation mode is physical-table; model input is image-only.

What should I know about the gold?

This is an open-gold public test set. Do not use gold in inference or tuning; disclose training overlap. Source gold is preserved, including merged-cell and repeated-label conventions. The inherited mapping parser collapses duplicate keys and has a few delimiter/zero-mapping edge cases. The manifest and protocol document these limitations and empty-set conventions.

MAKE YOUR WORK COMPARABLE

Add your model
to the leaderboard.

Inference runs on your own hardware. The dataset, evaluator and static leaderboard are open and free to access.

Open a submission discussion ↗
  1. 01

    Load the images

    Import the v1.0.1 test split from SciKnowOrg/Scientific-Table-Reconstruction. Give the model only each table image.

  2. 02

    Reconstruct and score

    Save all 610 predictions as rows of strings, with explicit failures. Run the pinned CPU evaluator for all twelve measures.

    Evaluator & prediction format ↗
  3. 03

    Share the evidence

    Publish predictions, prompt, model/code revisions, settings and training-overlap disclosure. Maintainers rescore before adding an entry.

    Download metadata template ↓