Steal the numbers, skip the R&D loops

A literature survey of handwritten digit recognition for a children's math practice app — what ten-plus published studies actually measured, which fine-tuned numbers transfer to an 8-year-old writing with a finger, mouse, or stylus, and which ones don't.

Research date: November 13, 2026 Target: two-digit answers, browser canvas, child writers Every accuracy figure carries its dataset and conditions

The short answer

Yes — the published work covers this problem well enough to stop hand-rolling matchers. The literature converges on one architecture for this exact use case, and it explains both failure modes the current matcher keeps hitting.

1

$P / $Q point-cloud template matcher on the stroke data + a hard rejection gate

Zero download, ~100–200 lines of JavaScript, New BSD licensed. It matches on the raw stroke points — no rasterization — and is invariant to stroke number, order, and direction, which is exactly how children's digit articulation varies. Its distance score doubles as an interpretable, calibratable garbage gate. Templates can be collected from the actual child, and user-dependent matching is the highest-accuracy regime in every $-family study.

2

Small TensorFlow.js CNN (EMNIST Digits) as second stage / upgrade path

Highest accuracy ceiling (~99% on adult benchmarks) in a model under 0.5 MB, proven in-browser. But softmax confidence alone cannot reject garbage — random noise scores 91% "confidence" — so it needs a margin rule plus a garbage class trained on real scribbles. Best used as a second opinion when the template matcher is uncertain, or as the primary once child data exists for fine-tuning.

3

MyScript iink — only with budget for a commercial license

Best-in-class quality and math-aware, but paid and cloud-dependent on the JavaScript path, with no published accuracy numbers. Do not use Tesseract (handwriting unsupported, under 60%) or plain DTW (matches $1's accuracy at ~80× the cost, with no rejection).

Failure mode 1

Correct numbers score too low

Expected when templates and thresholds come from adult data. Child handwriting scores 2 to 35 points below adult handwriting depending on age and method — as low as 64% for 5-year-olds on the $-family versus 98–99% for adults. The fix is calibration on the actual population: templates collected from the child, thresholds set on the child's own writing.

Failure mode 2

Scribbles score too high

Expected from any forced-classification matcher. $1, DTW, and CNNs always return a best match — there is no "this isn't a digit" answer built in, and raw softmax "confidence" is not confidence at all. The fix is a layered rejection gate: a distance threshold, a top-1 vs top-2 margin rule, and a garbage class trained on real scribbles.

EVIDENCE

The child penalty, measured

Three independent studies that tested the same kind of system on adults and children. This gap — not matcher choice — is the biggest variable in the whole problem.

Recognition accuracy: adults vs children

Percent correct, as reported by each study. Different datasets and methods — compare within each pair, not across pairs.

0% 25% 50% 75% 100% $-family gestures UF INIT, touchscreen 98–99% 64% · age 5 Gurmukhi CNN 425 writers, ages 5–80 97.40% 95.14% Arabic letters CNN Hijja child dataset ~99% ~93% Adults Children

An 8-year-old sits between the 5-year-old and adult ends of these ranges — better than 64%, still well below the adult figures that headline most papers. The Gurmukhi study is the optimistic case: only a ~2.3-point penalty, but its training data already included children. The Arabic CNN reached ~93% on children only when training mixed adult + child data.

CASE STUDIES

What each study measured — and what to take from it

Twelve studies and systems, spanning stroke matchers, bitmap methods, neural nets, and the dead ends. Filter by verdict, or read straight through.

Steal it

$1 Unistroke Recognizer

Wobbrock, Wilson & Li — UIST 2007 · University of Washington Ace Lab

A geometric template matcher in ~100 lines: resample the stroke to N equidistant points, rotate the indicative angle (centroid → first point) to 0°, scale to a reference square, translate the centroid to the origin, then golden-section search ±45° for the best alignment. Score = 1 − d / (0.5·√(size²+size²)). Tested on 4,800 pen gestures from 10 adults on a Pocket PC — 16 UI gestures, not digits. Accuracy climbs with templates: 97.27% with 1 template, ~99% at 3+, 99.55% at 9. It has no rejection mechanism — it always returns a best match — but it shows the steepest N-best score falloff of the matchers tested, which makes the top-1/top-2 margin informative. Two digit-specific traps in the defaults: full rotation invariance confuses 6/9, and non-uniform scaling destroys the aspect ratio that separates 1 from 7 from 4. Official JavaScript implementation, New BSD License.

99.02%
overall · adult UI gestures
StealThe full pipeline and its constants: N=64 resampling, ±45° golden-section search at 2° / 10 iterations, and the score formula. Disable full rotation invariance for digits.

Paper: kevinli.net (PDF) · Project: depts.washington.edu/acelab

Steal it — closest analogue

$N Multistroke Recognizer, in a school math tutor

Anthony & Wobbrock — Graphics Interface 2010

Extends $1 to multistroke input by auto-generating every stroke-order and direction permutation of each template, with bounded rotation invariance and 1-D gesture support. This is the closest published analogue to the target use case anywhere in the survey: 96.6% on 20 algebra symbols entered in situ by middle and high schoolers using a math tutor prototype, with 15 templates per symbol (96.7% with 9). Two cheap optimizations matter for a browser: a start-angle prefilter saves 79.1% of comparisons and gains 1.3% accuracy; comparing only same-stroke-count templates saves 89.5% and gains 1.7%. An HTML/JavaScript version exists.

96.6%
school-age writers · math symbols
StealBounded (not full) rotation invariance, 9–15 templates per class, and both prefilters. This study is the realistic accuracy anchor for school-age writers doing math.

Paper: cs.cmu.edu (PDF)

Steal it — recommended primary

$P Point-Cloud Recognizer & $Q

Vatavu, Anthony & Wobbrock — ICMI 2012 · $Q: MobileHCI 2018

Treats a gesture as an unordered point cloud and matches clouds with an approximate Hungarian assignment — so it is invariant to stroke number, order, direction, and type with no combinatoric overhead. Over 99% user-dependent accuracy with 5+ training samples per gesture; over 99% user-independent with data from 10 participants; ~98% average, beating $N in both settings. $Q is an optimized $P: up to 142× faster with no accuracy loss (9.4 s → 191 ms per classification on a Cortex-A7). For children — who draw a 4 top-first or bottom-first, in one stroke or three — articulation invariance is the single most valuable property in the whole $-family. The catch is the child flag below: the same UF group that uses $P with children reports the family dropping to 64% at age 5.

>99%
user-dependent · 5+ samples/class
StealThe point-cloud matching model itself, and the operating point: 5+ templates per digit, collected from the actual child, puts you in the highest-accuracy regime the literature reports.

$P: depts.washington.edu/acelab · $Q: init.cise.ufl.edu (PDF) · Child figures: UF INIT project page

Ceiling reference only

Dynamic Time Warping on stroke sequences

Bahlmann (UNIPEN) · Iwana et al. 2019 · Alon et al. 2005

DTW elastically aligns two point sequences, absorbing speed and timing variation. On UNIPEN (adult writers, ~15,000 numerals): plain 1-NN with DTW reaches 98.5% on digits — beating an LSTM's 96.5% — and a 2005 prototype hit 1.90% error, but at the cost of 10,630 DTW computations per test digit in brute force. In Wobbrock's head-to-head, DTW (99.15%) statistically tied $1 (99.02%) at roughly 80× the cost, and the $1 paper warns DTW is "sometimes too flexible in matching" — flexibility that helps correct digits and hurts garbage rejection. Multistroke DTW explodes combinatorially (a 4-stroke "E" admits 384 order/direction sequences). No DTW work found includes garbage rejection.

98.5%
1-NN DTW · UNIPEN adult digits
TakeDTW's number as the accuracy ceiling for stroke matching (~98.5% adult) — then use $1/$P, which match it at a fraction of the cost. Don't ship DTW itself.

Bahlmann, frog on hand · Iwana et al. (PDF) · Alon et al. (PDF)

Use with caution

Hausdorff distance + k-NN (sketch Sudoku)

Multilingual sketch-based Sudoku game study

One-sided Hausdorff distance — h(A,B) = max over a∈A of min over b∈B |a−b| — with k-NN at k=5, majority vote of the five smallest distances. It recognized Chinese and Hindi numerals simultaneously at 93% overall (10-fold cross-validation), most numerals above 90%; curvy Hindi digits were harder, and the method is noise-sensitive. The companion lesson from Lindblad et al. (2014) is the durable one: in a comparative study, sum-of-minimal-distances beat Hausdorff, chamfer, and symmetric-difference on monotonicity and noise sensitivity. Pure Hausdorff max-min lets one stray point dominate the score — exactly what children's stray marks and scribbles produce.

93%
Chinese + Hindi numerals · k=5
Stealk=5 voting if you ever need a fallback matcher — and the distance-choice rule: prefer averaged distances (what $1/$P compute) over any max-min distance.

Academia.edu — Sudoku chapter · Lindblad et al. 2014

Steal the strategy

Two-digit strings: segmentation + per-digit scoring

Lu, Chi & Shi — ICASSP 1998 · Connected-components CNN preprint, 2022

Two independent lines of work on the two-digit problem. Lu et al. segmented connected two-digit strings using skeleton feature points and fuzzy-rule ranking of cut paths: 92.4% correct on NIST SD3 with 4.7% of strings explicitly rejected — notable for building a reject option into a digit-string pipeline at all. The 2022 approach is simpler and better suited here: split the string into connected components, predict the digit count per component, and classify each component separately (a 2-digit component needs a 20-class classifier, not 100). That reached 96.1% on NIST19 for 2–6 digit strings. With online stroke data the split is nearly free: group strokes by x-range overlap, split on a clear x-gap, score each digit independently — and if digits touch with no clean gap, reject and ask for a rewrite rather than guessing.

96.1%
2–6 digit strings · NIST19
StealComponent-based splitting + independent per-digit scoring, with rejection (not a guess) when no clean split exists.

Lu et al. (PDF) · EasyChair preprint (PDF)

Use with caution

Chamfer distance + deformable templates

Shi, Gunn & Damper — IEEE TPAMI 2003

Nonlinear active shape models matched by chamfer distance transform: 96.5% writer-independent across 280,000 Chinese radicals (98 classes), and cited deformable-template work reached 99.25% on a 2,000-character NIST SD1 digit subset. Chamfer distance itself is cheap and noise-robust — but this is a heavyweight model-fitting pipeline built for scanned offline characters, not a drop-in browser matcher for live strokes. Nothing here transfers as code; the transferable fact is that distance-transform matching is a solved, robust family — already covered more cheaply by $P.

96.5%
writer-independent · Chinese radicals
TakeConfidence that distance-based matching is sound — then implement the light version ($P), not this pipeline.

eprints.soton.ac.uk (PDF)

Warning, not a recipe

HOG / zoning features + SVM & k-NN — and the ARDIS collapse

MNIST benchmarks · Liu et al. 2003 · ARDIS dataset study

On MNIST (adult, offline, single digits) handcrafted features look strong: HOG + RBF-SVM 97.82%, HOG + KernelKnn 98.4%, plain kNN 97.31%, PHOG + SVM 99.21%. Then the ARDIS result: the same models trained on MNIST and tested on ARDIS (Swedish historical digits) collapse — HOG–SVM 56.18%, kNN 50.15%, SVM 43.40%. Published MNIST numbers do not transfer across handwriting populations, and child finger-writing is a different distribution from MNIST. No browser library exists for HOG+SVM; it would be hand-rolled JavaScript for numbers that won't survive contact with the real user.

56.18%
HOG–SVM · MNIST → ARDIS transfer
StealOnly the cheap auxiliary features — stroke count, aspect ratio, direction histograms, ink density — as inputs to rejection gates. Not the classifier.

ARDIS benchmark · KernelKnn + HOG vignette

Steal it — second stage

CNNs on MNIST / EMNIST, deployed with TensorFlow.js

Cohen et al. 2017 (EMNIST) · Dettmar 2019 (TF.js deployment)

Adult benchmarks are near ceiling: MNIST state of the art ~99.87–99.89%, EMNIST Digits ~99.74–99.84%, a plain CNN 99.18% on MNIST. EMNIST Digits (from NIST SD19 adult census-form writers) has 240,000 training / 40,000 test images. The browser deployment pattern is proven and light: Dettmar's documented app trains a small CNN in Keras — Conv16 → Pool → Conv32 → Pool → Conv64 → Pool → Dense128 → softmax — converts it with tensorflowjs_converter, and runs fully in-browser on finger/mouse canvas input at under 0.5 MB for the harder 62-class EMNIST task; a 10-class digit model is smaller and more accurate. Its preprocessing recipe is the stealable part: crop to the ink bounding box, pad to square, resize to 24×24, pad 2px to 28×28, invert, normalize to [0,1], scale by devicePixelRatio. One gap: no widely-used pretrained TF.js MNIST model with a stable public download URL was verified — teams train their own. And the CNN path's weak point is garbage: see the rejection studies below.

99.74%
EMNIST Digits · 241k-param CNN
StealThe architecture, the <0.5 MB size budget, and the exact canvas preprocessing recipe. Budget for training your own model — there is no verified off-the-shelf download.

Dettmar — TF.js web app · EMNIST Digits CNN (PDF) · MNIST CNN 99.89%

Skip it

Tesseract OCR (including Tesseract.js)

Tesseract FAQ + multiple independent evaluations

Tesseract is trained on printed text; its own FAQ says handwriting results "will be poor." Independent sources agree: handwriting comes out as nonsense, expected accuracy under 60%, with very neat block capitals at best 40–60%. Single isolated digits are its worst case — the LSTM and line segmentation assume lines of printed text, there is no linguistic context, and 1/l/I and 0/O confusions are endemic even on print. Tesseract.js costs ~2 MB of WASM core plus ~10 MB of English traineddata to deliver that. It publishes no official accuracy figure. Apache 2.0 and in-browser, but accuracy disqualifies it outright.

<60%
handwriting · expected
TakeNothing. Rule it out so it never re-enters the design conversation.

zro7 — Tesseract in the browser · Epitome tool notes

Calibrates expectations

Children's handwriting, measured directly

Alwagdani & Jaha — Sensors 2023 · Gurmukhi age-stratified dataset — SN Computer Science 2026 · UF INIT

Three direct measurements of the child penalty. Alwagdani & Jaha: children's handwriting has "lower quality, higher variation, and larger distortions" than adults', and most systems were never trained or tested on child data — a CNN on the Hijja child dataset reached ~93% only when training mixed adult + child data, against ~99% for adult Arabic systems. The Gurmukhi dataset (335,245 characters, 425 writers aged 5–80) gives the cleanest age gradient: the same CNN scores children 95.14%, intermediate adults 97.40%, older adults 90.42% — a ~2.3-point child penalty even with child data in training. And Lisa Anthony's UF group, which uses $P to score children's gestures, states the $-family achieves 98–99% for adults but as low as 64% for 5-year-old children. Shipping children's math apps (Calc Rush, MathPen, Learn Math) converge on the same practical pattern: on-device recognition, per-digit input boxes, and recognition openly treated as experimental. None publishes an accuracy figure.

64%
$-family · 5-year-olds (vs 98–99% adults)
StealThe design pattern from shipping apps — per-digit input boxes, on-device, experimental-by-default — and the training rule: mix child data in, or expect the penalty in full.

Alwagdani & Jaha (MDPI) · Gurmukhi dataset (Springer) · Calc Rush · Learn Math FAQ

Steal it — the missing half

Garbage rejection: the out-of-distribution literature

Hendrycks & Gimpel — ICLR 2017 · Pottier & Burel 1993 · Nagendraprasad & Gupta 1993

The studies that explain — and fix — scribbles outscoring real digits. Hendrycks & Gimpel's max-softmax-probability baseline separates correct from out-of-distribution examples well (97% AUROC on MNIST error detection), but they warn that "softmax probability has a poor direct correspondence to confidence": random Gaussian noise gets 91% prediction confidence from an MNIST classifier. A raw confidence threshold will never zero out scribbles. What works instead: margin rejection — Pottier & Burel rejected when the gap between the two highest outputs was small and reached 1% error at only 4% rejection on postal digits; Nagendraprasad & Gupta's bank-check system rejected below a max activation of 0.7 and additionally required a "clear winner" gap over the runner-up. $1 implementations do the same in distance space: reject when the best-template distance exceeds a calibrated maximum, and classify returns null. The principled upgrade is a garbage class trained on real scribbles (background-class / outlier-exposure approach) — with the patent literature's caveat that a background class can only sit near a few known classes, so it complements thresholds rather than replacing them.

91%
"confidence" given to pure noise
StealAll three layers: distance/activation threshold, top-1 minus top-2 margin, and a garbage class trained on real scribbles. Never a raw confidence cutoff alone.

Hendrycks & Gimpel (PDF) · Pottier & Burel (PDF) · US11295240B2 — garbage class · SwingStates $1 rejection

THE LOOT

16 tuned numbers to steal

Every constant, threshold, and rule worth lifting, each with the study it came from. These are the fine-tuned values the R&D loops were re-deriving by trial and error.

#Number / ruleSource
1Resample each stroke path to N=64 equidistant points (any 32–256 is adequate)$1 paper
2Rotate indicative angle to 0°, then golden-section search ±45°, 2° threshold, 10 iterations$1 paper
3Score = 1 − d / (0.5·√(size²+size²)) — distance normalized by half the bounding-box diagonal$1 paper
4No full rotation invariance, no non-uniform scaling for digits — bound rotation tightly and preserve aspect ratio (full invariance confuses 6/9 and 1/7)$N design
5Templates: 3+ per digit minimum (~99% at 3+ in $1); 5+ for $P user-dependent >99%; 9–15 for best results ($N: 96.6% at 15). Include articulation variants — open/closed 4, barred/plain 7, plain vs flagged 1$1 · $P · $N
6Reject when best-template distance exceeds a calibrated maximum (classify returns null). The threshold must come from real data — distances aren't comparable across algorithmsSwingStates $1 practice
7Reject when the top-1 − top-2 margin is small. Margin rejection achieved 1% error at 4% rejection on postal digits; $1's steep N-best falloff makes margin informativePottier & Burel
8CNN rejection floor: max softmax activation below 0.7 → reject, plus a required "clear winner" gap over the runner-upNagendraprasad & Gupta
9Calibrate rejection thresholds at the 5th percentile of validation confidences (~95% acceptance of known digits); validate on held-out garbage, not just digitsOpen-set recognition tutorial
10Never trust raw softmax for garbage: random noise scores 91%. Add a garbage class trained on real scribblesHendrycks & Gimpel · US11295240B2
11CNN canvas preprocessing: crop to ink bbox → pad to square → resize 24×24 → pad 2px to 28×28 → invert, normalize [0,1]; scale by devicePixelRatioDettmar TF.js app
12Browser CNN architecture: Conv16→Pool→Conv32→Pool→Conv64→Pool→Dense128→softmax(10); converted model <0.5 MBDettmar TF.js app
13Segment two-digit answers by x-gap / connected components; component splitting + CNN reached 96.1% on 2–6 digit NIST19 stringsEasyChair preprint
14Prefer averaged distances over pure Hausdorff max-min — sum-of-minimal-distances beat Hausdorff/chamfer on noise sensitivityLindblad et al. 2014
15$N-style speed: start-angle prefilter saves 79.1% of comparisons (+1.3% accuracy); same-stroke-count filter saves 89.5% (+1.7%)$N paper
16Hausdorff + k-NN fallback: one-sided Hausdorff, k=5 majority vote → 93% on Chinese+Hindi numeralsSudoku chapter
REJECTION GATE

The garbage-rejection playbook

Assembled from the rejection studies above, in the order to build it. No single layer is sufficient — the literature is unambiguous that forced classification plus one threshold still leaks scribbles.

  1. Distance threshold on the best match

    If the best template's distance exceeds a calibrated maximum, return "not a digit" instead of the least-bad guess (the SwingStates $1 pattern). The threshold is per-recognizer and must be calibrated on real data — including real garbage.

  2. Margin rule: top-1 must clearly beat top-2

    Reject when the gap between the best and second-best candidate is small. On postal digits this alone reached 1% error at 4% rejection (Pottier & Burel); the bank-check system paired its 0.7 activation floor with the same "clear winner" requirement. Applies to $P distances and CNN softmax alike.

  3. Calibrate at the 5th percentile

    Set thresholds at the 5th percentile of validation scores, accepting ~95% of known-good digits, and validate against held-out scribbles — not just held-out digits. A threshold tuned only on digits has never seen the thing it's meant to stop.

  4. Train a garbage class on real scribbles

    Collect actual scribbles from actual use and train them as an explicit background class (the outlier-exposure approach). Caveat from the patent literature: a background class can only sit near a few known classes, so it layers on top of the thresholds — it doesn't replace them.

  5. Structural pre-gates (engineering inference — validate before shipping)

    Not a sourced claim, but a cheap one to test: real digits typically take 1–3 strokes, while a zigzag scribble has many direction reversals and extreme aspect ratios. Stroke count, reversal count, and ink density can pre-reject obvious garbage before matching even runs.

Two-digit answers

Segment strokes into digits by x-coordinate grouping — strokes whose x-ranges overlap belong to one digit; a clear x-gap splits digits. With online stroke data this is nearly free and avoids the hard connected-digit segmentation problem of offline OCR, which only reaches ~92–96% even with sophisticated cut-path methods. Score each digit independently, then combine. If the digits touch with no clean gap, reject and ask for a rewrite rather than guessing. One arithmetic warning (engineering inference): whole-number accuracy is roughly the product of per-digit accuracies — 0.97² ≈ 0.94 — so per-digit accuracy needs to be ~97.5%+ for 95% two-digit accuracy.

FLAGS

Do not misapply these numbers

Every headline figure in this survey comes with conditions. These are the transfers the report explicitly warns against.

OPEN QUESTIONS

What the research could not verify

  • No published accuracy exists for the exact target case — 8-year-olds writing two-digit numbers with a finger on a web canvas. The closest data points are $N on middle/high schoolers (96.6%, algebra symbols) and the UF finding of 64% for 5-year-olds (gestures). Any accuracy promise for the real user has to come from the app's own testing.
  • No stable public URL for a pretrained, production TF.js MNIST digit model was verified; the official TF.js example hosts only a partial (digits 0–4) model. Teams train their own.
  • Exact TensorFlow.js runtime bundle size (varies by version and build) and MyScript pricing were not verified from primary sources.
SOURCES

Primary sources