Skip to slides

NEUROTRAILBLAZERS · TEACHING MATERIALS

Segmentation and Proofreading

Reading version. Text reflows with your screen and zoom. Figures retain their available captions. Expand teaching notes for additional context and, where supplied, visual descriptions.

Contents · 19 slides
  1. 1. Segmentation and Proofreading
  2. 2. Session outcomes (60 minutes)
  3. 3. Pedagogical arc
  4. 4. Why proofreading is scientific QC
  5. 5. From voxels to objects
  6. 6. Ultrastructure-informed correction
  7. 7. The merge/split dial, on one real object
  8. 8. Before and after a human fixed it
  9. 9. Synapse-aware correction checks
  10. 10. Checking a reconstruction against plausible morphology
  11. 11. Boundary failure case
  12. 12. Identity-sensitive correction context
  13. 13. Metrics and release gates: what each one is blind to
  14. 14. Operational proofreading loop
  15. 15. Misconceptions to correct
  16. 16. Activity
  17. 17. Rubric checkpoint
  18. 18. Segmentation outside neuroscience, for intuition
  19. 19. References and attribution

Slide 1 of 19

H01 electron microscopy with object segmentation and original 2 µm scale bar

Human cortex · H01
Object segmentation over electron microscopy

H01 release · Lichtman Lab / Harvard & Connectomics at Google · CC BY 4.0
Shapson-Coe et al. (2024) · doi:10.1126/science.adk4858

Technical Course · Unit 08

Segmentation and Proofreading

Proofreading is allocation under a fixed budget, not cleanup. Rank by effect on the endpoint, not by conspicuousness.

Slide 2 of 19

Session outcomes (60 minutes)

Slide 3 of 19

Pedagogical arc

Slide 4 of 19

Why proofreading is scientific QC

Correction policy determines analysis validity.

Source: assets_outreach source decks, Module14 L2 S03. Historical/context visual.

Figure description and teaching notes

Instructor script: "The pipeline is deliberately tuned to over-segment. It prefers splits to merges, because splits are cheaper to repair. That design decision is why proofreading is mostly joining."
Where both segmentation families fail, structurally: thin processes (a 60 nm spine neck may appear in one or two 40 nm sections); steep z-trajectories; tightly apposed membranes under weak staining; artifact regions; rare morphologies — including the boundaries of the volume.
Ask where the measurement points are: a pipeline without a fixed evaluation set of neurons can only report aggregate scores, and aggregate scores are how a merge regression ships.

Slide 5 of 19

From voxels to objects

Enforce explicit error-class coding in logs.

Source: assets_outreach source decks, Module14 L2 S08. Historical/context visual.

Figure description and teaching notes

Everything upstream of this point is repairable; everything downstream inherits whatever came through. Put the error taxonomy (unit §2) next to it: split, merge, glia–neuron merge, orphan fragment, false synapse, missed synapse, wrong synapse partner.
Instructor script: "Splits are visible and bounded. Merges are invisible and unbounded. A split leaves evidence of itself — a neuron that stops in mid-neuropil. A merge leaves an object that looks like a neuron and is not."

Slide 6 of 19

Ultrastructure-informed correction

Before touching anything: what does this correction change about the endpoint, per minute of annotator time?

Source: Pat Rivlin training materials (MICrONS proofreading deck), RIV-ULTRA S06.

Figure description and teaching notes

This carries the Unit 05 reading skills into proofreading. Every correction decision ultimately returns to the voxels — a soma, a nucleus, the organelle content on either side of a boundary. Set the frame now: the question about any candidate correction is its effect on the endpoint, not how obvious the error looks.

Slide 7 of 19

The merge/split dial, on one real object

H01 ships two agglomerations of the same data: c2 calls this one object; c3 calls it two. Neither is a bug.

Source: H01 release (Lichtman Lab, Harvard; Connectomics at Google), CC BY 4.0 · Shapson-Coe et al. 2024, doi:10.1126/science.adk4858 · site render.

Figure description and teaching notes

Across 104 proofread cells, c3 needed 1.6x fewer merge fixes and 2.1x more split fixes than c2. Aggressive agglomeration trades splits for merges; conservative trades merges for splits.
Instructor script: "Which one would you proofread on? It depends on your endpoint. If you are counting inputs per cell, a merge invents inputs; if you are tracing long-range axons, splits cost you the path." Note also what 104 proofread cells out of 16,087 neurons means in practice: most of the volume has never been checked by a human.

Slide 8 of 19

Before and after a human fixed it

Green = genuinely this cell. Red = 11,038 voxels the algorithm wrongly absorbed.

Source: H01 release (Lichtman Lab, Harvard; Connectomics at Google), CC BY 4.0 · Shapson-Coe et al. 2024, doi:10.1126/science.adk4858 · site render.

Figure description and teaching notes

Instructor script: "The membrane it crossed is real but faint. Would you have caught it?" Give the room ten seconds on the left panel before pointing.
Then the second lesson: note where the error is NOT. At the soma, the automated segmentation needs no correction at all. Errors concentrate at thin processes and faint boundaries, which is why triage should look there rather than where the object is largest.

Slide 9 of 19

Synapse-aware correction checks

Apply the Unit 05 synapse criteria before crediting a detection.

Source: Pat Rivlin training materials (MICrONS proofreading deck), RIV-ULTRA S09.

Figure description and teaching notes

A synapse assigned to a merged object still scores as correct under synapse precision — exactly what that metric is blind to. False synapses inflate degree and hit weak, one-synapse connections hardest; missed synapses deflate degree non-uniformly by synapse size. Both are found by human verification on a sample, not by browsing.

Slide 10 of 19

Checking a reconstruction against plausible morphology

A merged object otherwise looks like a perfectly ordinary neuron.

Source: Pat Rivlin training materials (MICrONS proofreading deck), RIV-ULTRA S11.

Figure description and teaching notes

Implausible-morphology detection is how merges get found at all. Automated detectors look for an object with two somata, or with both ribosomes and presynaptic vesicle clusters — the Unit 06 cue-conflict alarm. Humans then adjudicate a ranked queue rather than browsing the volume.

Slide 11 of 19

Boundary failure case

Show when to stop and escalate instead of over-correcting.

Source: Pat Rivlin training materials (MICrONS proofreading deck), RIV-ULTRA S23.

Figure description and teaching notes

Estimate the cost to fix before committing. A forty-minute trace through a difficult region loses to five five-minute corrections elsewhere — unless the cell is in your analysis set and the error sits near the root of the arbor, where it disconnects everything distal. If the boundary cannot be resolved with the evidence in view, the correct action is to log it with a reason and route it to adjudication.

Slide 12 of 19

Identity-sensitive correction context

Every edge direction in the final graph rests on the axon/dendrite call.

Source: Pat Rivlin training materials (MICrONS proofreading deck), RIV-AXDEN S13.

Figure description and teaching notes

A reversed edge is not noise: it deletes a true edge and adds its opposite (Unit 06 §4). Audit the edges whose direction would change your conclusion rather than auditing uniformly.

Slide 13 of 19

Metrics and release gates: what each one is blind to

Metric Blind to
VI (split + merge components) Object size; a total can improve while merges get worse
ERL Merges, unless explicitly penalized
Edge precision / recall Weights a 1-synapse and a 50-synapse edge equally
Synapse F1 Assumes correct segmentation underneath
Figure description and teaching notes

Rule from unit §3: use at least two metrics from different rows, and always report VI's split and merge components separately. A single VI number can improve while merges get worse, because the split component dominates — a real and common way to ship a regression.
Then the metric that actually matters: the effect on your endpoint. Take a random 20-cell sample, proofread it exhaustively, recompute the endpoint, report the shift. "Exhaustive proofreading of a 20-cell sample changed the ratio from 3.1 to 2.8" says more about data quality than any VI value.

Slide 14 of 19

Operational proofreading loop

  1. Triage by expected downstream impact.
  2. Correct with local + global consistency checks.
  3. Update targeted metrics.
  4. Route unresolved cases for adjudication.
  5. Gate release on predefined thresholds.
Figure description and teaching notes

Work the unit's triage example with one annotator-hour. A: a split near the soma of an analysis-set cell, ~60% of the arbor, 10 min. B: a glia–neuron merge on another analysis-set cell, 20 min. C: a conspicuous two-soma merge on cells outside the analysis set, 15 min.
Check C's partner lists first (2 min) — it could corrupt the endpoint indirectly if either fused neuron is presynaptic to an analysis cell. It is not. Then B before A: an unfixed A delays a cell; an unfixed B poisons one. Spend the remaining time on the next analysis-set candidates, not on C.
Stopping rules must be stated in advance, measurable and tied to the endpoint. The strongest: stop when a second independent pass over a 20-cell sample changes the endpoint by less than 5%.

Slide 15 of 19

Misconceptions to correct

Slide 16 of 19

Activity

Submit one correction log containing:

Figure description and teaching notes

Every edit records who, when, what and ideally why. That record is what lets you roll back a bad batch, spot one annotator's systematic drift, and reconstruct the state of an analysis at any past time.

Slide 17 of 19

Rubric checkpoint

Slide 18 of 19

Segmentation outside neuroscience, for intuition

An image and its segmentation: every pixel assigned to exactly one object.

Source: Wikimedia Commons, File:Image-segmentation-example.jpg and File:Image-segmentation-example-segmented.png. CC0 1.0.

Slide 19 of 19

References and attribution