Read slides as a page
H01 electron microscopy with object segmentation and original 2 µm scale bar Nanoscale · Module 21

Reproducibility and FAIR Principles in Connectomics

Teaching Deck

Human cortex · H01
Object segmentation over electron microscopy

H01 release · Lichtman Lab / Harvard & Connectomics at Google · CC BY 4.0
Shapson-Coe et al. (2024) · doi:10.1126/science.adk4858

Module 21 · Nanoscale

Learning Objectives

  • Apply FAIR principles to connectomics data products
  • Define minimum reproducibility metadata for analysis releases
  • Build transparent methods/parameter logs for peer reuse
  • Identify hidden-curriculum norms in reproducibility expectations
Module 21 · Nanoscale

Capability Target

Publish a reproducibility-ready connectomics package (data + methods + metadata + limitations) that an external group can audit and reuse.

Module 21 · Nanoscale

Concept Focus

1) FAIR as implementation checklist

  • Technical: findable identifiers, accessible storage, interoperable formats, and reusable metadata each require concrete engineering choices.
  • Plain language: "FAIR" only counts if someone else can actually find, open, and use your work.
  • Misconception guardrail: posting files online makes work FAIR.
Module 21 · Nanoscale

Core Workflow

  • Define release scope (dataset slice, code commit, parameter set).
  • Add machine-readable metadata and provenance fields.
  • Validate rerun path in a clean environment.
  • Write methods/limitations notes for external users.
Module 21 · Nanoscale

Core Workflow (continued)

  • Publish with changelog and deprecation policy.
Module 21 · Nanoscale

Run of Show (60 min)

  • 00:00-06:00 | Framing: the silent bug
  • 06:00-16:00 | The five-element checklist, modeled
  • 16:00-30:00 | Guided practice: audit your own work
  • 30:00-40:00 | Clean-environment rerun
Module 21 · Nanoscale

Run of Show (60 min) (continued)

  • 40:00-50:00 | Known limitations, written honestly
  • 50:00-57:00 | Competency check
  • 57:00-60:00 | Exit ticket
Module 21 · Nanoscale

Misconceptions to Watch

  • Misconception guardrail: posting files online makes work FAIR.
  • Misconception guardrail: a notebook that ran end-to-end once is proof of reproducible science.
  • Misconception guardrail: reproducibility norms are common sense that any careful trainee will infer without being taught.
Module 21 · Nanoscale

Studio Activity

Scenario: Your lab plans to release a connectomics analysis package to collaborators.

Module 21 · Nanoscale

Activity Output Checklist

  • Evidence-linked artifact submitted.
  • At least one limitation or uncertainty stated.
  • Revision point captured from feedback.
Module 21 · Nanoscale

Assessment Rubric

Minimum pass

  • All five provenance elements present: dataset release ID, materialization number, code commit hash, environment specification, parameter configuration.
  • Re-run instructions testable by a peer without contacting the author.
  • Limitations name concrete failure modes, excluded samples, and failed runs rather than generic hedges.
Module 21 · Nanoscale

Assessment Rubric

Strong performance

  • Clean-environment rerun actually attempted, with a friction log and remediations ordered by cost.
  • Hidden norms made explicit: version identifiers in figure legends, a changelog, and a deprecation note.
  • ID churn quantified whenever identifiers cross versions, and reported in the methods.
Module 21 · Nanoscale

Assessment Rubric (continued)

  • Documentation is audit-friendly: an external reader can locate every provenance element from the README alone.
Module 21 · Nanoscale

Assessment Rubric

Common failure modes

  • Missing version identifiers for data or code.
  • Methods that omit key parameters or the environment specification.
  • "Reproducible in principle" claims without a validation rerun.
  • Limitations sections written as boilerplate rather than as concrete guidance.
Module 21 · Nanoscale

Exit Ticket

Take one prior analysis output and add:

  1. provenance metadata,
  2. reproducibility instructions,
  3. a 5-line limitations section.
Module 21 · Nanoscale

References (Instructor)

  • Wilkinson et al. (2016) - FAIR Guiding Principles.
  • Peng (2011) - Reproducible Research in Computational Science.
  • Project-specific release documentation for H01/MICrONS/FlyWire.
Module 21 · Nanoscale

Teaching Materials

  • Module page: /modules/module21/
  • Session kit: /teaching/sessions/module21/
  • Worksheet: /assets/worksheets/module21/module21-activity.md
Module 21 · Nanoscale

Read, review, or teach

Read these slides as a page

Open the complete lesson

The reading version includes figure descriptions and discussion guidance.

Nanoscale text: CC BY 4.0. Scientific figures retain their individual credits.

Module 21 · Nanoscale

Pre-class preparation (15 min async) Bring one analysis you have run, in whatever state it is in. It does not need to be tidy; untidy is more useful here. Read Technical Unit 04, section 2, on materialization versions and root-ID instability. Minute-by-minute plan 00:00-06:00 | Framing: the silent bug Prompt: "Your notebook ran fine last month and gives a different number today. Nothing in your code changed. What happened?" Establish that analysis against an unpinned segmentation is the most common silent correctness failure in this field. 06:00-16:00 | The five-element checklist, modeled Instructor walks one real analysis through: dataset release ID, materialization number, code commit hash, environment specification, parameter configuration. Show what breaks when each one is missing, in turn. 16:00-30:00 | Guided practice: audit your own work Learners score their brought-in analysis against the five elements. Most will fail two or three; say so in advance to make that safe. Produce a remediation list ordered by how cheap each fix is. 30:00-40:00 | Clean-environment rerun Attempt a rerun of a partner's analysis from their instructions alone, without asking them questions. Log every point of friction. The friction log is the deliverable, not the successful rerun. 40:00-50:00 | Known limitations, written honestly Each learner drafts a limitations paragraph naming concrete failure modes, excluded samples, and failed runs — not generic caveats. Discuss why this is a hidden-curriculum norm: reviewers expect it, and almost nobody is taught to write it. 50:00-57:00 | Competency check Submit: completed five-element record, friction report on a partner's package, and one limitations paragraph. 57:00-60:00 | Exit ticket "One thing in my current work that another person could not reproduce today." Formative checkpoints At 30 minutes: every learner has identified at least one missing element in their own work. A learner reporting five out of five has probably not audited honestly — check. At 50 minutes: limitations paragraphs name specific failure modes rather than generic hedges.