Module 21 Activity Worksheet

Module: Module 21: Reproducibility and FAIR Principles in Connectomics
Duration: 4-5 hours
Generated from the module page. Edit modules/module21.md, not this file.

Capability target

Publish a reproducibility-ready connectomics package (data + methods + metadata + limitations) that an external group can audit and reuse.

You are done when you can demonstrate this, not when you have filled in every box below.

Before you start

Check that you have:

• Basic data-processing workflow familiarity
• Basic manuscript methods section familiarity

Bring one question you already have about this topic. Write it here so you can check at the end whether it was answered:

My question:

Questions this module answers

Keep these in view. At the end, answer each in one sentence.

1. What minimum metadata is needed for third-party reuse?
   • Your answer:
2. How should dataset/code versioning be documented in publications?
   • Your answer:
3. Which reproducibility norms are implicit and must be taught explicitly?
   • Your answer:

The task

Scenario: Your lab plans to release a connectomics analysis package to collaborators.

1. Build a FAIR metadata sheet for one analysis output.
2. Create a reproducibility checklist with pass/fail criteria.
3. Draft a “known limitations” section and one deprecation note.
4. Peer-test another team’s package for reuse friction.

What you hand in

• FAIR metadata form
• Reproducibility checklist + validation log
• Reuse friction report with remediation recommendations

Working checklist

Tick as you go. If you skip a step, write why — a skipped step with a stated reason is a decision; a skipped step without one is a gap.

• Define release scope (dataset slice, code commit, parameter set).
• Add machine-readable metadata and provenance fields.
• Validate rerun path in a clean environment.
• Write methods/limitations notes for external users.
• Publish with changelog and deprecation policy.

Evidence and reasoning

Fill one row per claim you make in your artifact. A claim without a limitation is not finished.

Confidence. For your main claim, mark one and say why:

• High — two or more independent lines of evidence agree
• Medium — one strong line, or several that share a weakness
• Uncertain — the deciding evidence is not available to me

Why:

One alternative I considered and rejected, and the reason:

Misconception self-check

These are the errors this module is designed to prevent. Confirm you did not make them, or note where you nearly did:

• I did not assume: Posting files online makes work FAIR.
• I did not assume: A notebook that ran end-to-end once is proof of reproducible science.
• I did not assume: Reproducibility norms are common sense that any careful trainee will infer without being taught.

Session timing (facilitator reference)

Rubric

Score yourself before anyone else does. Where you fall short, name the specific next action rather than a general intention.

• Minimum pass
  • All five provenance elements present: dataset release ID, materialization number, code commit hash, environment specification, parameter configuration.
  • Re-run instructions testable by a peer without contacting the author.
  • Limitations name concrete failure modes, excluded samples, and failed runs rather than generic hedges.
• Strong performance
  • Clean-environment rerun actually attempted, with a friction log and remediations ordered by cost.
  • Hidden norms made explicit: version identifiers in figure legends, a changelog, and a deprecation note.
  • ID churn quantified whenever identifiers cross versions, and reported in the methods.
  • Documentation is audit-friendly: an external reader can locate every provenance element from the README alone.
• Common failure modes
  • Missing version identifiers for data or code.
  • Methods that omit key parameters or the environment specification.
  • “Reproducible in principle” claims without a validation rerun.
  • Limitations sections written as boilerplate rather than as concrete guidance.

My self-assessment:

• Strongest part of my work, and the evidence for that:
• Weakest part, and the specific next action:

Exit prompt

Take one prior analysis output and add:

1. provenance metadata,
2. reproducibility instructions,
3. a 5-line limitations section.

Your answer:

Peer review (swap worksheets)

Reviewing someone else’s reasoning is the fastest way to see the gaps in your own. Assess the evidence quality, not whether you agree with the conclusion.

• Is every claim paired with specific evidence?
• Is at least one limitation stated, and is it a real one?
• Is the confidence level justified by the number of independent evidence lines?
• One thing this person did better than me:
• One question I would ask them:

Module page: /modules/module21/ · Slides: /modules/slides/module21/ · Facilitator guide