Principal investigator · Pre-submission reproducibility check · Methods and detection data submitted as context
Convergence exists but shares one detection method across all studies. Method-independent reproducibility: Contested.
Finding is the confidence grade itself. Confidence grade is the finding. For an open research question, the grade — not a numeric score — is the complete result.
The ensemble returns a Contested finding on the reproducibility of the reported off-target editing rate. Multiple studies report broadly consistent off-target rates, which superficially looks like robust replication. But the Contrarian's Strong objection is unresolved: nearly all of these studies rely on the same detection assay, so their agreement may reflect a shared methodological blind spot rather than independent confirmation. Convergence without methodological variation is not the same as reproducibility. The within-assay claim (results replicate when the same detection method is used) is Probable; the method-independent claim (the true off-target rate is as reported, regardless of assay) is Contested. The actionable output: confirm with an orthogonal detection method before treating the rate as established.
Settled ground: CRISPR-Cas9 produces off-target edits at some non-zero rate. Several independent groups report off-target rates in a broadly similar range for the guides in question. Detection sensitivity depends heavily on the assay used.
Contested terrain: Whether the reported rate is a property of the editing or of the detection method. Whether cross-study agreement constitutes independent replication given shared methodology.
Unknown territory: How the reported rate behaves under a mechanistically orthogonal detection assay. Whether the shared assay systematically under- or over-counts a class of off-target events.
Knowledge gaps entered: (1) Orthogonal-assay replication — not present in the uploaded literature. (2) Head-to-head assay-bias quantification — does not exist for these samples.
Methodological-independence concern flagged: The replication set is not methodologically independent — the studies share a common detection assay. Agreement among non-independent measurements does not raise confidence in the same way independent replication does. The apparent robustness may be an artefact of shared method bias.
Evidence ceiling: the method-independent reproducibility claim is capped at Contested until an orthogonal assay confirms the rate. The within-assay reproducibility claim (same method, same result) is Probable. No node reaches Established without method-independent confirmation.
@Cartographer — Steelman: Multiple groups reporting a consistent rate is genuine evidence, and the guides and conditions are well-characterised. A working biologist would reasonably treat this as reproducible.
Strong objection [Phase 1]: "The replication attempts use a single detection assay across all studies. Convergence without independent methodological variation is not reproducibility — it is the same measurement repeated. If the shared assay has a systematic blind spot for a class of off-target events, every study inherits it and they will agree while all being wrong in the same direction." Resolution condition: Replicate the rate with a mechanistically orthogonal detection assay on the same samples.
Consistency across independent groups is genuine evidence, and the guides and editing conditions are well-characterised. Treating the rate as a working value is reasonable.
"The replication set shares a single detection assay. Convergence without independent methodological variation is repetition, not reproducibility — a shared blind spot would make every study agree while all being wrong in the same direction."
Replicate the rate with a mechanistically orthogonal detection assay on the same samples
Single-assay replication is common in this subfield because orthogonal assays are costly and low-throughput. The convergence is still reassuring within its limits.
"The manuscript states the off-target rate as a property of the edit rather than of the measurement. Even on the within-assay reading, the claim should be scoped to the assay used."
Scope the reported rate to the detection method in the manuscript text
Pragmatist action item: Scope the rate to the assay and commission an orthogonal-assay replication before submission
No retracted papers. One preprint down-weighted below the primary-claim threshold. Methodological-independence gap flagged across the replication set (shared detection assay). Self-citation ratio within field norms. Phase boundary clearances issued at P1/2 and P2/3.
For open research questions, the confidence grade is the complete finding — calibrated against the evidence base, not against a binary resolution outcome.