Medical devices & diagnostics
Evaluations and environments for medical device and diagnostic submissions.
A device or companion-diagnostic submission spans Design History Files, bench and analytical validation reports, risk management matrices, clinical concordance tables, and regional deficiency letters. We deploy workflows, build offline and online evaluation suites, and post-train models on the records device teams actually sign.
- FDA, EU MDR / IVDR & NMPA device pathways
- PMA ·
510(k) · CDx - design controls & verification traceability
- 21 CFR 820 ·
ISO 13485 - hazard, dFMEA & clinical risk alignment
- ISO 14971
- in Biopharma Bench, including CDx & device seats
- 24 programs
Systems on the desk
eQMS & DHF repositories · Veeva Vault / RIM · LIMS & bench-test archives · Requirement traceability matrices
Starting records
PMA / 510(k) / De Novo technical sections · Analytical & clinical validity reports (CDx / IVD) · Verification & validation (V&V) protocols & raw data · ISO 14971 risk files & dFMEA / pFMEA tables · FDA Additional Information & NMPA Buzheng notices
Governing authority
21 CFR 820.30 / 814 / 807 & ISO 13485 / 14971 · Executed V&V data over summary claims · Predicate & reference-assay concordance bounds
Deployment
Device and diagnostics teams lose months when a 510(k), PMA supplement, or regional registration package contradicts its own underlying bench reports, risk files, or clinical concordance tables.
- 01
510(k), De Novo & PMA submission authoring and audit
FDA CDRH · Substantial equivalence · SSEDReconciling device descriptions, predicate comparison tables, performance-testing summaries, and labeling across 510(k), De Novo, and original or supplemental PMA submissions so every claim traces to an approved report.
Substantial equivalence matricesPMA Summary of Safety & Effectiveness (SSED)30-day notice & PAS supplementsIFU & labeling redlines- 02
Companion diagnostics (CDx) & IVD lifecycle
PMA P170019-class · Analytical & clinical validityAuditing analytical validation (precision, reproducibility, LoD, interfering substances), clinical concordance and bridging studies, and variant-to-therapy claims across multi-biomarker genomic and immunohistochemistry assays.
PPA / NPA concordance workbooksLoD & precision line-data tablesCDx biomarker label expansionsSpecimen stability & matrix equivalency- 03
Design controls, DHF & ISO 14971 risk traceability
21 CFR 820.30 · ISO 13485 · IEC 62304Checking completeness and closed-loop traceability across user needs, design inputs, design outputs, verification and validation protocols/reports, software anomaly logs, and dFMEA / pFMEA risk controls.
Design Traceability Matrices (DTM)ISO 14971 risk management reportsBiocompatibility (ISO 10993) & electrical safety (IEC 60601)Software V&V & cybersecurity packets- 04
Global device registration & deficiency responses
FDA AI letters · EU MDR / IVDR · NMPA BuzhengPreparing point-by-point responses to FDA Additional Information holds, Notified Body clinical evaluation (CER / PER) non-conformities, and China NMPA technical review supplementary notices (Buzheng) from primary engineering records.
FDA Additional Information (AI) responsesNMPA Buzheng (补正) supplementary dossiersEU MDR / IVDR Clinical Evaluation ReportsType-testing & audit findings
Trace design inputs to V&V evidence
Map design inputs and hazard mitigations across bench-testing protocols, software verification logs, biocompatibility reports, and sterilization validation.
Design Input / Hazard ID
Executed V&V Protocol & Raw Log
Traceability Matrix Check
→Flags any summary claim in the submission draft whose acceptance criterion or sample size diverges from the executed test report.
Evaluation
We build the offline and online evaluation stack for medtech and diagnostics AI teams: staging DHF and submission folders, testing both component-level extraction and full regulatory/quality engineer seats, and co-authoring rubrics with your specialists and our Fellows.
- 01
Your DHF & submission archives, or our reconstructed worlds
Historical dossiers · CDx & device worlds in Biopharma BenchWe build from your historical 510(k), PMA, and DHF folders, or evaluate on our reconstructed device and companion-diagnostic environments (including FoundationOne CDx PMA lifecycle and NMPA device registration seats).
- 02
Component task families + end-to-end engineer seats
Table & tolerance checks · full submission deliverablesComponent families test isolated steps: matching IEC 60601 / ISO 10993 test clauses to design inputs, recomputing PPA/NPA confidence intervals from raw line data, or catching unclosed software anomalies. End-to-end seats grade the full role: a Regulatory Affairs engineer drafting a deficiency response or a Quality lead auditing a Design Traceability Matrix before submission.
- 03
Co-built with your internal specialists & our Fellows
Internal RA/QA leads + external CDRH / MDR / NMPA practitionersYour regulatory and quality engineers define house templates and risk gates; our practitioner Fellows author the coupled requirements and adjudicate runs.
- 04
Offline benchmarks & online production evaluation
Model & harness comparison · continuous production scoringCompare models, PDF/table parsers, and harnesses on a frozen offline benchmark before release, then monitor and sample live authoring and review runs in production.
Post-training, off-the-shelf data & strategy
Medtech AI teams and frontier labs license our device and companion-diagnostic environments, coupled rubrics, and reference deliverables for RL, SFT, and pre-release benchmarks.
- 01
Architecture diagnosis & deployment strategy
Harness vs. model attribution · autonomy gatingEvaluation tells you whether a miss came from retrieval, context management, tool design, or base-model reasoning, and which workflows are ready for production versus human-in-the-loop gates.
- 02
RL environments & coupled verifiers
Isolated sandboxes · partial-credit rewardResettable company and project sandboxes where grading runs in a separate container. Coupled requirements pair each finding with its governing conclusion so rollouts cannot game partial credit.
- 03
SFT deliverables, trajectories & off-the-shelf corpora
Gold outputs · step-by-step traces · licensed packsPractitioner-written reference deliverables, adjudicated agent trajectories, and licensed corpora of filings, validation packages, and project submittals for supervised fine-tuning and pre-release benchmarks.
Inside a device & CDx seat
Biopharma Bench v0.2 · CDx & Device Registration (FCL / DVN)
On a CDx filing or a device deficiency response, the summary record and the primary record disagree. The deliverable has to follow the primary record.
Our companion-diagnostic environment (FoundationOne CDx PMA P170019 lifecycle) and medical-device registration environment (a Sichuan Class II video laryngoscope registration answering supplementary-information notice 械一（2025）337号 and an on-site QMS inspection) require agents to reconcile each claim against the test reports, verification tables, and inspection records on the desk.
Record on desk
CDx copy-number & rearrangement calling
Surface reading
94.3% CNA and 100% rearrangement PPA against FoundationOne LDT read as standalone verification
What the primary record establishes
Standalone orthogonal verification of CNA and rearrangements was reserved and not executed for this freeze
Record on desk
CDx precision study
Surface reading
19 itemized samples vs. 18 analyzed reported as an open discrepancy
What the primary record establishes
One sample excluded for contamination (1.9% against the < 1% limit); 18 analyzed is the correct count
Record on desk
QMS inspection NC-5 (film sealer validation)
Surface reading
Report concludes the 4.5–7 m/min range is confirmed
What the primary record establishes
Tested setting was 4.8 m/min, and the FRB-770 analog dial cannot be set or repeated precisely at 4.5
| Record on desk | Surface reading | What the primary record establishes |
|---|---|---|
| CDx copy-number & rearrangement calling | 94.3% CNA and 100% rearrangement PPA against FoundationOne LDT read as standalone verification | Standalone orthogonal verification of CNA and rearrangements was reserved and not executed for this freeze |
| CDx precision study | 19 itemized samples vs. 18 analyzed reported as an open discrepancy | One sample excluded for contamination (1.9% against the < 1% limit); 18 analyzed is the correct count |
| QMS inspection NC-5 (film sealer validation) | Report concludes the 4.5–7 m/min range is confirmed | Tested setting was 4.8 m/min, and the FRB-770 analog dial cannot be set or repeated precisely at 4.5 |
Cross-record checks taken from the FCL-P-03, FCL-P-05, and DVN-P-02 rubrics.
- 01Documents that standalone orthogonal verification of copy-number and rearrangement calling was reserved and not executed, so that support rests on the LDT comparison, precision, and tumor-purity studies.
- 02Reconciles the CDx precision count to 18 analyzed samples by attributing the difference from 19 itemized samples to one contamination exclusion.
- 03Limits FFPE slide stability to the 6 months on file, with the 12- and 15-month timepoints still open.
- 04Names the NC-5 mechanism: the sealer validation tested 4.8 m/min but concluded the full 4.5–7 m/min range.
- 05Describes NC-5 and NC-7 rectification as pending, since the rectification tracker still shows both as not started, rather than claiming completed fixes.
Benchmarks
Medical device and companion-diagnostic assignments are scored alongside drug programs in Biopharma Bench v0.2.
- Biopharma Bench v0.2 leaderboardIncludes FoundationOne CDx (PMA P170019) & NMPA device registration seats→
- Introducing Biopharma Bench v0.2Expanded device registration, CGT, and GMP inspection coverage→
- Eval: inspect runs, deliverables, and requirementseval.raycaster.ai/benchmarks/biopharma-bench↗
- Biopharma deployment overviewCMC, quality, clinical, and drug-submission workflows→
Next step
Tell us what you need to deploy, evaluate, or train.
Workflow deployment, offline and online evaluation frameworks, post-training environments and data, or paid review as an AI Fellow.