Test Method Validation for Visual Inspection of Software Setup in Heart-Lung Machine Production
A cardiovascular device manufacturer required validated evidence that production operators could reliably distinguish between acceptable and unacceptable software settings and data entry during Heart-Lung Machine (HLM) production. The challenge involved validating visual inspection methods across multiple software setup categories including function selection, general verification, manual operation, data entry, and password entry. Critical concerns included ensuring operator repeatability and reproducibility across varying software interfaces, preventing software configuration errors that could impact device safety, and maintaining compliance with medical device software validation requirements.
Project Metrics
Tools & Standards
Problem Definition
A cardiovascular device manufacturer required validated evidence that production operators could reliably distinguish between acceptable and unacceptable software settings and data entry during Heart-Lung Machine (HLM) production. The challenge involved validating visual inspection methods across multiple software setup categories including function selection, general verification, manual operation, data entry, and password entry. Critical concerns included ensuring operator repeatability and reproducibility across varying software interfaces, preventing software configuration errors that could impact device safety, and maintaining compliance with medical device software validation requirements.
System Constraints
Engineering Decisions
Developed 5 software setup categories: Function Selection, General Verification, Manual Operation, Operator Data Entry, Password Entry
Created 30 test samples (18 'True-Pass', 12 'True-Fail') using photos and GUI illustrations
Implemented MRD ranking system (1-3 scale) to assess inspection difficulty
Selected worst-case samples covering all MRD ratings and software categories
Used randomized presentation order to minimize learning effects
Conducted testing over two days to simulate real-world inspection variability
Required operators to review all applicable work instructions before testing
Validation Strategy
- All 'True Pass' units (18 samples × 2 operators × 2 trials = 72 evaluations) correctly identified as Pass
- All 'True Fail' units (12 samples × 2 operators × 2 trials = 48 evaluations) correctly identified as Fail
- Reproducibility AC1 statistic: 1.0000 (p < 0.0001) - perfect agreement between operators
- Repeatability for Operator 1: 1.0000 (p < 0.0001)
- Repeatability for Operator 2: 1.0000 (p < 0.0001)
- Stability demonstrated across two-day testing interval
- System suitability confirmed through consistent operator performance
Risk & Mitigation
Final Outcome
Successfully validated visual inspection methods for software setup across all five categories with perfect operator agreement. Achieved 100% repeatability and reproducibility, demonstrating that trained operators can reliably detect software configuration errors. The validation covered 22 technical reviews and established a robust inspection process for HLM production, providing documented evidence of operator capability and process reliability for regulatory compliance.