Abstract

Test-Retest Reliability of Assessment of Anaesthetist�¢����s Performance in Managing Serious Adverse Events on a Screen-Based Computer Simulator

Serious adverse events (SAE) during anaesthesia procedure are uncommon but potentially fatal. Traditional methods to assess the performance of anaesthetists in SAE management are largely subjective and unreliable. In this study, we assessed the test-retest reliability of the assessment of performance of anaesthetists in managing SAE during anaesthesia procedure by determining if different observations assessing the same clinical scenario on a computer screen-based anaesthesia simulator would have agreement on the rating of the anaesthetist’s performance. Methods: Three SAE scenarios on a computer screen based simulator were used for assessment. From a pilot study, the minimum number of trials beyond which there is little or no improvement of performance was determined for each SAE scenario. Six fresher anaesthesia postgraduates were enrolled. Each participant undertook ten trials for each SAE scenario and submitted the trial-logs in pdf format to the investigator. SAE detection latency, initial treatment latency period, SAE treatment completion period was collected from the simulator trial-logs. Test-retest reliability (internal consistency) between trials was assessed with Pearson’s Product Moment Correlation Coefficient. Scatter plot of improvement of latency period and treatment time versus the number of simulator trials were used to characterize the learning curves. Regression model fitting of the learning curve was used to find the best fit model for the learning curve. Results: There was significant test-retest reliability and internal consistency between the trials for each of the three SAE scenarios. Conclusion: PC based simulated SAE scenarios are effective and reliable for the assessment of performance of fresher anaesthesia postgraduates.

• PDF