Year: 2026 | Month: April-June | Volume: 11 | Issue: 2 | Pages: 69-75
DOI: https://doi.org/10.52403/gijhsr.20260208
The Pocket Tutor - Generating Competency-Based Clinical Vignettes with Artificial Intelligence for Medical Postgraduate Competitive Exam Preparation
Swapnil Banerjee1, Manisha Agarwal2
1Final Year MBBS Student, Dr. KNS Memorial Institute of Medical Sciences, Barabanki, IND
2Department of Obstetrics & Gynaecology, Dr. KNS Memorial Institute of Medical Sciences, Barabanki, IND
Corresponding Author: Swapnil Banerjee
ABSTRACT
The transition of medical licensing examinations toward application-based clinical vignettes necessitates high-quality question banks. However, manual drafting of complex multiple-choice questions places a profound cognitive burden on educators. This study evaluates the efficacy of constrained Large Language Models to generate standardized, high-yield medical assessments. A cross-sectional, dual-cohort study was conducted involving undergraduate medical students (n=230) and senior medical faculty (n=32). A constrained prompt was engineered using Gemini 3 Pro to generate competency-based clinical vignettes. Participants evaluated the AI-generated content via digital surveys. A blinded Turing Test, embedding authentic past year questions among AI modules, assessed indistinguishability. Expert faculty rated clinical accuracy highly (Mean=4.38±0.71), with 96.9% certifying the content as clinically safe. The student cohort reported strong exam parity, with 81.3% finding the AI difficulty aligned with standard examinations. In the Turing Test, 75.0% of faculty and 47.0% of students could not distinguish AI-generated vignettes from human-authored questions (p=0.005). Furthermore, 90.0% of students desired to integrate the tool into exam preparation, while 93.8% of faculty considered it a viable drafting aid. Highly constrained artificial intelligence can successfully architect structurally sound, competency-based clinical assessments. By passing a clinical Turing Test among educators, this methodology serves as a reliable mock-testing tool for trainees and a time-saving resource for medical faculty.
Keywords: Artificial intelligence, Competency-based assessment, large language models, medical education, Undergraduate training.