To evaluate the psychometric quality of multiple-choice question (MCQ) examinations across four core courses in a Diploma in Optometry program at the University of Bisha, Saudi Arabia, with a focus on item difficulty, discrimination, distractor efficiency, and test reliability.

Methods: This retrospective descriptive study analyzed item performance data from final MCQ exams in four core optometry courses (Geometric Optics and Refraction, Ocular Anatomy and Physiology, Clinical Optometry Practice, and Ocular Diseases). For each item, we calculated the difficulty index (P-value), point-biserial discrimination index, and number of non-functional distractors (NFDs; distractors chosen by <5% of students). Items were categorized by difficulty (easy, acceptable, difficult) and discrimination (excellent, good, acceptable, poor) using established cut-offs. Distractor efficiency was classified according to the number of NFDs per item (0–3). Internal consistency reliability for each exam was estimated using Cronbach’s alpha (Kuder–Richardson 20). Descriptive statistics summarized item metrics, and Pearson correlation analysis examined the relationship between item difficulty and discrimination.

Results: A total of 200 MCQs from four exams (40–60 items per course; 28–30 students per exam) were analyzed. Mean difficulty across exams ranged from 55.3% to 75.2%, with 14% of items classified as easy (P ≥ 0.78), 10% as difficult (P < 0.25), and 76% as acceptable. Mean discrimination indices ranged from 0.22 to 0.35, and exam reliability was acceptable to good (Cronbach’s α = 0.76–0.89). Overall, 29% of items showed excellent discrimination (≥0.35), 23% good, 18.5% acceptable, and 29.5% poor, including ~1.5% with negative discrimination. Regarding distractor efficiency, 34% of items had no NFDs, 35% had one NFD, 21.5% had two NFDs, and 9.5% had three NFDs. Combining indices, 55% of items were classified as good, 35% as needing revision, and 10% as poor. A significant negative correlation was observed between item difficulty and discrimination (r ≈ –0.32), with moderately difficult items demonstrating the highest discrimination.

Conclusion: The analyzed exams demonstrated generally acceptable psychometric properties, with most items falling within recommended ranges for difficulty and discrimination and all tests showing good internal consistency. However, a substantial proportion of items contained multiple non-functional distractors or suboptimal discrimination, particularly in the basic science course. Routine item analysis and systematic revision of flawed items are recommended to enhance MCQ quality, support fair and reliable assessment, and strengthen the optometry diploma curriculum.