Context: The National Board of Osteopathic Medical Examiners uses the Osteopathic Manipulative Treatment (OMT) Global Rating Tool to score candidate performance in OMT during standardized patient encounters. To the authors' knowledge, no validity research has been published on the underlying constructs measured by this instrument to date.

Objective: To evaluate whether the OMT Global Rating Tool measures 1 holistic skill or multiple skill sets.

Methods: Structural equation models were fit to data from first-time test-takers who took the Comprehensive Osteopathic Medical Licensing Examination-USA Level 2-Performance Evaluation (COMLEX-USA Level 2-PE) between July 7, 2014, and April 30, 2015. Two theoretical models were evaluated: a model that posited a single underlying skill set, and a model that posited 2 related but distinct skill sets.

Results: The 1-factor and 2-factor models were fit to data from 4673 third- and fourth-year osteopathic medical students. Fit statistics indicated that the data were best fit by a model representing 2 factors: pre- and post-OMT technique assessment and OMT technique (r=0.47).

Conclusion: The OMT Global Rating Tool seems to measure 2 distinct but related skills. Although the results of this study are applicable to broad populations and cannot be used to provide skill-specific subscores, it might be helpful to remediation efforts to target pre- and post-OMT technique assessments and OMT techniques as distinct areas for intervention.