Senior Psychometrician & Research Lead

• Posting Title: Senior Psychometrician & Research Lead
• Department: Assessment Services
• Employment Type: Full-Time
• Minimum Experience: Experienced
• Work Environment: Remote eligible (with up to 10% travel)

Position Summary

The Senior Psychometrician and Research Lead provides high-level technical expertise in psychometric analysis and applied research to support ABEM’s high-stakes medical certification examinations. This dual-focus role manages psychometric services across the assessment lifecycle while simultaneously leading research initiatives—including study design, data analysis, and manuscript preparation—to ensure the validity, fairness, and reliability of physician certifications.

Essential Duties & Responsibilities

• Assessment Strategy: Collaborate with the Principal Psychometrician to guide the design, implementation, and analysis of innovative examination processes.
• Technical Operations: Develop and execute computer programs for examination scoring, equating, and item analysis; review and interpret complex results.
• Quality Control: Conduct analytical studies to implement methodologies for improving examinations and measuring the effectiveness of process innovations.
• Equitable Assessment: Review and interpret Differential Item Functioning (DIF) analyses and lead standard-setting activities.
• Data Stewardship: Provide psychometric support for data exchanges with vendors and other medical boards, maintaining organized and secure data banks.

Research Responsibilities

• Study Leadership: Design and execute research studies using ABEM’s physician data, focusing on health equity, clinical performance metrics, and public health.
• Evidence Generation: Identify and access external datasets (e.g., CMS, ACGME) to complement internal resources.
• Scientific Contribution: Support the preparation and submission of research manuscripts for peer-reviewed journals as a co-author.
• Stakeholder Communication: Translate complex psychometric findings into accessible formats for the Board of Directors and internal stakeholders.

Minimum Qualifications

• Education: Doctoral degree (Ph.D. or Ed.D.) in Psychometrics, Educational Measurement, Quantitative Social Science, or a related field.
• Experience: Minimum of 7 years of professional experience encompassing both psychometric practice and applied research.
• Technical Skills: Proficiency in R, or SAS software.
• Specialized Knowledge: Demonstrated experience with standard setting, equating, and DIF analyses.
• Communication: Proven record of peer-reviewed publications and the ability to present complex data to non-technical audiences.

Preferred Qualifications

• Advanced expertise in assessment design and healthcare research.
• Familiarity with accreditation standards and medical specialty certification.