Introduction Instrumental deliveries account for 10–13% of births in the UK. Achieving competence in vacuum delivery is a mandatory requirement of the RCOG core curriculum. OSATS are the current tool used for formative and summative work based assessment of practical skills. Performance of each subtask is classified as competent or ‘needs help’. The level of expertise for each subtask is not individually defined limiting the detail of the feedback for formative assessment.

Aim To ascertain the inter-rater reliability of a new formative assessment tool where the construct validity has already been established.

Methods 22 videos of vacuum delivery conducted by obstetricians of varying grade were reviewed independently by two assessors. The formative assessment tool was completed for each video recording. There were a total of 12 subtasks, each scored between 1 and 5.

Results The scores for each video ranged from 39 to 60. The total scores for each video recording were compared using the Cohen Kappa co-efficient, with a resulting co-efficient of 0.62 (SE 0.08, 95% CI 0.44 – 0.78) indicating strong reliability.

Discussion Reduction in training hours has made experiential learning more challenging and therefore effective training and assessment tools are increasingly important. This new tool aims to give more detailed feedback and provide a thorough assessment of the necessary steps in performing a competent vacuum delivery.

We believe this tool shows promise as an alternative to the current standard of assessment.