OSCE Redesign for the Clinical Assessment of Skills and Competencies (CASC)
Introduction
Consideration has been given to the chosen format for OSCEs geared towards the new CASC exam. The current format of the OSCEs available on the site adopts the scoring method used by North American institutions and permits calculation of the pass mark for the exam as well as reflecting a truly objective method of assessment.
However, the scoring scheme does not reflect that used by the Royal College and there are advantages in using the approach currently adopted by the exam.
The following information is comprised of information provided by the college on the exam, combined with some educated guesses and some details from those involved in the development of the exam. Please note that I have no way of knowing the content of the exam!
The expected standard for the CASC
Since candidates will be taking the CASC after 30-48 months of training, the standard is that expected of an ST2-3. In this sense, the 'age' of the candidate is one to two years later than the current Part I OSCE exam and higher standards will be expected. The constructs of each OSCE are likely to test more complex reasoning and management skills and may combine a variety of skills in a single OSCE.
Format of the MRCPsych OSCEs
Each OSCE will have a number of components:
- The OSCE 'construct'
- Instructions for the candidate
- Instructions for the role player
- Marking sheet
- Examiner's Briefing Notes
- List of equipment needed
As a candidate, you will only be given #2. However, the construct and the examiner's briefing notes will be very relevant to how you should approach the station. The construct indicates what the purpose of the station is and will probably reflect one of the general categories for OSCE stations (e.g. communication/ examination skills/ etc.) For example, an OSCE on assessment and risk assessment will describe the purpose of the OSCE as: "Candidates are expected to demonstrate an ability to assess risk to self and others from a competent assessment of mental state and relevant social factors".
Examiners are given information on how to rate performance in the additional 'Briefing Notes' which will give additional information on the construct of the station. The briefing notes may explain the areas that the candidate is expected to cover and are particularly relevant for examiners who may be assessing OSCEs out with their own specialty area.
Scoring MRCPsych CASC OSCEs
The approach currently used in the MRCPsych Part I OSCE and the one likely to be continued in the new CASC exam uses an A (excellent) - E (Severe Fail) rating for a variety of 'domains', one of which is always going to be 'Communication'. The weighting for each domain will vary according to the type of OSCE. Examiners will be blind to the weighting, and if the examiners are blind, you can be pretty sure that you will be as well!
The Part I OSCE example marking sheet (PDF, 11Kb) is no longer available on the College website but you can download it from this site. There is no reason to believe that assessment of candidate performance will have changed significantly and some have suggested that a non-objective assessment schedule is one reason why the exam has been renamed from an OSCE to the CASC.
There will also be a global rating (also marked A-E) which is intended to be an indication of the extent to which the candidate met the OSCE objectives/ construct rather than an 'average' of all the domains. It is therefore possible that the construct aims can be met whilst performing badly on specific domains. However, the weighting of particular domains still means that you will probably have to do reasonably well on most heavily-weighted domains as well as achieve the construct aims.
Redesigning the marksheet for trickcyclists.co.uk OSCEs
In order to address a variety of goals, one of the most important being an educational objective rather than mirroring the CASC exam, the marksheet will be redesigned as follows:
- Five domains for each station will be chosen, the first of which will be communication.
- Weighting (totalling 100%) will be applied to each domain.
- The examiners' Briefing Notes will make specific suggestions on skills/ objectives to be demonstrated for each domain.
- Whilst marking sheets will adopt an A-E scheme for rating, a numerical score will also be applicable.
- A total score of 100 for the station is determined.
Calculating the pass mark
To calculate the pass mark for OSCEs from trickcyclists.co.uk, the following method is chosen:
- The mean score of all students scoring a global rating of 'C' (equivalent to 'borderline pass' on the old schem) on each station becomes the pass mark for this station
- The mean scores on all stations are summed and this becomes the pass mark for the overall examination
Exam Blueprints
Usually, in order to ensure that there is a good balance of questions, a blueprint needs to be designed. An example blank one can be found here. The idea is that you place ticks in the respective boxes, so that you have questions that test your key areas (i.e. history taking, practical skills, etc.) and that your main learning areas (i.e. depression, anxiety, psychosis, etc.) are all covered as well.
It is very likely that the CASC exam will be focusing on a much narrower range of skills than the Part I OSCE and the expectation will be that more 'basic' skills and competencies will have been tested in the workplace-based assessments.
To receive information about new, or updated OSCEs, please subscribe to the mailing list:
