This month our Journal Club group met to discuss Multiple Mini Interviews as the final stage of application assessment into the Bachelor of Science in Nursing program. Due to popular demand and a high number of faculty members who were unable to attend this session, we will be repeating this Journal Club topic at our next meeting on MONDAY June 17th at 12:00. If you would like to attend this meeting, please RSVP to Allana.hearn@alumni.ubc.ca.
The paper under discussion was:
Perkins, A., Burton, L., Dray, B., & Elcock, K. (2013). Evaluation of a multiple-mini-interview protocol used as a selection tool for entry to an undergraduate nursing programme. Nurse Education Today, 33(5), 465–9.
This article is available here.
MMIs at the UBC School of Nursing
In Spring 2008, the UBC School of Nursing replaced traditional panel-style interviews for Mini Multiple Interviews. The UBC School of Nursing application process is extensive and thorough. The first stage includes an official transcript and a supplemental application. Once an applicant’s transcript and supplemental application are assessed, they may be invited for an in-person MMI interview, and final selection is made on the basis of interview performance. Each year the school receives between 400 and 500 applications, interviews 190-200 applicants and admits 120.
In 2009, the School streamlined the MMI process down from ten in 2008 to six interview stations for practical reasons. Faculty members and other stakeholders such as former patients and patient advocates form the members who conduct the interviews. Since 2009 the process has remained more or less the same with annual amendments to the questions. These questions use scenarios that relate to knowledge of the healthcare system, ethical decision making, critical thinking, evidence-based practice and shared decision makin, effective communication, and empathy and compassion.
Discussion: MMIs – A Worthy Assessment Tool?
A benefit of MMIs is that the candidate is exposed to a variety of interviewers improving validity of assessment. There are six interviewers (instead of just two or three as the UBC School of Nursing used previously). Interviewers independently rank applicants and their scores are tallied at the end of the day. If there is a personality clash between interviewer and interviewee or if an applicant responds to a particular MMI scenario in a way badly, the ranking that results would only count for 1/6 of the applicant’s total ranking. The resulting reduction in bias is of benefit to both the applicants and the School of Nursing. Although MMIs cannot screen all potential red flags, perhaps they are able to do so better than other assessment techniques.
We discussed whether or not the MMI method is able to assess an applicant’s ethical judgement, communication skills and integrity more than other traditional interviews. In the past, the UBC School of Nursing conducted panel interviews asking applicants to describe situations where they used skills such as conflict resolution or leadership skills. Each interview was about 30 minutes and was conducted with two interviewers.
In MMIs we are asking applicants to explain what they would do in these situations, rather than how they have dealt with similar situations. Perhaps these MMIs allow applicants to tell how they would like to react, rather than how they have reacted in the past and therefore likely would react in the future. Nevertheless, some human resource and selection theories suggest that applicants will provide a more authentic answer if you ask them to provide examples from their own life and experiences (rather than conjecture their own actions in a “what-if” scenario. The former traditional style of interviews may have the benefiit of eliciting these types of answer
The UBC School of Medicine uses ten interviewers. Six interviewer stations are used in the School of Nursing as the interview process can be completed within an hour and it uses 40% fewer resources. It was suggested that perhaps we could keep the number of stations at six, but use the extra time that 4 more stations would have required to assess the applicants in another manner. In the past, an English test was conducted and a math test has also been suggested.However, exploring those options, or adding another form of personality testing would increase the resources required significantly.
In terms of outcomes at the UBC School of Nursing our outcomes are excellent. We have very low attrition rates and very high completion and CRNBC exam pass rates. However, it is hard to say how much of this is due to the MMI process but it seems that in general applicant assessment seems effective in helping us select bright and dedicated student nurses. However, many School’s in Canada do not intrerview at all, and the process is very resource intensive.
Appraisal of the Paper
Perkins et al. evaluated the MMI applicant interviewing process introduced at Kingston University School of Nursing. They administered a basic 3-point questionnaire to both interviewees and interviewers about their perceptions of the MMI experience. We questioned the quality of the research due to this perhaps over simplistic questionnaire. A 3-point attitude continuum is a bit basic. Furthermore, it was unclear how the unsolicited comments were obtained.
In the article the author describes MMIs as being an efficient way to assess applicants on non-cognitive skills like integrity, ethical judgement and empathy. Although we all agreed that MMIs are an efficient way to assess these skills, we found it hard to categorize them as non-cognitive. Yes, they are non-technical nursing skills, but these soft skills often require a high level cognitive processing.
Additional Resources
To access the Powerpoints for this workshop click on the folloeing link: EKM Journal Club 3
