Archives: assessment

Disability and accommodations

CDBC/BCAAN Education Bite – originally emailed June 21, 2024

Purpose of accommodations in education

Accommodations enable the student to access the curriculum (including exams) without interference from the disability. Accommodations do not necessarily ensure success or the ability to perform to one’s potential (Lovett & Lewandowski, 2015). Schools have policies in place to make sure that students from all backgrounds have equal opportunities and access to education.

Disability diagnosis, accommodations, and the “seven sins of clinicians” (Harrison & Sparks, 2022)

  1. Bias and conflict of interest – Be aware of the tendency to search for and favour evidence to support prior hypotheses; there is a higher risk when you have a dual relationship (e.g., you are a treating psychologist doing an assessment).
  2. Naivety – Include objective review + self/collateral report, use performance validity testing and symptom validity testing.
  3. Base rate blindness – When a condition is rare, first consider more obvious possibilities.
  4. Cherry-picking – Do not choose to focus on one or more low subtest scores; variability is the norm. For those in the CDBC/BCAAN network, check out a webinar on this topic.
  5. Counting apples as oranges – Cognitive processing speed alone does not predict performance on academic tests. It is best to use performance on timed academic measures.
  6. Psychometric slight of hand – Average scores are not evidence of disability, even with superior abilities in other areas.
  7. Blissful ignorance – Be aware of the research, for example on when and how much extra time is reasonable.

Extra time – evidence based recommendations

Harrison, Pollock and Holmes (2022) reviewed the literature around evidence for extra time in the post-secondary setting and concluded that the evidence suggested extra time being appropriate for students with learning disabilities (with 25% typically sufficient to equalize access), while those with ADHD did not benefit from extra time. A “stop the clock” type of accommodation was thought to be more useful for those with ADHD. There was not sufficient research to make conclusions about extra time in students with Autism.

For students mental health conditions such as anxiety, the authors said, “there must be a clear link between the functional impairments that arise from the disorder and a need for additional time for tests and examinations. For example, is the student reporting a tendency to freeze and panic in test situations, which they claim is impairing their ability to complete the test in the typical time allocated? In such cases, might it make more sense to recommend stop-the-clock breaks to allow the student to implement anxiety reduction strategies rather than simply remaining longer in an anxiety-producing situation?”

Canadian/BC resources

Recommended listening

For a great podcast on biases and errors in decision making (including ways we can do better), check out Dr. Stephanie Nelson the Testing Psychologist Podcast.

References

Lovett, B. J., & Lewandowski, L. J. (2015). Testing accommodations for students with disabilities: Research-based practice. Washington, DC: American Psychological Association.

Harrison, A.G., Pollock, B., & Holmes, A. (2021). Provision of extended time in post-secondary settings: A review of the literature and proposed guidelines for practice. Psychological Injury and the Law, 15, 295-316. https://link.springer.com/article/10.1007/s12207-022-09451-3

Harrison, A.G., Pollock, B. & Holmes, A. (2022), Provision of Extended Assessment Time in Post-secondary Settings: A Review of the Literature and Proposed Guidelines for Practice. Psychological Injury and the Law, 15, 295–306. https://doi.org/10.1007/s12207-022-09451-3

Harrison, A.G. & Sparks, R. (2022). Disability diagnoses: Seven sins of clinicians. Psychological Injury and Law, 15, 268-286. https://doi.org/10.1007/s12207-022-09449-x

Composite score extremity effect: An explanation

CDBC/BCAAN Education Bite – originally emailed April 12, 2024

I hope this could be helpful for trainees, or those who like a review of statistical concepts. 

It seems counter-intuitive when index scores of 70, 71, and 72 lead to an overall composite score of 67. This has been called the composite score extremity effect. This effect accounts for the fact that imperfectly correlated scores are always more extreme (further from the population mean) than if you were to simply average the scores. As explained in Schneider (2016) –

Composite scores are more extreme when:

  1. The composite is made up of a higher number of individual scores
  2. The individual scores have a lower correlation with each other
  3. The individual scores are farther away from the mean

Schneider explains this in simple terms: “though it is unusual to have a particular deficit, it is even more unusual to have that deficit and several more. A composite score that summarizes all of these deficits would have to take this comparative rarity into account. It is for this reason that a composite score that consists of many low scores is lower than the average of those scores” (p. 8)

An important concept in this is regression to the mean. For a composite comprised of two similar subtests, where there is a low score on subtest A (70), the predicted score for subtest B might 78, due to regression to the mean. If the person scored 70 on both A and B, the composite would be under 70, as it is unusual for people to be equally extreme on both subtests. The technical/mathematical explanation can be found here starting on page 10.

References

Barnett, A.G., van der Pols, J.C., & Dobson, A.J. (2005). Regression to the mean: what it is and how to deal with it, International Journal of Epidemiology, 34(1), 215–220. https://doi.org/10.1093/ije/dyh299

Schneider, W. J. (2016). Why Are WJ IV Cluster Scores More Extreme Than the Average of Their Parts? A Gentle Explanation of the Composite Score Extremity Effect (Woodcock-Johnson IV Assessment Service Bulletin No. 7). Itasca, IL: Houghton Mifflin Harcourt.

DSM-5 Revisions to ASD definition

Two small wording changes for clarity have been made in the revision of the DSM-5 (DSM-5-TR) entry for ASD released March 2022.

Quoting from the spectrum news article from March 17, 2022 (https://www.spectrumnews.org/news/dsm-5-revision-tweaks-autism-entry-for-clarity/)

The DSM-5, released in 2013, indicated that an autism diagnosis requires “persistent deficits in social communication and social interaction across multiple contexts, as manifested by the following”: deficits in social-emotional reciprocity, in nonverbal communicative behaviors used for social interaction, and in developing, maintaining and understanding relationships. The first text revision in the new DSM-5-TR adds two words to that description: “as manifested by all of the following.”

The second change swaps out a single word describing the “specifiers” that can accompany an autism diagnosis. Whereas the DSM-5 wording instructs clinicians to specify if a person’s autism is “associated with another neurodevelopmental, mental or behavioral disorder,” the DSM-5-TR version reads: “associated with a neurodevelopmental, mental, or behavioral problem.” It still instructs clinicians to use additional diagnostic codes whenever appropriate, but it no longer requires specifiers to be diagnosable conditions.

Changes have also been made to Intellectual Disability. The new terminology is “intellectual developmental disorder (intellectual disability)”.

Please see the APA fact sheets for more information: https://www.psychiatry.org/psychiatrists/practice/dsm/educational-resources/dsm-5-fact-sheets