Author Archives: jengle

Composite score extremity effect: An explanation

CDBC/BCAAN Education Bite – originally emailed April 12, 2024

I hope this could be helpful for trainees, or those who like a review of statistical concepts. 

It seems counter-intuitive when index scores of 70, 71, and 72 lead to an overall composite score of 67. This has been called the composite score extremity effect. This effect accounts for the fact that imperfectly correlated scores are always more extreme (further from the population mean) than if you were to simply average the scores. As explained in Schneider (2016) –

Composite scores are more extreme when:

  1. The composite is made up of a higher number of individual scores
  2. The individual scores have a lower correlation with each other
  3. The individual scores are farther away from the mean

Schneider explains this in simple terms: “though it is unusual to have a particular deficit, it is even more unusual to have that deficit and several more. A composite score that summarizes all of these deficits would have to take this comparative rarity into account. It is for this reason that a composite score that consists of many low scores is lower than the average of those scores” (p. 8)

An important concept in this is regression to the mean. For a composite comprised of two similar subtests, where there is a low score on subtest A (70), the predicted score for subtest B might 78, due to regression to the mean. If the person scored 70 on both A and B, the composite would be under 70, as it is unusual for people to be equally extreme on both subtests. The technical/mathematical explanation can be found here starting on page 10.

References

Barnett, A.G., van der Pols, J.C., & Dobson, A.J. (2005). Regression to the mean: what it is and how to deal with it, International Journal of Epidemiology, 34(1), 215–220. https://doi.org/10.1093/ije/dyh299

Schneider, W. J. (2016). Why Are WJ IV Cluster Scores More Extreme Than the Average of Their Parts? A Gentle Explanation of the Composite Score Extremity Effect (Woodcock-Johnson IV Assessment Service Bulletin No. 7). Itasca, IL: Houghton Mifflin Harcourt.

IQ Stability – Research review

CDBC/BCAAN Research Bite: Originally emailed December 15, 2023

Understanding long-term stability of IQ scores is essential, especially when we are making decisions about diagnoses expected to be stable over time (into adulthood).  Watkins et al., (2022) found that with an approximately 2 ½ year interval, only the VCI, VSI and FSIQ were stable enough for normative comparisons, while unusual differences between subtests or index scores were unlikely to be repeated at retest. Previous research has shown stability in IQ scores for those with intellectual disabilities (Whitaker et al., 2008). Eichelberger and colleagues (2023) showed that those with lower-end IQ scores (below 100) were less likely to be stable over time. This suggests we need to be particularly cautious about diagnosing intellectual developmental disorder in preschool children with “borderline” type profiles. There is also evidence that IQ scores are more dynamic in individuals with autism (Prigge et al., 2022).

References

Watkins, M. W., Canivez, G. L., Dombrowski, S. C., McGill, R. J., Pritchard, A. E., Holingue, C. B., & Jacobson, L. A. (2022). Long-term stability of Wechsler Intelligence Scale for Children-fifth edition scores in a clinical sample. Applied neuropsychology. Child, 11(3), 422–428. 

https://doi.org/10.1080/21622965.2021.1875827

Whitaker, S. (2008). The Stability of IQ in People With Low Intellectual Ability : An Analysis of the Literature. Intellectual and Developmental Disabilities, 46(2), 120–128. http://eprints.hud.ac.uk/id/eprint/4283/1/stability_of_IQ_%282008%29.pdf

Eichelberger, D. A., Latal, B., Kakebeeke, T. H., Caflisch, J. A., Jenni, O. G., & Wehrle, F. M. (2023). The influence of preschool IQ on the individual-order stability of intelligence into adulthood. Acta Paediatrica, 112(10), 2161–2163. https://doi.org/10.1111/apa.16925

Prigge, M. B. D., Bigler, E. D., Lange, N., Morgan, J., Froehlich, A., Freeman, A., Kellett, K., Kane, K. L., King, C. K., Taylor, J., Dean, D. C., King, J. B., Anderson, J. S., Zielinski, B. A., Alexander, A. L., & Lainhart, J. E. (2022). Longitudinal Stability of Intellectual Functioning in Autism Spectrum Disorder: From Age 3 Through Mid-adulthood. Journal of Autism and Developmental Disorders, 52(10), 4490–4504. https://doi.org/10.1007/s10803-021-05227-x

 

 

The science of literacy – Recommended reading

CDBC/BCAAN Research Bite: Originally emailed February 9, 2024

Book recommendation: Language at the Speed of Sight. The author, Mark Seidenberg, is a major advocate for translating the science of reading into education. He explains things such as the science behind the “Cambridge hoax” where we can easily read words with mixed up letters. Try some of his reading demos. Listen to Seidenberg talk about the science of reading in a video on his website. A couple of important points:

  1. Language development is essential for reading development.
  2. Explicit instruction is necessary to scaffold reading, but implicit learning plays an important role, and implicit learning speeds up the process of learning to read.
  3. Phonemic awareness develops in conjunction with learning about print; it is a result of being a skilled reader, not a precursor. He details this here,  also available in a recording.
  4.  To show the impact of reading on phonological awareness, he points to research which shows that when asked to say “sword” without the “s” good readers often say “word.” He makes the point that phonological awareness training should be paired with print/letters.

For more high quality information on the science of reading, check out – Reading Rockets: “A national public media literacy initiative offering information and resources on how young kids learn to read, why so many struggle, and how caring adults can help.”

Executive functioning intervention – Research review

CDBC/BCAAN Research Bite: Originally emailed March 8, 2024

Adele Diamond’s group published a systematic mega-review of 179 studies up to 2015 (Diamond & Ling 2019) on intervention to improve EFs. Importantly, this paper required each study to have at least one behavioural measure, include a measure of generalization, include a control group, and not be purely correlational.

Mindful movement practices (e.g., taekwondo & t’ai chi) showed the strongest results for improving EFs. Mindful movement exercises did better than more sedentary mindful practices. 30-40 minute sessions were more helpful than those less than 30 minutes. School-based programs such as Promoting Alternative Thinking Strategies, the Chicago School Readiness Project, and Tools of the Mind came in second, and were particularly good for improving inhibitory control. Both types of approaches showed better results than computerized cognitive training (e.g., CogMed).

Some key principles the paper summarized:

  1. EF can be improved.
  2. Generalization to untrained tasks usually occurs for similar tasks, but rarely improves on unpracticed tasks. Broader transfer effects happen with practice of a wider range of skills.
  3. EF training must include continued challenge, where the task adapts to keep challenging the person.
  4. Effects can last over time, but generally grow smaller (use it or lose it).
  5. Those with more EF challenges benefit the most from intervention.
  6. More training is generally better, with the exception of aerobic exercises with no evidence of greater EF benefits from longer programs.
  7. For the most part, length of the sessions matters, with sessions > 30 minutes showing better results than those less than 30 minutes.
  8. Spaced (distributed) practice produces better long-term outcomes than massed practice.
  9. Benefits are mostly clearly seen on complex, multi-component EF tasks

In addition, there seems to be strong evidence for the role of physical activity in improving EF in individuals with ADHD. A PLoS One meta-analysis (Song et al., 2023)   found a significant impact of physical interventions on inhibitory control, working memory and cognitive flexibility in children and teens with ADHD. While there was a benefit to all kinds of physical activity, results were in line with the first paper above in that moderate exercise and those which require adaptive/flexible skills, and which endure over time show the most promise.

UPDATE: ‘debate and request for further dialogue’ for the research from a group of researchers https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6969305/

References

Diamond, A. & Ling, D. S. (2019). Review of the evidence on, and fundamental questions about, efforts to improve executive functions, including working memory. In J. Novick, M.F. Bunting, M.R. Dougherty & R. W. Engle (Eds.), Cognitive and working memory training: Perspectives from psychology, neuroscience, and human development, (pp.143-431). New York, NY: Oxford University Press. (pdf)

Song, Y., Fan, B., Wang, C., & Yu, H. (2023). Meta-analysis of the effects of physical activity on executive function in children and adolescents with attention deficit hyperactivity disorder. PloS one, 18(8), e0289732. https://doi.org/10.1371/journal.pone.0289732