Today the CBC had a story on the most recent results of the PCAP tests, which are administered to grade 8 students across the country.Unlike many of my colleagues in education I think have some sort of national assessment of student performance is a good thing for tracking accountability and educational efficacy. Which is not to say they should be the only measure of these.
These assessments were given at a sample of schools across Canada–a rnadomized stratified sample. 32,000 students wrote the exams across the country, 8,000 wrote it in French. Featured in the story are comparisons between provinces, including:
- In Math only students in Ontario and Québec score above the national average
- In Science only students in Alberta and Ontario scored above the national average
- In Reading only students in Alberta and Ontario scored above the national average
But before you pack your bags for Calgary or Etobicoke…think about these findings. First, the national averages are comparing each provincial sub-sample to the overall sample. And here’s the distribution of the samples on a per province basis:
- BC 3559
- AB 3515
- SK 2918
- MB 3110
- ON 5883
- QB 5237
- NB 2664
- NS 2843
- PEI 484
- NL 1861
- YT 305
- NWT or NT zero
Thus, a plurality of the sample were from Ontario, roughly 17 per cent of the overall sample. This is, interestingly enough, probably too small a sub-sample: Ontario holds approximately 35% of the total population of Canada. So from the outset we know the sampling strategy wasn’t based on province/territory:national population. In fact, the full report explains the sampling was done at the school level. The samples were built based on having entire schools’ grade 8 students write the exams.
Second, given that these are standardized scores–not the actual student scores, but a standardization across all of the sample, which assume the student performance is a Gaussian distribution–the mean of 500 applies to all of the assessments. As well, the standard deviation used for these adjusted scores is 100.
Third, drilling down you’ll find the Differences between provinces aren’t actually that large. For math PEI had a mean score of 460–the lowest mean of all provinces. Ontario and Quebec scored 507 and 515 respectively. So these are not normally distributed even after standarization. No one province does exceptionally well in math–PEI’s not doing well, but isn’t doing poorly either. For science Alberta and Ontario were “high” (515 and 510) and the Yukon low (478). For reading Ontario was above the mean (515) and the Yukon the lowest (465).
If the standard deviation was 100, all these results are well within the range expected. None of the provinces or territories are “outliers” and the differences between them are not nearly as pronounced.
Nothing earth-shattering here; share the raw testing data with us and we might have something. Full report available here.
