P.E.I. principals advised to reduce PISA exemptions

Island principals were asked to review the lists of students they put forward this year to be exempted from having to participate in PISA testing, CBC has learned.

The test is being administered this month at schools across P.E.I. The last time it was written in 2015, Island students saw a significant increase in their achievement level relative to other provinces and other countries.

But that test also saw an unprecedented number of Island students exempted from having to write the test — 14 per cent of students were kept out of the draw to select students to write the test, or one out of every seven.

• P.E.I. shows significant improvement in international student testing
• 1 in 7 P.E.I. students excluded from writing PISA test

Groups of randomly-selected 15-year-old students write the test for the International Programme for Student Assessment, or PISA, considered the global benchmark when it comes to assessing the performance of school systems around the world.

If our measure of how well we educate our students isn't accurate, it's misleading the public.— Tess Miller

But in March the Department of Education asked P.E.I. school principals to review this year's lists. 

The department told CBC News that this year all provinces were asked by PISA to pay careful attention to exemption rates and keep them to a minimum. 

"Given Prince Edward Island's high exclusion rates in PISA 2015, every effort is being made to ensure we adhere to the intended criteria for exclusions established by the PISA consortium," wrote Elizabeth Costa in an email sent March 6. Costa is manager of achievement and accountability with P.E.I.'s Department of Education.

Highest exemption rate in Canada

P.E.I.'s exemption rate in 2015 was nearly three times the maximum rate of five per cent allowed under quality standards set out by the Organisation for Economic Co-operation and Development, or OECD, which administers PISA.

P.E.I.'s rate in 2015 was the highest in Canada and higher than the exemption rate in any other country.

UPEI Education professor Tess Miller has been trying to find out why so many students ended up exempted from PISA in 2015.

"The whole purpose behind PISA is to provide an indication of how well we educate our students," Miller said.

"If our measure of how well we educate our students isn't accurate, it's misleading the public."

Intention of PISA 'to be as inclusive as possible'

At the time, Costa told CBC News the high number of exemptions in the 2015 testing was the result of P.E.I.'s "highly inclusive school system where children with cognitive disabilities are included in our regular schools." The province also pointed to the fact its results were recognized as valid by the PISA consortium.

Most of P.E.I.'s exclusions in 2015 — 11.7 per cent of the total eligible student population — were designated under a category for students with intellectual disabilities.

In her email to principals in March, Costa clarified that "students with special education needs who have the ability to understand the instructions and to respond to the assessment should participate."

Noting PISA does not provide test results on either an individual or a school level, Costa told principals "the intention of PISA is to be as inclusive as possible."

• PCs, Green leader unimpressed with Liberal role in improved test scores
• 'More work to do,' says education minister on PISA test improvements

CBC News contacted the P.E.I. Teacher's Federation, which represents school principals as well as teachers.

A spokesperson for the federation wrote in an email: "It is our current understanding that principals are not those that determine the exclusions. Principals are given guidelines and they submit information based on those guidelines." 

Province advised in 2014 to reduce exemptions

Documents obtained by CBC News through Freedom of Information show that Island educators were advised in the lead-up to the last round of PISA testing to reduce the number of exemptions — but instead, that number went up.

In October 2014 Pierre Brochu, at the time the lead person responsible for administering PISA in Canada, made a presentation to Island principals.

The presentation he delivered noted the PISA school participation rate on P.E.I. of 99 per cent was "excellent." However it also said "the student exclusion rate is very high. PISA requires that no more than five per cent of students overall be excluded."

• National student testing shows significant improvement on P.E.I.
• P.E.I. to review standardized testing used in schools

At the time Brochu would have been referring to P.E.I.'s exemption rate from the test written in 2012, which was 8.3 per cent.

Other documents obtained by CBC News include questions from the province's deputy minister at the time as to how a change in the way PISA was administered in the province in 2015 might affect test scores.

That was the year the province switched from having all students write the test (minus those who were exempted), to having a random sample of students write the test, as is done in other provinces.

'Could not go much lower I hope'

"I would like to see how a sample versus a population approach would [affect] our score," wrote deputy minister Sandy MacDonald to Costa in August 2014.

"Could not go much lower I hope."

MacDonald also asked whether P.E.I. "will be able to exclude certain schools that may have a lower SES than others." 

SES, MacDonald has since clarified, refers to socio-economic status, which he explained has been shown to have a significant effect on student achievement.

MacDonald, now with Holland College, told CBC News he feels including all schools in the testing puts P.E.I. at a disadvantage against provinces like Quebec, where only 52 per cent of schools selected to write PISA in 2015 actually did so. Because of that, the province's results included a caution against a possible bias in the figures.

The answers to MacDonald's questions came from Brochu, who said the change to a sample shouldn't affect overall results, and that schools could not be exempted "based on their SES."

More P.E.I. news