The 'accountability reform satisfaction paradox' is not just the name of the unsuccessful prog rock band started by researchers here at The Key. It refers to findings from The Key's recent State of Education report which show that 80% of primary school leaders do not think that the primary accountability reforms will more accurately reflect their school’s overall performance. By contrast, 69% of secondary school leaders told us that the new progress 8 measure will more accurately capture the overall performance of their school.
An early press shot of The Accountability Reform Satisfaction Paradox. Backstage, Munich: 1976.
This got me questioning why there was such a stark divide. Both are major reforms to accountability which have been a while in the making and will use a new scoring system to measure performance against the floor standards. It’s overly simplistic to compare these reforms as entirely like-for-like but the scale of the changes is nonetheless similar for both phases.
Below, I explore some of the key differences in the design and implementation of the reforms, in an attempt to sound out potential reasons for this variance in satisfaction between the phases. This is ultimately speculative; I would love to hear from leaders from either phase about their thoughts, experiences and/or recurring dreams/nightmares in the comments.
An opt-in year for progress 8
An obvious difference between the implementation of the two accountability reforms was the opt-in year for progress 8. Not every school chose to do so, but schools could opt-in a year early and see what their progress 8 scores would have looked like for the 2015 year 11 cohort. The data from opt-in schools is publicly available, meaning even schools that didn't opt-in have been able to familiarise themselves with the reform. The point score scales for Key Stage 4 qualifications and 2015 attainment 8 estimates have also been published. These cannot be used for exact modelling, but they do give a clearer picture of the component parts.
Primary accountability reforms, on the other hand, do feel like a murkier picture. We know 100 will represent the expected standard on the scaled score, but that is all we really know. We don’t know the conversion rates from the raw scores (yet; these are due on July 5), or how much higher or lower the scores will spread from 100. It is possible that more secondary leaders feel confident in progress 8 as a reflection of school performance because they have a clearer picture of how it will look in practice.
The emphasis on progress
I would argue that, from a floor standards perspective, secondary schools are seeing a larger shift away from the previous system. Progress 8 will be the only measure used in the floor standard. At primary level, there is still a combination of attainment and progress measures. As there is still an importance placed on reaching an expected standard of literacy and numeracy to start secondary school, it’d be difficult to remove attainment entirely.
But students entering Key Stage 5 without having achieved the expected standard in English and maths at Key Stage 4 have to resit these qualifications, so it isn’t as though there is an abandonment of attainment in these subjects at secondary level either. Is it possible that schools see the progress their students make, not the attainment threshold they do (or don’t) reach, as a more accurate reflection of the school’s influence on achievement?
Communication about the reforms
This academic year, there have been more high-profile causes of dissatisfaction around primary school tests and subsequent performance measures. The teacher assessment deadline was changed, the commission on the assessment without levels report was later than first promised, the teacher assessment levels are based on an interim framework set to change again next year, reception baseline assessments were abandoned following a review of their comparability and two cases of leaked tests have been reported (albeit in different circumstances).
As aforementioned, the implementation of progress 8 was relatively gradual, what with an opt-in year. It’s not to say people didn’t voice their concerns about its implementation, but there has been a lot of time to voice them. 2015/16 has felt a more turbulent year for primary assessment than secondary. It is not a stretch to imagine that this catalogue of changes will have an effect on primary school leaders' confidence in the accountability measures.
Have I missed anything?
As I said, these are possible explanations but by no means a definitive list. But it's an important conversation to be had, and one that can hopefully lead to constructive suggestions. Who knows, they may even find their way to the DfE to inform future accountability decisions and implementations.