Posted on

NAPLAN: How to Cook the Books

The Myschool website is a huge database. It was originally set up by the federal government as a snapshot of how schools were performing in Australia to help parents choose a school for their child. However, like any database it is subject to “garbage in, garbage out”.

When you type in the name of the school where the website says “Find a school”, there is lots of information about the school. When you click on ‘NAPLAN’ there is also lots of information about the NAPLAN results. At a recent ResearchEd conference, Prof. John Hattie stated that the important information was ‘student gain’. This represents how the school has contributed to children’s learning.

Looking at these graphs, which have a nice steep straight line, might initially give an impression that the school is doing a great job.

(N.b. – click on images to view in full size)

But ticking the box ‘Schools with similar students’ might give a more nuanced picture, as this compares the school with schools which have a similar Socio Economic Status (SES). (There are many studies, which show that there is a close relationship between high SES and academic achievement.)

Ticking the other boxes, ‘Students with the same starting score’ and ‘All schools’ gives the full picture. This particular school is doing well compared to all schools because it has a high Socio Economic Status (SES) and is in an affluent area. However, it is not doing quite so well compared with other schools with a similar SES.

There is another statistic which is important and that is the ‘Percentage of students in this year level for whom previous NAPLAN results (2 years prior) are available’.

On this page, NAPLAN results for the selected school relate only to matched students. Matched students are those who sat two consecutive NAPLAN tests at the same school and have results at two year levels. Results are shown only for schools with five or more matched students.

This school appears to be doing incredibly well compared to other schools of a similar SES, until you look at the ‘Percentage of students in this year level for whom previous NAPLAN results (2 years prior) are available: 60%’.

This means that only 60% of students sat NAPLAN in both years. However, clicking on attendance gives a rate of for the school of 98%. So where were the other 40%?

There is anecdotal evidence that some schools pressure the parents of children who have dyslexia, dyscalculia or other issues which may lead to under achievement, to absent their kids on NAPLAN testing days. This boosts the schools averages, as above and skews the data. Garbage in, garbage out!

Some anecdotal reports:

https://theconversation.com/declining-NAPLAN-participation-rates-are-likely-skewing-the-data-42821

http://www.couriermail.com.au/news/queensland/NAPLAN-bias-schools-exclude-children-with-communication-impairments/news-story/b13c34fcba09bfe6a05d1590878de609

https://www.amazeeducation.com.au/media/special-needs-kids-excluded-from-NAPLAN/

According to The 2015 NAPLAN report:

‘Since 2010, when withdrawals were first reported, there has been a general increase in the withdrawn rate. Despite this trend, the large percentage of students participating each year ensures that results are reliable and valid at the state, territory and national level.’ (P.325).

Are they equally valid at the school level?