Working through the pain towards success
Getting assessment right throughout KS3 and 4 obviously makes a huge difference to pupil outcomes. Many things I have read on the subject have inspired me to make changes in my practice. However, many have me shouting at the screen (you will see this is a common theme). Whilst I agree completely in principal with comments like:
“Whether assessments are large scale or teacher conducted, the principle of fairness requires that data-collection methods allow students with physical disabilities, learning disabilities, or limited English proficiency to demonstrate the full extent of their science knowledge and skills.”
“Statistical techniques require that both sexes and different racial and ethnic backgrounds be included in the development of large-scale assessments. Bias can be determined with some certainty through the combination of statistical evidence and expert judgment. For instance, if an exercise to assess understanding of inertia using a flywheel results in differential performance between females and males, a judgment that the exercise is biased might be plausible based on the assumption that males and females have different experiences with flywheels.”
“The requirement that assessment exercises be authentic and thus in context increases the likelihood that all tasks have some degree of bias for some population of students. Some contexts will have more appeal to males and others to females. If, however, assessments employ a variety of tasks, the collection will be "equally unfair" to all. This is one way in which the deleterious effects of bias can be avoided.”
(All quotations from https://www.nap.edu/read/4962/chapter/7#78)
These statements do not take into account what the pupils will have to sit at the end of the course. Giving support to a pupil during lessons is one thing, however supporting them in assessments when they will not receive any in external exams is setting them up to fail. Also the experience of pupils and context is irrelevant as there are no boy/girl exams; ultimately all the pupils will be sitting the same exam, as this often used cartoon shows.
In an ideal world I would love to be able to use alternative assessments to showcase the pupils strengths and what they can do, many pupils perform far better just from having the exam questions read to them. But the issue will always be showing comparability, it is bad enough trying to work out what a 9 is!
At the heart of this project was a simple idea. Modifying our scheme quick quizzes so that they can be used at the start and end of a topic to show pupil progress, which questions were answered poorly to inform intervention and to give pupils relevant feedback that hopefully will support their learning journey. All these tests will be completed online so that marking and data analysis can be automatically completed. Hopefully saving time for the teacher and giving the pupils fast feedback. Easy, how hard can it be to use Word and Excel…
Fast forward a couple of weeks and I am screaming at the screen (again). “NO, I didn’t want to copy the numbers, I didn’t SELECT the numbers… WHY HAVE YOU COPIED THE BLOODY NUMBERS?”
WORD is an evil genius. Having spent ages modifying and reformatting tests to fit the pages and checked the fonts all matched and were the same size. I copied them into the document with the cover sheet and then Word struck. It didn’t change everything, just the odd font size here and there, as well as shifting a few lines a couple of Tabs back. Took just as long again to go back through them all and re-correct them. I was pretty certain that EXCEL hated me too; I thought it was getting back at me for upsetting WORD. (I have always been particularly good at swearing, but I believe this project has made me world class.) EXCEL just seemed to randomly change formula.
I spent an evening setting up the data analysis spreadsheet to compare test scores for the before and after results. Everything worked like a charm. Did a couple practices runs, great, sorted.
Come the following day my trainee teacher had done the pre and post topic tests online and sent the pupils their results. I was keen to see how the spreadsheet performed. I copied and pasted the two sets of data into the spreadsheet, clicked on page three to see the magic happen and… sea of red and blank cells. I remained remarkably calm, staring blankly at the screen, with just an eye twitching. Bloody EXCEL. I did find out, whilst trouble shooting the spreadsheet, that it was actually office 365 that screwed me over. It turns out that when you send the pupils their results it adds another column to the data sheet, and it was this that messed up all the formula, lesson learned.
I am hopeful that now the worst of the set up is done, it should provide the department with a quick and simple way to show progress, whilst giving pupils faster and personalised individual feedback. I am also hopeful that my liver will recover in time for Christmas, after all, there is a lot of money to spend in the swear box.
Shane Brooking
Deputy Science Leader (Key Stage 3) The Ramsey Academy
Addendum
Since writing this (not realising that the date for submission was so far off) we have used the online multiple choice tests before and after topics as we planned. This has helped highlight pupils who have made great progress and those that did just as well at the start compared to the end. However, the most powerful use has been to be able to look at the individual questions and see how many were correct for a particular question at the start compared to the end. A great example of this is my Year 9 group where only two pupils answered a question right at the start and three answered correctly at the end. The spreadsheet highlighted this issue clearly and following a bit of intervention it transpired that my Year 9 group were unaware that heroin is a recreational drug, they assumed recreational meant legal. We have also been able to set the quizzes as revision for our KS3 groups in preparation for their end of year exams. Once set the data comes back to the teacher and can be sent back to the pupils in minutes.