Using a 'Taxonomy of Errors' to Enhance Student Responses

I am writing this post to outline and to describe a technique I learnt some years ago from a long-forgotten colleague that has had a very positive impact on my teaching and the learning of my students:  the Taxonomy of Errors.  As well as describing the technique I want to show how I have developed it in recent weeks, how I have used it proactively with my students and what impact it has had on their written work.  Hopefully it may be useful to one or two of you.

In essence, the Taxonomy of Errors is a response to that perennial problem faced by teachers of dealing with a class that are all making the same mistakes.  It is a method for trying to ensure that students learn from each others' mistakes as well as from their own.  It is a method for trying to ensure that when they next attempt the same task, they will improve markedly rather than incrementally because they have addressed a range of foibles in their work.

In practice, the Taxonomy of Errors is little more than a summary of all the feedback that you have given to the students as individuals with a focus on the comments you have written time after time after time.  Sometimes I have even been known to rank them in order of frequency!!!

This is an example of a Taxonomy of Errors (ToE) for my top set English class.  I took them over two weeks ago and immediately got them to complete a full practice exam for me so I could see what they could do in the heat of battle.  This is the ToE for the important question on the Writing Section, which requires them to analyse language across two non-fiction texts and make comparisons.  As you can probably guess this response was not perhaps their finest hour, mainly because they had forgotten exam techniques in ensuring that they performed well in their controlled assessments.  This strategy is part of my portfolio of tools for getting them prepared for their exams later this month.  You will notice that the vocabulary is negative.  Although I sometimes write ToEs positively (there is an example later) the intention is to be bluntly honest about what went wrong, and the fact that the list is based on the whole class makes it easier to be so (although I am bluntly honest individually too!).

Here is another example of a Taxonomy of Errors, this time in response to a collection of timed essays from my AS Sociology students.  In this ToE I made an attempt to separate out the basic errors (at the top) and the more complex error (at the bottom) to ensure that they understood the things that they really ought not have done, regardless of their ability, and the genuine areas for further learning.  I always find that at this time of the academic year students facing imminent external examination make all kinds of foolish errors and they need to be scuttled in order to allow them to get at the more important stuff that will genuinely allow them to achieve higher grades (we always call them the Sheep Grades because they are B, A, A* - Geddit?). I always present this feedback at the very start of the lesson after I have marked the work and am increasingly linking the feedback directly to the activities of the next lesson so that they improve upon their work immediately.

Here's my latest feedback to my Y11 English students, and in it I have focused even more on the creation of a genuine Taxonomy of Errors (that may have been how it was intended and I have just found my way to it the long way around!!): from the basics that are genuinely beneath this groups of students, to the intermediate and advanced.  This feedback was from an essay on An Inspector Calls where the students were (to a greater or lesser extent) all guilty of simply trying to rewrite their controlled assessment work on Arthur and Sheila rather than respond about the Inspector as they had been asked.  I was therefore able to show them how the error at the intermediate level was preventing them from accessing the higher grades, and thereby making the feedback on higher level errors virtually redundant (most students achieved only between 17 and 21 marks out of 30 because of their intermediate errors).

Here is a very typical response from one of the students that led to this Taxonomy of Errors, and was even one of the better ones because he made sure to refer to the Inspector on four occasions.  In the end, though, his best observations were reserved for the character of Sheila and he would be reliant upon the leniency of the examiner at best and, at worst, reliant upon another question coming up that allowed him to crowbar in his understanding of the character of Sheila.  In response to the ToE then, I devised a sequence of activities that was focused on using 15 minutes of their 45 to plan effectively to answer the actual question, not the one that they wish had been set.  This involved brainstorming what they knew already about the Inspector, selecting apposite quotes (ensuring that at least one of these was from the stage directions to allow the response to include reference to stagecraft - from the Advanced section of the ToE), exploring the language of the quotes for dominant and subversive interpretations and then evaluating the quotes in light of the essay question.

And here is the response that the same student generated during this double lesson in response to the activities I had set; activities which had been informed by the Taxonomy of Errors (at this stage of the year my lessons are almost entirely planned in response to their emergent needs). I'm not claiming that it is a startlingly better response, but it does tackle the intermediate error of not focusing on the question to ensure he has a chance at the top marks.  Further to this it addresses the advanced error of a lack of reference to stagecraft and deepens his use of language analysis from a straightforward discussion of the word 'horrible' to a more convincing analysis of the words 'taking charge masterfully' in a better quotation that, again, had more resonance with the actual question he had been asked to answer.

So there you have it.  In my classroom the Taxonomy of Errors is used for three purposes.  In its most simple guise it tells the students what mistakes have been made, by others as well as themselves, so that they can get a sense of their achievements alongside those of their peers.  At a more sophisticated level the Taxonomy of Errors allows me to rank the impact of different errors on their marks and/or grades by showing them how basic or intermediate errors can undermine work that in other ways might have the potential to achieve highly.  But the Taxonomy of Errors is at its most effective when it informs my planning so that students are taught (or re-taught) the knowledge or skills that had been demonstrated so poorly in their submitted work.  The Taxonomy of Errors is at its most potent when it is used in this way and results in the students being asked to edit or rewrite the error-strewn original in a conscious attempt to improve it.  The Taxonomy of Errors is at its most rewarding when it helps make marking have a genuine impact on learning.



Practitioner-Led INSET: Marginal Learning Gains at Canons

On Monday we welcomed our staff back for the start of a

 new term and new year and for the fourth of our crucial Canons OPP INSET sessions of the year.  For this INSET day our main focus was  on in-class evaluation processes, but we wanted to start the day with an activity that involved all teachers in understanding the concept of Dave Brailsford's Marginal Gains ethos that underpinned the successes of cycling's Team Sky and Team GB in 2012.

 

The start of the session involved asking staff to identify Dave Brailsford (I've never seen PE staff so keen to contribute to an INSET Q&A - Thanks Alex!).  We then watched the video of Brailsford explaining to BBC Breakfast the theory behind Marginal Gains and how it helped his teams achieve success this year.  A summary of the key points outlined in his interview can be seen to the right, which was shared with staff in the context of Canons.

Work has already been done at Canons to disaggregate the Ofsted outstanding criteria.  Following a suggestion by Alex Quigley at Huntington School (@huntingenglish to the twitterati), the aim of this session was to 'crowdsource' these ten disaggregated criteria, disaggregating Ofsted outstanding even further to allow it to be conceptualised in terms of concrete classroom practice.

 

The key point made throughout this session was that as an already outstanding school aiming to become great, we need to be looking to Marginal Gains to help us on that journey.  We also insisted that the vast majority - if not all - of these Marginal Gains were already present somewhere within the school in the classrooms of our colleagues.  This informed the first part of the 'crowdsourcing activity'.

 



Identifying Marginal Learning Gains

Teachers were asked to take ten minutes to consider the ten disaggregated areas of the Ofsted outstanding criteria and consider what aspects of their own classroom practice they were proud of and that would constitute a potential Marginal Learning Gain to other teachers.  They were asked to "sweat the small stuff" and "look after the pennies to save the pounds", concentrating on the 1% gains that would be relatively easy for colleagues to take on. 




Good ideas from our own classrooms

Over the course of the ten minutes some staff got straight to it, whilst others spent time considering their contributions.  Some moved around a large number of tables whilst some decided to stick with one or two.  The aim was to have at least twenty suggestions for each area, requiring a contribution of at least four ideas from each teacher in the room.  The best oral feedback I have had so far is that the focus on our own strengths was really heartening for staff.

 

Literacy, oracy and numeracy highlighted

Of course, during this process there were a few areas of the Ofsted outstanding criteria that received more comments than others.  To help ensure all areas received equal attention I buzzed around shouting out where we could do with more comments and reminding colleagues of the wonderful work that they had done or were doing that I had seen or heard about: Not a bad way for any member of SLT to begin the year, shouting out positives.

 

Once the ten minutes were up I called for staff to stop and, as they expectantly and longingly looked at their chairs, asked them to find an area of the disaggregated Ofsted outstanding criteria where they felt less confident and, in discussion as groups to go through the suggestions made in the first activity to identify the Top Ten Tips on the sheet that could be passed on to all teachers, including newly appointed NQTs and our next wave of PGCE students.



Interdependent learning

Over the next ten minutes each table identified their own ways to reduce down the suggestions that they had inherited.  Some got straight to it and circled the ones that stood out most to them.  Others took time to consider how different suggestions could be amalgamated to ensure that no good idea was lost in the process.  Some groups appointed someone to make decisions for them and some gravitated towards a powerful personality.  Some bickered and others agreed wholeheartedly.  Some struggled to complete the task so rounded were their discussions.  It was not unlike watching learners of any age complete a task rooted in interdependence and collaboration: a pleasure and a privelege.


Reducing lists to Top Ten MLG tips

The point of asking the teachers to create a Top Ten Tips was to ensure that the final ideas were refined enough, and by asking them to lead on this distillation process in an area of relative weakness or uncertainty for them was to ensure that the tips created would be relevant to those most in need (or in want) of their development.  Having involved them as experts in specific areas we wanted to engage them as the critically reflective practitioners we know them to be.

 

 





The purpose behind the whole activity was twofold.  Firstly, to take the time at the start of the new year to remind our staff that they are outstanding at what they do.  Secondly, to support them in their individual and collective quests to get ever better.  We asked them, in their own time after the day to look over the Top Ten Tips in the ten disaggregated areas of Ofsted outstanding criteria and identify eight areas of practice the would like to tweak.


Very much a labour of love

In order to show teachers the value of the activity that they had participated in and the importance of their contribution to that process we made a decision very early on to have the 100 Marginal Learning Gains identified by our staff during the day printed professionally on the day itself.  Using the artwork and the same quality paper as we use for our school brochures we commissioned our usual printers to undertake the task for us.  The aim was to use the plenary session for the day to give this quality product back to its authors.  The big shame of the day was that technical hitches at the publishers meant that they arrived 30 minutes late!!

Intro & contents of our MLG booklet

As well as using the artwork of our school brochure we decided to lace the booklet with images related to cycling, Team Sky and Team GB to retain the theme that had dominated during the presentation and to reinforce the notion of excellence (or greatness) that is at the heart of what we are doing.  Using shutterstock for selecting these images ensured that we were copyright-safe.

MLG booklet printed the same day!!

The work that was put into the booklet by myself and our school Office Manager, Jackie, was well worth the time and the very reasonable price.  A photocopied booklet printed in school could not possibly, however lovingly crafted, have conveyed the same sense of quality and professionalism.  Having collated the ten groups of Top Ten Tips for each area of the disaggregated Ofsted outstanding criteria, we typed them and sent them off to the publisher where they were aligned with fabulous pictures of our staff and students.

MLG Wheel for Self-Evaluation

The main focus of the booklet, though, is the Marginal Learning Gains wheel that sits at its centre in between photos of Bradley Wiggins (representing individual success) and Team Sky (representing school success).  With thanks to Zoe Elder (@fullonlearning) amongst others, we have borrowed the ideas of others to produce a beautiful 'crowdsourced', expertise-laden, development-focused booklet that is all about the Marginal Learning Gains and I couldn't be any prouder of the process that our staff went through in this INSET day or the product that they created.

 

SOLO from the assessment cradle to the assessment grave

On Monday this week the results of my Dissertation (and therefore final Masters level) were released by Middlesex University. When I finally was able to logon to the unihub portal - the first time I had tried in my three years on the school-based programme - they only gave me my mark and didn't say what level of award it was equivalent to. I had a hazy recollection that it was in a handbook I had been given and so I looked it up and there, to my utter surprise, was SOLO taxonomy staring back at me!!!

To say I was stunned is an understatement. Although SOLO wasn't a part of my dissertation's conclusions about what good assessment should look like, I found my way to it immediately after thanks to @biomadhatter, and the way SOLO dovetailed with the conclusions of my research was serendipitous in the extreme: it was as if my findings were preparing me for the revelatory nature of SOLO in action. But what shocked me was the fact that the same levels of understanding I had been teaching Y7 English and Y12 Sociology students to apply to their work were being used at Masters level to distinguish between a distinction (extended abstract), a merit (relational) and a pass (multistructural).

Just the previous week I had launched SOLO with my Y12s in a lesson where I got them to highlight their A2 mark scheme for AQA Sociology according to SOLO taxonomy (see my post 'Keeping up with the Soloists'). The results were crystal clear; that extended abstract was the top band, relational the top half of the middle band, multistructural the bottom half of the middle band and so on. This was a powerful lever for me in getting students to appreciate the value of SOLO.

And so, being a fan of serendipity, I decided to use the Middlesex university mark scheme for Masters level dissertations to mark my AS Sociologists' essays, rather than the A-Level mark scheme to see if it would work as well. The results were amazing.

The first of the essays I marked was a real mixed bag, which using a traditional mark scheme would have been really difficult to evaluate. Because I would have been looking at a number of the different mark bands I would have had to cobble together a best fit statement that would have offered little to the student. Instead I am able to point out where she is prestructural, but also where she is multistructural and, at her best, relational. My concluding comments are therefore far more effective at giving her feedback, feed up and feed forward: a proper evaluative response for her to draw strength from and use to identify improvements.

In this second student's response there was far less variability of understanding, and so I was able to focus my feed forward on moving up from multistructural in certain places, to secure a fully relational response as a springboard to aiming at completely extended abstract thinking (I'm hopeful of her achieving an A grade next year, and so this mark scheme fits very well with her targets for the coming year).

This final example is a close-up on the feedback, feed up and feed forward I provided for a third student who has the potential to achieve a grade A or even A* next year. Although I can see things that I might do differently (it was my last lesson with them and I had to mark these essays as they worked interdependently) the thing I am most pleased about with these evaluations is that I found writing the positives and negatives equally easy. When using the A-Level mark scheme I always find that I operate on a deficit model with students, pointing out all the faults whilst finding it hard to look back and see the achievements.

The difference that I have noted from this first foray into using SOLO taxonomy for written evaluation (notice I am not using the word assessment as it seems too shallow for the feedback, feed up and feed forward I have given) is that I am able to evaluate each paragraph and even sentence independently of the whole. This precision of evaluation has allowed me to really accurately pinpoint which concepts they have not grasped, or where they have not applied theory to sociological contexts, or where they have failed to evaluate sociological theories effectively. In doing so, it has helped me to promote the importance of consistency across the essay to them and they were quick to identify sections where they needed to go away and revise concepts or rewrite essays. If I had had more time with them I would have asked them to rewrite the section they felt most able to improve, perhaps after having spent time getting ideas from their peers based upon my evaluation.

I told the students afterwards that I had marked their essays using a Masters level mark scheme. What amazed them most was that SOLO taxonomy could be used to evaluate their work as well as mine, as well as GCSE coursework and as well as KS3 portfolios. It was clear that they realised that thinking about levels of understanding (as exemplified by the SOLO concepts of Extended Abstract, Relational, etc) was something that could connect them to their studies beyond A-Levels, and the fact that they were realising this in our last lesson before their UCAS preparation week was not lost on them, or me.

And what surprised me most? The fact that not one of them asked what grade they had received. And the fact that none of them asked what grade I had received. SOLO was enough for them. And that's enough for me.