DIRTy ToEs - Differentiation by Assessment

My main target as a teacher for this coming year builds upon the work I have done previously on using Taxonomies of Errors (ToEs) which you can read about in an earlier post on this blogsite. Although they are fantastic tools for sharing with students the common mistakes made by the whole class (to compliment the individual feedback given through my marking), these taxonomies do not of themselves cause students to make improvements to their practice. 

So, my target is to use DIRTy activities (Dedicated Improvement & Reflection Time for the uninitiated) in conjunction with my ToEs in order to ensure that the hours I spend on assessing an evaluating students' work are actually worth it. This post is my reflection on the first of my DIRTy ToEs lessons, with my Y12 Sociology class. 

The essay I had marked was one that I had asked them to do at home in timed conditions about the strengths and limitations of official statistics as a method for sociological research. 

In this follow-up lesson I wanted them to take the findings from an interview they had conducted with a family member on mortality rates and learn from their previous 'methods question' errors (individually and as a class) and act upon this learning. Having given them a whole period to feedback in groups and as a whole class about their positive and negative experiences of the interview process, comparing these experiences to the recognised strengths and limitations of the methodology, I then have them back their marked essays. 

When I mark essays at A-Level I scrawl my immediate thoughts on the essays as I read them, but then type up my overall impressions of strengths and areas for improvement on Fronter. I do this so that my comments are un-loseable and to enable the students to look at the comments I have made over time to see if any comments recur again and again. As this was their first formal feedback I printed these off and attached them to the essays. Here's what one of them looks like for Student A. 

After giving the students time to consider my comments and marking I then spoke with them about the Taxonomy of Errors for the whole class. As usual I categorised these into low level errors (ones that stop students achieving a pass), mid-level errors (ones that get in the way of a C) and high-level errors (ones that might prevent the awarding of the highest grades). This is what they looked like this time around.

At the lower level the issue of 'lifting' came about because I asked them to complete their essays at home and, being new AS students, they were more than reliant on the textbook. It's a problem that needs nipping in the bud, and the taxonomy approach helps greatly with this. 

At the mid-level, where most students found themselves, there is a greater array of errors. At this stage in the year, however, I am always keen to address the issue of wasted words (hence the point about repetitive conclusions) and the lack of examples to support points.

At the high level I was keen to stress the need to develop their understanding of 'validity' as a key theoretical construct for the subject, as it lags behind their understanding of 'reliability' and 'representativeness'. The other key point I wanted to focus on was students not accepting a supposed methodological strength or weakness without evaluating it or challenging it. 

The next step was to set them DIRTy activities that asked them to 'put right' an error from the class taxonomy that also featured on their individual feedback sheet. For those students who over-relied on the textbook the task was to rewrite a section in their own words. The students at mid or high levels of errors were given a little more choice to reflect the greater variety of errors that were made. 

The tasks they were given were:

Student A (whose feedback was shown earlier) chose to add a section on 'validity' from the higher DIRTy ToEs tasks. Her improvements looked like this:

All well and good. Not perfect but an improvement on what had gone before. 

Where the lesson really worked was in the main task that followed the DIRTy ToEs activity. 

In this activity, students were asked to go beyond the mistakes that they had already made and aim to skip the mistakes that those who had achieved more highly than them had made. If you want to strive for rapid progress, then this is the way to go!!!  Because Student A was already at the top of the taxonomy (do I need an even higher category for the highest level mistakes that haven't yet been made?) she had another stab at the validity issue and provided something much more coherent than even the DIRTy ToEs task. 

But Student A wasn't the only one to really improve the quality of her work in this lesson. Student B had made the mistake of copying chunks from the textbook, as this shows. 

 

She was required, by the DIRTy ToEs activity to take this section of her initial essay and rewrite it into her own words. This is what she produced. 

Much better!  And note the further addition of a sentence, showing evidence of evaluation and improvement. For her new essay Student B decided to focus on the following targets from the Mid-Level ToE. 

The section below shows how she managed to address the second of these targets. Notice the use of the qualifier 'can' used more than once in this paragraph. 

A third student, Student C, was halfway between A and B in her original essay, and her errors were at the mid-level of the ToE. 

As you can see from my marking in the picture above, one of the things that she needed to focus on in order to move closer to a C grade was to add specific examples of official statistic datasets to illustrate her general points about the method.  She responded wonderfully well to this in the DIRTy ToEs activity. 

As with Student B, C elected to retain the addition of examples from the mid-level ToE, as well as adding targets from the high-level ToE. 

She never managed to explore the term validity as she hoped, preferring instead to consolidate on her use of examples. 

This isn't a problem. When I feed back to her next time I will remind her of the unmet target and before then I shall be addressing the whole class's understanding of validity as a key issue that needs to be addressed. Remember that this post-DIRTy ToEs activity was about aiming to avoid mistakes that others above them had made: very much a stretch activity. The fact that she has addressed a significant issue for her and then consolidated it in a second piece of writing is good enough for me. 

Conclusion: Next steps on DIRTy ToEs

The addition of DIRTy activities to my already well-established Taxonomy of Errors technique has helped my teaching a great deal. 

Without the DIRT, my ToEs would simply be lists of what did and didn't work with a hopeful assumption that students follow up on these in their own time.  Without the ToEs, my DIRTy activities would be undifferentiated and decontextualised from the needs of the students. 

I shall be continuing to get my ToEs DIRTy in the coming months because the learning that was done this lesson, and the consequent improvement made by students, was exceptional. It took almost all of a triple period to get the students to that point, though, and I don't want to have to invest that amount of time in the future, so keeping it regular is going to be a key challenge for me. But I am also aware that repetition is a double-edged sword and, because of this, I want to look to vary the DIRTy activities as much as possible and even look to see how I can modify the ToEs too. 

I'll let you know how I get on. 

Using a 'Taxonomy of Errors' to Enhance Student Responses

I am writing this post to outline and to describe a technique I learnt some years ago from a long-forgotten colleague that has had a very positive impact on my teaching and the learning of my students:  the Taxonomy of Errors.  As well as describing the technique I want to show how I have developed it in recent weeks, how I have used it proactively with my students and what impact it has had on their written work.  Hopefully it may be useful to one or two of you.

In essence, the Taxonomy of Errors is a response to that perennial problem faced by teachers of dealing with a class that are all making the same mistakes.  It is a method for trying to ensure that students learn from each others' mistakes as well as from their own.  It is a method for trying to ensure that when they next attempt the same task, they will improve markedly rather than incrementally because they have addressed a range of foibles in their work.

In practice, the Taxonomy of Errors is little more than a summary of all the feedback that you have given to the students as individuals with a focus on the comments you have written time after time after time.  Sometimes I have even been known to rank them in order of frequency!!!

This is an example of a Taxonomy of Errors (ToE) for my top set English class.  I took them over two weeks ago and immediately got them to complete a full practice exam for me so I could see what they could do in the heat of battle.  This is the ToE for the important question on the Writing Section, which requires them to analyse language across two non-fiction texts and make comparisons.  As you can probably guess this response was not perhaps their finest hour, mainly because they had forgotten exam techniques in ensuring that they performed well in their controlled assessments.  This strategy is part of my portfolio of tools for getting them prepared for their exams later this month.  You will notice that the vocabulary is negative.  Although I sometimes write ToEs positively (there is an example later) the intention is to be bluntly honest about what went wrong, and the fact that the list is based on the whole class makes it easier to be so (although I am bluntly honest individually too!).

Here is another example of a Taxonomy of Errors, this time in response to a collection of timed essays from my AS Sociology students.  In this ToE I made an attempt to separate out the basic errors (at the top) and the more complex error (at the bottom) to ensure that they understood the things that they really ought not have done, regardless of their ability, and the genuine areas for further learning.  I always find that at this time of the academic year students facing imminent external examination make all kinds of foolish errors and they need to be scuttled in order to allow them to get at the more important stuff that will genuinely allow them to achieve higher grades (we always call them the Sheep Grades because they are B, A, A* - Geddit?). I always present this feedback at the very start of the lesson after I have marked the work and am increasingly linking the feedback directly to the activities of the next lesson so that they improve upon their work immediately.

Here's my latest feedback to my Y11 English students, and in it I have focused even more on the creation of a genuine Taxonomy of Errors (that may have been how it was intended and I have just found my way to it the long way around!!): from the basics that are genuinely beneath this groups of students, to the intermediate and advanced.  This feedback was from an essay on An Inspector Calls where the students were (to a greater or lesser extent) all guilty of simply trying to rewrite their controlled assessment work on Arthur and Sheila rather than respond about the Inspector as they had been asked.  I was therefore able to show them how the error at the intermediate level was preventing them from accessing the higher grades, and thereby making the feedback on higher level errors virtually redundant (most students achieved only between 17 and 21 marks out of 30 because of their intermediate errors).

Here is a very typical response from one of the students that led to this Taxonomy of Errors, and was even one of the better ones because he made sure to refer to the Inspector on four occasions.  In the end, though, his best observations were reserved for the character of Sheila and he would be reliant upon the leniency of the examiner at best and, at worst, reliant upon another question coming up that allowed him to crowbar in his understanding of the character of Sheila.  In response to the ToE then, I devised a sequence of activities that was focused on using 15 minutes of their 45 to plan effectively to answer the actual question, not the one that they wish had been set.  This involved brainstorming what they knew already about the Inspector, selecting apposite quotes (ensuring that at least one of these was from the stage directions to allow the response to include reference to stagecraft - from the Advanced section of the ToE), exploring the language of the quotes for dominant and subversive interpretations and then evaluating the quotes in light of the essay question.

And here is the response that the same student generated during this double lesson in response to the activities I had set; activities which had been informed by the Taxonomy of Errors (at this stage of the year my lessons are almost entirely planned in response to their emergent needs). I'm not claiming that it is a startlingly better response, but it does tackle the intermediate error of not focusing on the question to ensure he has a chance at the top marks.  Further to this it addresses the advanced error of a lack of reference to stagecraft and deepens his use of language analysis from a straightforward discussion of the word 'horrible' to a more convincing analysis of the words 'taking charge masterfully' in a better quotation that, again, had more resonance with the actual question he had been asked to answer.

So there you have it.  In my classroom the Taxonomy of Errors is used for three purposes.  In its most simple guise it tells the students what mistakes have been made, by others as well as themselves, so that they can get a sense of their achievements alongside those of their peers.  At a more sophisticated level the Taxonomy of Errors allows me to rank the impact of different errors on their marks and/or grades by showing them how basic or intermediate errors can undermine work that in other ways might have the potential to achieve highly.  But the Taxonomy of Errors is at its most effective when it informs my planning so that students are taught (or re-taught) the knowledge or skills that had been demonstrated so poorly in their submitted work.  The Taxonomy of Errors is at its most potent when it is used in this way and results in the students being asked to edit or rewrite the error-strewn original in a conscious attempt to improve it.  The Taxonomy of Errors is at its most rewarding when it helps make marking have a genuine impact on learning.



Improving my written feedback

My department and I are working in together to try out different strategies to improve the feedback that we give to pupils and the impact that it has on their learning. We want to move away from the traditional 'Tick or cross 20 sums' approach beloved of maths teachers since time began.

I chose the triads based on my knowledge of complimentary strengths of each of my team but each group has come up with their own action plan to research, share and develop individually and as a group and we'll be reporting our findings back to each other before taking best practice forward to create what I hope will be an effective, consistent and sustainable departmental policy.

This blog post will outline one of the things I've been playing with to improve the frequency and quality of my written feedback, which to be honest has always been a bit ropey - I know the right thing to do but don't always make the time to do it, for anybody other than my year 11s. Increasingly so as I've gained more responsibilities, inside and outside of school. I've compensated for it by becoming very good with verbal feedback but the thing that really made me think was when the head observed me for our whole school review and said the quality of written feedback is what stopped her giving my lesson a 1, it was the first time I'd dipped below a grade 1 for an observed lesson for several years. I argued a bit but deep down knew it could be better. Only question? How to improve without throwing my work -life balance entirely out of the window?

Brief notes below... ( it was orig going to be an email to my fellow triad members as we agreed that this is how we'd keep each other posted regarding what we had been trying out).


Once a week. Year 9 class in the double lesson ( 2hrs)

Make sure I speak to every student in the class individually and mark their book/work. Tell them I will be doing this at start of lesson. Not every single question but specific questions that I have targeted as they demonstrate the skills for the lesson or may require deeper understanding.

Talk to the students about the work as I normally would and ask them a supplementary question related to what they have done.

They answer verbally but I also write this question down in their books as we are talking and get them to write their response. Evidence of on going dialogue between me and students.

If the students are clearly stuck or making the same repeated mistake. I write a very clear comment that will help them to improve immediately. I talk to the student re the comment and they have to respond to it verbally and in writing either whole I am there or within 5 min, then call me back over. I've noticed that most students naturally go back and correct the questions that they have got wrong after this.


Sometimes I look back at work earlier in week and check understanding re that. If they aren't sure re their response, I leave them and give them time to do it before I return.

The beauty of this is that it isn't really any extra work- I'm just having a written record of what is happening verbally within my classroom. It requires me to have a quality individual conversation with every single student during the (2hr) lesson. So forces me to reduce teacher talk to the whole class and plan an activity where they learn but can work autonomously. I used to worry I would not get around every body but at various points I ask them to remind me if I have missed anyone and it works out.

As we have a normal conversation anyway, students don't ignore my comments and are fine responding to them in writing as we already have verbally.

I'm finding out some interesting things too. I saw a graph had been crossed out and asked the student why. They then gave a beautiful and detailed explanation of how after seeing somebody else's work they realised that the "jumps were wrong" so decided to start again. I write my question down and asked then to write exactly what they had told me as it clearly demonstrated that they had taken on board the need for correct scale. Then I responded to their comment too.

True dialogue about maths and all done as part of regular class routine. Result.

Lets hope that my colleagues A and M feel that my method has an impact on students learning when we complete our peer work scrutiny in the next within the next week.