Reflecting on … critical use of student data.

Our inspired guest blog by former student @Lissyh1997 reminded me of the importance of handling student data with caution.

“Data is like garbage.  You’d better know what you’re going to do with it before you collect it.”  Mark Twain

As I prepared for the new term, I was struck by how much information about the pupils I had available.  This is before I’ve even met many of them in person.  Increasingly, the expectation is that students come to us much as food from the supermarket; pre-packaged with catch-all information progressively simplified into little coloured boxes.

As with many things, one broadly assumes the benefits outweigh the costs, although I’m unsure whether any rigorous research has been conducted to test this.  However, I do wonder whether the data we are given come with enough health warnings to help teachers avoid some of the dangers they present.  I have been reflecting on a few of these as I prepare to meet my new classes:

  • The Pygmalion and Golem Effects: In 1965 Rosenthal and Jacobson experimented with the impacts of labelling by convincing teachers that a (non-existent) test they had run on their students had identified certain students who were on the verge of going through the intellectual equivalent of a ‘growth spurt’ and whose progress would accelerate dramatically over the course of the next year.  They found that children profited from their teacher’s high expectations and made greater progress than those not so labelled.  There are various provisos to this “Pygmalion Effect”; including that the impact was limited to younger children and their paper explores a range of possible explanations.  However one implication is the possibility that low expectations can lead to poor outcomes for the student – the Golem Effect.  As a teacher, seeing my students for the first time through the filter of data, it is sobering to remind myself of the potential for my expectations to shape their outcomes.
  • Confirmation Bias: Along with the risks of self-fulfilling prophecy acting on the students, confirmation bias is a well-documented psychological tendency that it is worth teachers familiarising themselves with.  Essentially this is where we interpret new evidence in light of our existing theories.  This can be done in a variety of ways; we can look for evidence that supports our beliefs, disregard contradictory evidence as anomalous and give greater weight to information that fits comfortably with our current world-view.  It is not always a conscious process and can be hard to avoid, even when aware of the phenomenon.   The risk in education is that it can be easy to find confirmation of low expectations, even without realising we are looking for it.  All teenagers tend to miss the point sometimes, rush a bit of homework and submit an essay that is far below their best standard, not revise for the odd test or just have bad days.  If each instance of underperformance adds up in the mind of the teacher as an accumulated wealth of ‘objective’ evidence that they “can’t” or “won’t” do it, that their targets are too high, their ‘ability’ too low, or their skill-set mismatched to the subject it is hard to think how they might avoid low expectations.
  • Reliability and validity issues: As a general rule the data we use are worked out with large sets carefully tested to be as reliable as possible.  Mass testing and standardised methodologies help to ensure that reliability is as possible.  Self-fulfilling prophecy and confirmation bias may also help our systems to achieve this!  However we all have students who perform exceptionally well on exam day, confounding our expectations and careful, reliable testing over the previous years of teaching.  Unfortunately we have probably all experienced the reverse where underperformance strikes.  The same can be true of any of the testing which generates our data going into the relationship; perhaps that student doesn’t test well, or had an off-day, or was distracted during those tests.  Additional data can help (CAT scores and reading levels and KS2 SATs) but only so far.

Then there is the question of whether the data actually measure what they are supposed to – and the related question of whether I am using them for that purpose.  Many teachers can talk for hours about how the data we’re given are of questionable validity so I won’t explore this too much here.   However, coupled with the Golem Effect, confirmation bias and reliability issues for the individual student it is worth at least noting.  Sometimes we’re very sensitive to targets or information that we consider ‘too high’ or ‘inflated’.  This can certainly happen and the drive in all parts of the system to high expectations may well mean it is more likely than low expectations.  But I’m not sure I’ve always spent enough time looking for data flaws that go the other way where for some reason, or combination of reasons, my students have been given scores and targets that are too low and which I need to challenge and raise, rather than finding confirmation for, however inadvertently.

This is not to suggest that the data is pointless and can never be relied upon; big-picture and over time it can certainly be valuable.  If it sounds like my reflections suggest the data aren’t useful or should be ignored that is not the case.  Without some very convincing research to show otherwise, I’m operating on the assumption that for most students and most teachers the availability of data is a positive thing that can be well-used to support learning.  If I think back to the start of my teaching when I had very little information about most student I feel much better equipped going into a new class now.    Having seen that I have a GCSE student with a reading age of 9 I have been able to do some careful thinking about the range of Anglo-Saxon source material I am making available in my first lesson.

But I would make a case for caution and critical evaluation of the data from the very beginning.  Too often, if we allow our aspirations to be limited by the data in front of us and confirmation bias kicks in we are at risk of contributing to students’ challenges.    Of course, there are a lot of other factors that come into play.  However my goal as a teacher is to be a positive factor and not one of the hurdles my students have to overcome, and this is enough to give me pause.  The principle of falsifiability is a useful one here – to ask myself how would I know if these data are flawed, if they hide strengths either in the area they measure, such as literacy levels, or in other useful assets that aren’t directly measured such as motivation, emotional maturity or resilience and I find myself asking the following questions as I reflect on the data I’m looking at:

  • Am I reading too much into these data, and forming judgements that may limit my expectations too far?
  • If any initial expectations based on the data are misguided, how will I identify that this is the case and not fall into the trap of confirmation bias? What should I be looking for in this student’s contributions, work, ethos and attitude to learning that challenges the previous data and suggests the student may be capable of more?
  • Were these data to be fundamentally misleading for this student, understating their full potential, how would I know?
  • Is the AfL, teaching and questioning in my classroom giving all students opportunities to excel – to overcome the low expectations they may have of themselves or others may have of them?

Rosenthal, R., & Jacobson, L. (1968). Pygmalion in the classroom. The Urban Review, 3(1), 16-20.

Reflecting on… the experiences of one student from a ‘disadvantaged background’.

This is a guest blog written by a former student (@Lissyh1997). I have found it a powerful read as she recalled her experiences in education, positive and negative. @Lissyh1997 is now a trainee teacher and I know she is going to be an amazing addition to the profession.

Following the inequality that certain groups of pupils faced in the first set of A-Level results this year, it seems the perfect moment to confront the ‘injustices’ that pupils from certain backgrounds may face; not only on a national scale but also on an individual school level. Often students from disadvantaged backgrounds are pigeonholed, and assumptions are made as to how they learn, what they learn, and about their overall ability to learn in the first place. Therefore, do teachers themselves unconsciously contribute to bias? Are disadvantaged students set back by labelling from day one? How can this positively or negatively affect these students during their time throughout school and beyond?

Rosenthal and Jacobson, 1968) investigated the Pygmalion effect which demonstrated the potential of teachers in influencing performance of students based on teachers’ perceptions. They found labelling students creates a self-fulfilling prophecy, revealing a direct correlation between teacher expectation and student performance. Often, stereotyping or labelling students at the beginning of a course causes consequences as the class moves forward, adding to the belief that these students will continue to under-perform (Chambers, 1999). Coming from a disadvantaged background and qualifying for FSM my predictions were low entering secondary school. From year 7 I struggled thinking that it would be hard to succeed and wondered why I couldn’t be smart like some of my friends, often feeling inadequate when they were offered extension work to do or sat on the ‘smart’ table together.

But I suppose what really made me feel disheartened was heading to my bottom set English class. As (Gillborn & Youdell, 2000) state, disadvantaged pupils are over-represented in lower teaching sets and are at greater risk to reduced teacher expectations, pupil disruption and loss of self-esteem. I assume they made the decision to put me in bottom set English based purely on data, rather than my attitude and personality traits, unique like every student. At the time I was oblivious that perhaps I should not be in that set, but rather thought that I was bad at English and had no hope of ever improving. It was as though my potential was capped by being surrounded by students who struggled and misbehaved, and it unfortunately made me believe that I came under this umbrella too.  

Also, data is often offset by listening to parents, which means students whose parents understand, or know enough about education not to “trust the teachers” but argue their case have a good chance of moving their children up a set. In my case, neither of my parents had been to university or even knew what a set was. Luckily, I had a very supportive and invested form tutor who argued my case and made me aware that this was the wrong set for me. Sometimes students need teachers who are willing to challenge others’ decisions and question things.  As you can see, certain students are left without hope of achieving, or having chances to exceed potential solely because of labelling and that is not the purpose of education. I think it is unfair that a student could feel like they can achieve less based on a teacher’s use of data, and even today, I sometimes feel limited by the labels and doubt the quality of my essays.

As (Gillborn and Mirza, 2000) argue we cannot ignore that all pupils have a class, gender and ethnic identity, but we need to consider the difference they might make to pupil’s experience of schooling rather than putting students into certain boxes. Once again from personal experience, my ‘student profile’ had an impact on the way teachers viewed me. They concentrated more on statistics or previous students, rather than my individual story. The village I come from has a bad reputation with many students misbehaving, underachieving and showing little interest in learning. Therefore, teachers would often expect me to be the same, misbehave and not achieve much. Comments were often made ‘you are not like a typical [insert village] kid’ or ‘Are you sure you are from there?’. Looking back, I now realise that they had decided what I could achieve and what my attitude would be like based on background information, rather than who I was as an individual. The comments might seem harmless and light -hearted, but it shows that they had preconceptions about what I would be like in their classroom. Also, should I have been a ‘typical’ student from this village, would the fact I came from there really better explain my behaviour and attitude, than that I needed extra help with subjects or needing some pastoral support because of a home situation. It felt as if any student from this village, or other similar situations was written off and teachers had given up before even starting to teach or build a rapport.   It is important to remember that students are not just a number, but human beings with faces, names, dreams, and heartbreaks.   I wanted to feel not just like a student from this village who happened to exceed expectations and not fit the stereotype, but a successful student in my own right.

I believe that if we are going to have preconceived ideas about students, we should at least begin to use labelling as a positive concept and value data in order to help those students because as (Chambers, 1999) states, high expectations lead to high performance. I think we should use them to broaden horizons and challenge expectations rather than diminish potential, and this is what some amazing teachers did for me. The head of languages listened to me, truly listened, and reassured me everything would be ok. By not judging me and not grouping me into the category of a disadvantage pupil with a low prediction in languages, they allowed me to keep learning despite home situations. Moreover, they went above and beyond to help me see what I could achieve and spoke to me about university options, something I had never considered. Also, from the first week my form tutor had made his high expectations for me clear and throughout the 5 years, they continued to push me to be the best I could be. He gave me confidence to participate in lessons by giving me counters that I had to use up by the end of lessons, I could only use them up by participating. He recognised I was shy and made it his mission to include me. He taught me I had ideas worth sharing, should take risks and others could learn from me. He even gave me whole form responsibility. My Spanish teacher opened up new interests in the form of linguistics, or showing me a Latin book, and this made me realise I could enjoy other academic aspects of the subject and that he had faith in me to challenge myself. The head of maths was convinced I could get better than a D and offered extra help on a Wednesday afterschool until 6pm.

I personally believe it is about building students up and above are a few examples of how teachers decided not to label me and instead believe in me. Teachers should do, shift attitudes about students and evaluate data in front of them. Students who are ‘difficult’ or ‘disadvantaged’ could be team leaders, peer mentors, on sports teams and school councils. Instead of feeding negative stigma, we could encourage them to flourish, set goals and remind them they are not just the ‘bad behaviour’ they show.  After all, labels may stay with an individual throughout their entire life (Lauchlan & Boyle, 2007) and teachers have responsibility to reverse student’s views of themselves and allow equal opportunity. I think decisions made should be based on robust evidence alongside professional judgement and school context. Teachers ultimately have the ability to be a champion for these often- labelled students and prevent the widening of the opportunity and achievement gaps.

ALT Reflect.  Since reading this piece from @Lissyh1997 I have been reflecting on the following questions about my teaching:

  1.  What unconscious biases do I bring to the classroom as I make decisions about seating plans, groupings and differentiation?
  2. How carefully am I using data to raise my expectations for students, rather than lower them?
  3. Where will this student receive the push to excel in my subject and in their school career as a whole?

Introduce Yourself (Example Post)

This is an example post, originally published as part of Blogging University. Enroll in one of our ten programs, and start your blog right.

You’re going to publish a post today. Don’t worry about how your blog looks. Don’t worry if you haven’t given it a name yet, or you’re feeling overwhelmed. Just click the “New Post” button, and tell us why you’re here.

Why do this?

  • Because it gives new readers context. What are you about? Why should they read your blog?
  • Because it will help you focus your own ideas about your blog and what you’d like to do with it.

The post can be short or long, a personal intro to your life or a bloggy mission statement, a manifesto for the future or a simple outline of your the types of things you hope to publish.

To help you get started, here are a few questions:

  • Why are you blogging publicly, rather than keeping a personal journal?
  • What topics do you think you’ll write about?
  • Who would you love to connect with via your blog?
  • If you blog successfully throughout the next year, what would you hope to have accomplished?

You’re not locked into any of this; one of the wonderful things about blogs is how they constantly evolve as we learn, grow, and interact with one another — but it’s good to know where and why you started, and articulating your goals may just give you a few other post ideas.

Can’t think how to get started? Just write the first thing that pops into your head. Anne Lamott, author of a book on writing we love, says that you need to give yourself permission to write a “crappy first draft”. Anne makes a great point — just start writing, and worry about editing it later.

When you’re ready to publish, give your post three to five tags that describe your blog’s focus — writing, photography, fiction, parenting, food, cars, movies, sports, whatever. These tags will help others who care about your topics find you in the Reader. Make sure one of the tags is “zerotohero,” so other new bloggers can find you, too.

Introduce Yourself (Example Post)

This is an example post, originally published as part of Blogging University. Enroll in one of our ten programs, and start your blog right.

You’re going to publish a post today. Don’t worry about how your blog looks. Don’t worry if you haven’t given it a name yet, or you’re feeling overwhelmed. Just click the “New Post” button, and tell us why you’re here.

Why do this?

  • Because it gives new readers context. What are you about? Why should they read your blog?
  • Because it will help you focus your own ideas about your blog and what you’d like to do with it.

The post can be short or long, a personal intro to your life or a bloggy mission statement, a manifesto for the future or a simple outline of your the types of things you hope to publish.

To help you get started, here are a few questions:

  • Why are you blogging publicly, rather than keeping a personal journal?
  • What topics do you think you’ll write about?
  • Who would you love to connect with via your blog?
  • If you blog successfully throughout the next year, what would you hope to have accomplished?

You’re not locked into any of this; one of the wonderful things about blogs is how they constantly evolve as we learn, grow, and interact with one another — but it’s good to know where and why you started, and articulating your goals may just give you a few other post ideas.

Can’t think how to get started? Just write the first thing that pops into your head. Anne Lamott, author of a book on writing we love, says that you need to give yourself permission to write a “crappy first draft”. Anne makes a great point — just start writing, and worry about editing it later.

When you’re ready to publish, give your post three to five tags that describe your blog’s focus — writing, photography, fiction, parenting, food, cars, movies, sports, whatever. These tags will help others who care about your topics find you in the Reader. Make sure one of the tags is “zerotohero,” so other new bloggers can find you, too.

Introduce Yourself (Example Post)

This is an example post, originally published as part of Blogging University. Enroll in one of our ten programs, and start your blog right.

You’re going to publish a post today. Don’t worry about how your blog looks. Don’t worry if you haven’t given it a name yet, or you’re feeling overwhelmed. Just click the “New Post” button, and tell us why you’re here.

Why do this?

  • Because it gives new readers context. What are you about? Why should they read your blog?
  • Because it will help you focus your own ideas about your blog and what you’d like to do with it.

The post can be short or long, a personal intro to your life or a bloggy mission statement, a manifesto for the future or a simple outline of your the types of things you hope to publish.

To help you get started, here are a few questions:

  • Why are you blogging publicly, rather than keeping a personal journal?
  • What topics do you think you’ll write about?
  • Who would you love to connect with via your blog?
  • If you blog successfully throughout the next year, what would you hope to have accomplished?

You’re not locked into any of this; one of the wonderful things about blogs is how they constantly evolve as we learn, grow, and interact with one another — but it’s good to know where and why you started, and articulating your goals may just give you a few other post ideas.

Can’t think how to get started? Just write the first thing that pops into your head. Anne Lamott, author of a book on writing we love, says that you need to give yourself permission to write a “crappy first draft”. Anne makes a great point — just start writing, and worry about editing it later.

When you’re ready to publish, give your post three to five tags that describe your blog’s focus — writing, photography, fiction, parenting, food, cars, movies, sports, whatever. These tags will help others who care about your topics find you in the Reader. Make sure one of the tags is “zerotohero,” so other new bloggers can find you, too.