Are Your Interventions Working?: Measuring Motivation and Other Non-Cognitive Skills

We know that development of social and emotional skills underpins achievement and social mobility, but how can we know if what we’re doing is actually working? Rebecca Martin outlines some approaches to evaluating non-cognitive skills, making sure the support gets to the pupils who need it most.

Social and emotional skills, non-cognitive outcomes, soft skills—whatever you want to call them, they matter. Competencies such as motivation, self-efficacy, meta-cognition and conscientiousness have all been shown to underpin higher academic performance in pupils and lead to more successful outcomes later in their lives.¹

Create an account to read this article

£7.00+ VAT

One-off purchase

Purchase and Download today

Register for free

No Credit Card required

Register for free
Access to 3 free articles
Free TeachingTimes Report every month

array(8) {
  ["distinct_key"]=>
  string(10) "post#43680"
  ["title"]=>
  string(45) "Whole person Whole school Whole lot of impact"
  ["categories"]=>
  array(4) {
    [0]=>
    string(13) "Mental Health"
    [1]=>
    string(10) "Resilience"
    [2]=>
    string(13) "Special Needs"
    [3]=>
    string(9) "Wellbeing"
  }
  ["url"]=>
  string(63) "https://www.teachingtimes.com/ecj61-whole-person-school-impact/"
  ["post_date_timestamp"]=>
  int(1497962010)
  ["objectID"]=>
  string(12) "post-43680-3"
  ["_snippetResult"]=>
  array(2) {
    ["excerpt"]=>
    array(2) {
      ["value"]=>
      string(115) "Jennifer Shearman and Bea Stevenson write on developing a strategy for emotional health, resilience and well-being."
      ["matchLevel"]=>
      string(4) "none"
    }
    ["content"]=>
    array(2) {
      ["value"]=>
      string(422) "... York : Ballantine BooksEducation Endowment Foundation, https://educationendowmentfoundation.org.uk/Egalite, A. J., Mills, J. N. and Green, J. P. (2015). ‘The Softer Side of Learning: Measuring students’ non-cognitive skills’, Improving Schools, pp. 9th June 2016. doi: 10.1177/1365480215616313.Fair Education Alliance, www.faireducation.org.ukFamily Links, www.familylinks ..."
      ["matchLevel"]=>
      string(7) "partial"
    }
  }
  ["_highlightResult"]=>
  array(6) {
    ["title"]=>
    array(3) {
      ["value"]=>
      string(45) "Whole person Whole school Whole lot of impact"
      ["matchLevel"]=>
      string(4) "none"
      ["matchedWords"]=>
      array(0) {
      }
    }
    ["author"]=>
    array(1) {
      [0]=>
      array(3) {
        ["value"]=>
        string(13) "Bea Stevenson"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
    }
    ["excerpt"]=>
    array(3) {
      ["value"]=>
      string(115) "Jennifer Shearman and Bea Stevenson write on developing a strategy for emotional health, resilience and well-being."
      ["matchLevel"]=>
      string(4) "none"
      ["matchedWords"]=>
      array(0) {
      }
    }
    ["content"]=>
    array(4) {
      ["value"]=>
      string(4825) "answers.  We know that there is no perfect measure of key strengths.  If there was, it would be part of every government’s manifesto, every education minister’s targets, every setting’s Ofsted framework.  It would be used as a carrot for potential researchers and a stick to beat schools that didn’t quite ‘measure up’. In our report we will outline why measuring the positive attributes that are demonstrated in a broad range of children, in a systematic way, and over time, will highlight gaps that exist and more importantly will show why the work of our partners is making a positive impact.We believe that everyone can, and will, improve.  We know that there is a positive correlation between development of these key strengths and achievement.  We know that investing time and resources in developing and improving key strengths will improve educational attainment.  We also know that children from poor families are more likely to benefit from early years, in-school and beyond-school programmes. We want to directly celebrate instances of improvement and also highlight where improvement is not happening.  Only then can we dig deeper into whether the problem is one of access to interventions, or their effectiveness.Highlighting action does not in itself mean better outcomes for children.  We all know that.  But as a working group we are feeling more confident that we have the bedrock in place and are close to committing to a set of measures to which we will hold ourselves accountable.  After that, our effectiveness will be for all to judge.(The Fair Education Alliance (FEA)  established in 2012, is a national coalition, with representation from the education, business and the charities sector, united in closing the achievement gap between rich and poor.  Its 10-year mission is to identify and define the gap, and to see a measurable reduction in all measures by 2022 through campaigning, raising awareness and direct intervention.  In order to change this, the FEA has identified five ambitious Impact Goals).  Impact Goal Three Working Group Members: CCCU, Family Links, Allen & Overy, Barnardo’s, Debate Mate, ESU, Football Beyond Borders, Girlguiding, Hackney Pirates, NCB, Nurture Group Network, ReachOut, Right to Succeed, RSA, Step Up to Serve, UBS and Whizz KidzJessica Tanner is Research Coordinator at Family Links, a national charity dedicated to the emotional health of children, families and schools.Jennifer Shearman is Senior Lecturer at Canterbury Christ Church University. Bea Stevenson is Head of Social and Emotional Learning for Family Links.ReferencesBanerjee, R., Weare, K. and Farr, W. (2014), Working with ‘Social and Emotional Aspects of Learning’ (SEAL): associations with school ethos, pupil social experiences, attendance, and attainment. Br Educ Res J, 40: 718-742Canterbury Christ Church University, www.canterbury.ac.ukChapman, C., Muijs, D. and MacAllister, J. (2011). A Study of the Impact of Federation on Student Outcomes. Nottingham: National College.Dweck, Carol S.. (2008) Mindset: The new psychology of success New York : Ballantine BooksEducation Endowment Foundation, https://educationendowmentfoundation.org.uk/Egalite, A. J., Mills, J. N. and Green, J. P. (2015). ‘The Softer Side of Learning: Measuring students’ non-cognitive skills’, Improving Schools, pp. 9th June 2016. doi: 10.1177/1365480215616313.Fair Education Alliance, www.faireducation.org.ukFamily Links, www.familylinks.org.ukGoodman, A., Joshi, H., Nasim, B. & Tyler, C. (2015) Social and emotional skills in childhood and their long-term effects on adult life: A review for the Early Intervention FoundationJennings, P.A. & Greenberg, M.T. (2009). ‘The Prosocial Classroom: Teacher Social and Emotional Competence in Relation to Student and Classroom Outcomes’, Review of Educational Research, Vol.79, No.1. pp491 – 525.Jubilee Centre for Character and Virtues, http://www.jubileecentre.ac.uk/National Institute for Health Care and Excellence, (2015).Social and Emotional Wellbeing in Primary Education (NICE pathways). Public Health England, (2015). ‘Promoting Children and Young People’s Emotional Health and Wellbeing: A Whole School Approach’. Reach Out, www.reachoutuk.orgRicker, S. (2014). The Skillset more than 3/4 of employers want  Available at: http://advice.careerbuilder.com/ [Accessed 28th July, 2016]Weare, K. (2015). What works in promoting social and emotional wellbeing and responding to mental health problems in schools? National Children’s Bureau: Partnership for mental health and wellbeing in schools.Whatif Learning, www.whatiflearning.co.uk"
      ["matchLevel"]=>
      string(7) "partial"
      ["fullyHighlighted"]=>
      bool(false)
      ["matchedWords"]=>
      array(6) {
        [0]=>
        string(13) "interventions"
        [1]=>
        string(7) "working"
        [2]=>
        string(9) "measuring"
        [3]=>
        string(3) "non"
        [4]=>
        string(9) "cognitive"
        [5]=>
        string(6) "skills"
      }
    }
    ["categories"]=>
    array(4) {
      [0]=>
      array(3) {
        ["value"]=>
        string(13) "Mental Health"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
      [1]=>
      array(3) {
        ["value"]=>
        string(10) "Resilience"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
      [2]=>
      array(3) {
        ["value"]=>
        string(13) "Special Needs"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
      [3]=>
      array(3) {
        ["value"]=>
        string(9) "Wellbeing"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
    }
    ["tags"]=>
    array(4) {
      [0]=>
      array(3) {
        ["value"]=>
        string(19) "Emotional Wellbeing"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
      [1]=>
      array(3) {
        ["value"]=>
        string(23) "Fair Education Alliance"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
      [2]=>
      array(3) {
        ["value"]=>
        string(25) "Implementation Strategies"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
      [3]=>
      array(3) {
        ["value"]=>
        string(8) "Pedagogy"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
    }
  }
}

Mental Health

Whole person Whole school Whole lot of impact

20 Jun 2017

array(8) {
  ["distinct_key"]=>
  string(10) "post#43651"
  ["title"]=>
  string(42) "Capabilities: The new currency for success"
  ["categories"]=>
  array(5) {
    [0]=>
    string(6) "Family"
    [1]=>
    string(9) "Inclusion"
    [2]=>
    string(10) "Resilience"
    [3]=>
    string(13) "Special Needs"
    [4]=>
    string(9) "Wellbeing"
  }
  ["url"]=>
  string(57) "https://www.teachingtimes.com/ecj56-currency-for-success/"
  ["post_date_timestamp"]=>
  int(1497957968)
  ["objectID"]=>
  string(12) "post-43651-2"
  ["_snippetResult"]=>
  array(2) {
    ["excerpt"]=>
    array(2) {
      ["value"]=>
      string(127) "There is, sadly, no simple recipe for success in life. But every day we understand more about what the influencing factors are."
      ["matchLevel"]=>
      string(4) "none"
    }
    ["content"]=>
    array(2) {
      ["value"]=>
      string(387) "... G and Lucas, B (2013) What kind of teaching for what kind of learning. London: SSAT.Gutman, L and Schoon, I (2013) The impact of non-cognitive skills on outcomes for young people. London: Institute of Education. Available at: https://educationendowmentfoundation.org.uk/uploads/pdf/Non-cognitive_skills_literature_review ..."
      ["matchLevel"]=>
      string(7) "partial"
    }
  }
  ["_highlightResult"]=>
  array(6) {
    ["title"]=>
    array(3) {
      ["value"]=>
      string(42) "Capabilities: The new currency for success"
      ["matchLevel"]=>
      string(4) "none"
      ["matchedWords"]=>
      array(0) {
      }
    }
    ["author"]=>
    array(1) {
      [0]=>
      array(3) {
        ["value"]=>
        string(10) "Bill Lucas"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
    }
    ["excerpt"]=>
    array(3) {
      ["value"]=>
      string(127) "There is, sadly, no simple recipe for success in life. But every day we understand more about what the influencing factors are."
      ["matchLevel"]=>
      string(4) "none"
      ["matchedWords"]=>
      array(0) {
      }
    }
    ["content"]=>
    array(4) {
      ["value"]=>
      string(5660) "report from the DfE goes further than the EEF advice in seeking to define strategies which are most likely to help disadvantaged children in Supporting the attainment of disadvantaged pupils: articulating success and good practice24, there is a helpful summary:‘The most popular strategies, and those that schools considered to be the most effective, focused on teaching and learning, especially: paired or small group additional teaching; improving feedback; and one-to-one tuition.’If all young people are to be successful in all walks of life they will need a combination of knowledge, skills and capabilities of the kinds I have been exploring delivered with the kinds of explicit focus on best teaching methods just mentioned. In fact the All-Party Parliamentary Group (APPG) on Social Mobility form has suggested that ‘personal resilience and emotional wellbeing are currently too often missing. It noted that ‘developing the social and emotional skills which give young people the resilience, persistence and motivation to deal with the stresses and the rebuffs of everyday life, are key to being able to move up the social ladder.’25 It also argued that insufficient notice was being taken by policy makers of evidence that skills such as resilience, self-belief and persistence both underpin academic success and can be taught. The group therefore proposed that in addition to policy interventions to develop character focused on Early Years, schools should put in place programmes to develop character. Across the world governments are waking up the importance of a focus on capabilities in addition to one on knowledge and skills. Finland is moving in this direction26 and Australia has already got there with its capability-led curriculum27. Sadly the UK in general and England in particular has a considerable way to go, despite occasional positive noises. But if we want all British children to succeed, we should start embracing approaches to developing capabilities and character which have been shown to work, with a much greater sense of urgency.Bill Lucas is Professor of Learning and Director of the Centre for Real-World Learning at the University of Winchester. His most recent book, written with Guy Claxton, is Educating Ruby: what our children really need to learn. He is the co-founder of one of the largest teacher researcher networks – www.expansiveeducation.net.References The Sutton Trust http://www.suttontrust.com/research https://educationendowmentfoundation.org.uk/about Ofsted (2013) The impact of early education as a strategy in countering socio-economic disadvantage. London: Ofstedhttps://educationendowmentfoundation.org.uk/our-work/projects/achievement-for-all Hattie, J (2009) Visible Learning: A Synthesis of over 800 meta-analyses relating to achievement. Abingdon: Routledge.Lucas, B and Claxton, G (2010) New Kinds of Smart: How the Science of Learnable Intelligence is Changing Education. Maidenhead: Open University PressDuckworth, A and Seligman, M (2005) Self-Discipline Outdoes IQ in Predicting Academic Performance of Adolescents. Psychological Science. 16: 12; 939-944 Heckman J and Kautz T (2013) Fostering and measuring skills interventions that improve character and cognition.  Bonn:  Institute for the Study of Labor. IZA Discussion Paper, No. 7750. Available at: http://hdl.handle.net/10419/90084 Character Education Partnership (2015) 2015 National Forum on Character Education. Available at: http://character.org   See for example, Lickona, T, and Davidson, M (2005). Smart and good high schools: Integrating excellence and ethics for success in school, work, and beyond. Cortland, NY: Center for the 4th and 5th Rs (Respect and Responsibility).Washington, D.C.: Character Education Partnership.Demos (2011) Character is an essential ingredient of a good life and a good society. Available at:  http://www.demos.co.uk/projects/the-character-inquiry  http://www.jubileecentre.ac.uk Claxton, G and Lucas, B (2013) What kind of teaching for what kind of learning. London: SSAT.Gutman, L and Schoon, I (2013) The impact of non-cognitive skills on outcomes for young people. London: Institute of Education. Available at: https://educationendowmentfoundation.org.uk/uploads/pdf/Non-cognitive_skills_literature_review_2.pdfDuckworth A, Peterson C, Matthews M and Kelly D (2007) Grit: Perseverance and Passion for Long-Term Goals. Journal of Personality and Social Psychology 92(6): 1087-1101Duckworth A L and Seligman M (2006) Self-Discipline Gives Girls the Edge: Gender in Self-Discipline, Grades, and Achievement Test Scores. Journal of Educational Psychology 98(1): 198-208 Tough, P (2016) Helping children succeed: What Works and Why. London: Random House Bookshttp://www.educatingruby.org https://www.buildinglearningpower.com http://www.p21.org http://npdl.global https://educationendowmentfoundation.org.uk/resources/teaching-learning-toolkit/meta-cognition-and-self-regulation https://educationendowmentfoundation.org.uk/resources/teaching-learning-toolkit/social-and-emotional-learning https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/473974/DFE-RR411_Supporting_the_attainment_of_disadvantaged_pupils.pdfAll-Party Parliamentary Group on Social Mobility (2012) 7 key truths about social mobility. London: House of Commons. Available at: http://www.raeng.org.uk/publications/other/7-key-truths-about-social-mobility"
      ["matchLevel"]=>
      string(7) "partial"
      ["fullyHighlighted"]=>
      bool(false)
      ["matchedWords"]=>
      array(6) {
        [0]=>
        string(13) "interventions"
        [1]=>
        string(9) "measuring"
        [2]=>
        string(10) "motivation"
        [3]=>
        string(3) "non"
        [4]=>
        string(9) "cognitive"
        [5]=>
        string(6) "skills"
      }
    }
    ["categories"]=>
    array(5) {
      [0]=>
      array(3) {
        ["value"]=>
        string(6) "Family"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
      [1]=>
      array(3) {
        ["value"]=>
        string(9) "Inclusion"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
      [2]=>
      array(3) {
        ["value"]=>
        string(10) "Resilience"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
      [3]=>
      array(3) {
        ["value"]=>
        string(13) "Special Needs"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
      [4]=>
      array(3) {
        ["value"]=>
        string(9) "Wellbeing"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
    }
    ["tags"]=>
    array(7) {
      [0]=>
      array(3) {
        ["value"]=>
        string(3) "AfA"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
      [1]=>
      array(3) {
        ["value"]=>
        string(13) "Disadvantaged"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
      [2]=>
      array(3) {
        ["value"]=>
        string(18) "education outcomes"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
      [3]=>
      array(3) {
        ["value"]=>
        string(22) "Pedagogical strategies"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
      [4]=>
      array(3) {
        ["value"]=>
        string(17) "social background"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
      [5]=>
      array(3) {
        ["value"]=>
        string(15) "social mobility"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
      [6]=>
      array(3) {
        ["value"]=>
        string(23) "structured conversation"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
    }
  }
}

Family

Capabilities: The new currency for success

20 Jun 2017

array(8) {
  ["distinct_key"]=>
  string(10) "post#59757"
  ["title"]=>
  string(64) "How to … assess the impact of evidenced-informed interventions"
  ["categories"]=>
  array(2) {
    [0]=>
    string(21) "'How to do it' Guides"
    [1]=>
    string(24) "Professional Development"
  }
  ["url"]=>
  string(91) "https://www.teachingtimes.com/how-to-assess-the-impact-of-evidenced-informed-interventions/"
  ["post_date_timestamp"]=>
  int(1558370816)
  ["objectID"]=>
  string(12) "post-59757-2"
  ["_snippetResult"]=>
  array(2) {
    ["excerpt"]=>
    array(2) {
      ["value"]=>
      string(118) "What difference does the Research Learning Community approach make? Chris Brown provides a model for assessing impact."
      ["matchLevel"]=>
      string(4) "none"
    }
    ["content"]=>
    array(2) {
      ["value"]=>
      string(309) "... Teachers To Use Research To Improve Their Practice And The Benefits For Students That Occur As A Result, Research for All.■ Brown, C. (2017) Measuring impact and the scale-up of educational innovations: a working paper (London, UCL IOE).■ Brown, C., Daly, A. and Liou, Y-H. (2016 ..."
      ["matchLevel"]=>
      string(7) "partial"
    }
  }
  ["_highlightResult"]=>
  array(5) {
    ["title"]=>
    array(4) {
      ["value"]=>
      string(73) "How to … assess the impact of evidenced-informed interventions"
      ["matchLevel"]=>
      string(7) "partial"
      ["fullyHighlighted"]=>
      bool(false)
      ["matchedWords"]=>
      array(1) {
        [0]=>
        string(13) "interventions"
      }
    }
    ["author"]=>
    array(1) {
      [0]=>
      array(3) {
        ["value"]=>
        string(11) "Chris Brown"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
    }
    ["excerpt"]=>
    array(3) {
      ["value"]=>
      string(118) "What difference does the Research Learning Community approach make? Chris Brown provides a model for assessing impact."
      ["matchLevel"]=>
      string(4) "none"
      ["matchedWords"]=>
      array(0) {
      }
    }
    ["content"]=>
    array(4) {
      ["value"]=>
      string(5651) "involved)? How it was envisaged participants should engage with them? What was the value to them of doing so? Relevance – how was this introduced/how was it perceived?Reaction to the activity – how was it hoped participants would respond/how did they respond [might replace engagement]How was it hoped that respondent’s attitudes might change?5) LearningWhat learning it was hoped would result/what participants learned from the activities? How?Have participants gained new knowledge or skills? Has understanding of the domain changed/have perspectives changed?What access to new people has been gained and how do these help with learning?What access to new resources has been gained (e.g. new tools, methods, processes)?Do participants have access to new sources of information? What?Do participants feel confident to do what is required?6) Changes in behaviourHow was it intended that participants use the intervention? Does this differ from how participants are actually using the intervention? What support was provided to facilitate changes to their behaviour?Has it successfully replaced any previous activity?Has it unintentionally replaced other activities?In what situations are they using it most frequently?How might their use be enhanced?7) DifferenceWhat effect did the implementation have?Are teachers more successful, how?Are pupils more successful, how? (e.g. improvements in attainment, wellbeing, confidence, attendance etc.)Did any of this affect metrics that are used to evaluate performance?What has participant’s organization been able to achieve as a direct result of their engaging with the implementation?8) Reframing valueHas the roll-out of the intervention changed perspectives on what is important in terms of effective teaching and learning?Are there new success criteria or evaluation metrics that emerge as a result of this changed perspective?Has this new understanding translated into institutional changes, frameworks or systems?Professional Learning Task: Making an impact!Impact needs to be considered before as well during and after the introduction of new initiatives. Thinking about an initiative, policy or strategy that was successfully introduced to your school and one that was unsuccessfully introduced, what do you know in relation to the following:Successful introductionUnsuccessful introductionWhat were the activities involved in introducing the initiative, policy or strategy to staff?What encouragement, support or resource was offered or provided?What learning it was hoped would result of these activities? Did staff feel confident to do what is required?What support was provided to facilitate changes to staff behaviour?What difference did the initiative, policy or strategy make to teaching and learning?How might thinking about these questions in advance improve impact in the future?References■ Brown, C. (in press) Research Learning Communities: How The RLC Approach Enables Teachers To Use Research To Improve Their Practice And The Benefits For Students That Occur As A Result, Research for All.■ Brown, C. (2017) Measuring impact and the scale-up of educational innovations: a working paper (London, UCL IOE).■ Brown, C., Daly, A. and Liou, Y-H. (2016) Improving Trust, Improving Schools: Findings from a Social Network Analysis of 43 Primary Schools in England, Journal of Professional Capital & Community, 1, 1, pp. 69-91.■ Carley, K. and Krackhardt, D. (1999), Cognitive inconsistencies and non-symmetric friendship, Social Networks, 18, 1, pp. 1-27.■ Day, C. and Sammons, P. (2013) Successful leadership: a review of the international literature (Reading, CfBT Education Trust).■ Earl, L. and Katz, S. (2006) How Networked Learning Communities Work, Centre for Strategic Education Seminar Series Paper 155.■ Earl, L. and Timperley, H (2008) Understanding how evidence and learning conversations work, in: Earl, L. and Timperley, H. (Eds) Professional Learning Conversations: Challenges in Using Evidence for Improvement, (Springer).■ Earl, L. and H. Timperley (2015) Evaluative thinking for successful educational innovation, OECD Education Working Papers, No. 122, (OECD Publishing, Paris).■ Earley, P. (2013) Exploring the school leadership landscape: changing demands, changing realities, (London, Bloomsbury).■ Earley, P. and Porritt, V. (2013) Evaluating the impact of professional development: the need for a student-focused approach, Professional Development in Education, DOI: 10.1080/19415257.2013.798741■ Fielding, M., Bragg, S., Craig, J., Cunningham, I., Eraut, M., Gillinson, S., Horne, M., Robinson, C. and Thorp, J. (2005) Factors influencing the transfer of good practice, available at: http://webarchive.nationalarchives.gov.uk/20130401151715/http://www.education.gov.uk/publications/eOrderingDownload/RR615.pdf.pdf, accessed on 21 May, 2016.■ Flórez, M. T. and P. Sammons (2013) Assessment for learning: effects and impact (London, CfBT).■ Flyvbjerg, B. (2001) Making Social Science Matter (Cambridge, Cambridge University Press).■ Fullan, M. (2001). The new meaning of educational change (3rd ed.), (New York and London, Teachers College Press and RoutledgeFalmer).■ Fullan, M. (2007) The New Meaning of Educational Change. Fourth Edition, (New York: Teachers College Press).■ Fullan, M. (2011) Change Leader, (San Francisco, CA, Jossey-Bass)■ Gusky, T. (2000) Evaluating Professional Development, (Thousand Oaks, CA, Corwin Press).■ Hall, G. (2013) Evaluating change processes Assessing extent of implementation (constructs, methods and"
      ["matchLevel"]=>
      string(7) "partial"
      ["fullyHighlighted"]=>
      bool(false)
      ["matchedWords"]=>
      array(5) {
        [0]=>
        string(7) "working"
        [1]=>
        string(9) "measuring"
        [2]=>
        string(3) "non"
        [3]=>
        string(9) "cognitive"
        [4]=>
        string(6) "skills"
      }
    }
    ["categories"]=>
    array(2) {
      [0]=>
      array(3) {
        ["value"]=>
        string(21) "'How to do it' Guides"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
      [1]=>
      array(3) {
        ["value"]=>
        string(24) "Professional Development"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
    }
  }
}

'How to do it' Guides

How to … assess the impact of evidenced-informed interventions

20 May 2019

array(8) {
  ["distinct_key"]=>
  string(10) "post#43667"
  ["title"]=>
  string(63) "How to … assess the impact of evidence-informed interventions"
  ["categories"]=>
  array(2) {
    [0]=>
    string(21) "'How to do it' Guides"
    [1]=>
    string(24) "Professional Development"
  }
  ["url"]=>
  string(59) "https://www.teachingtimes.com/pdt192-assessing-the-impacts/"
  ["post_date_timestamp"]=>
  int(1497961045)
  ["objectID"]=>
  string(12) "post-43667-2"
  ["_snippetResult"]=>
  array(2) {
    ["excerpt"]=>
    array(2) {
      ["value"]=>
      string(94) "What difference does the RLC approach make? Chris Brown provides a model for assessing impact."
      ["matchLevel"]=>
      string(4) "none"
    }
    ["content"]=>
    array(2) {
      ["value"]=>
      string(309) "... Teachers To Use Research To Improve Their Practice And The Benefits For Students That Occur As A Result, Research for All.■ Brown, C. (2017) Measuring impact and the scale-up of educational innovations: a working paper (London, UCL IOE).■ Brown, C., Daly, A. and Liou, Y-H. (2016 ..."
      ["matchLevel"]=>
      string(7) "partial"
    }
  }
  ["_highlightResult"]=>
  array(5) {
    ["title"]=>
    array(4) {
      ["value"]=>
      string(72) "How to … assess the impact of evidence-informed interventions"
      ["matchLevel"]=>
      string(7) "partial"
      ["fullyHighlighted"]=>
      bool(false)
      ["matchedWords"]=>
      array(1) {
        [0]=>
        string(13) "interventions"
      }
    }
    ["author"]=>
    array(1) {
      [0]=>
      array(3) {
        ["value"]=>
        string(11) "Chris Brown"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
    }
    ["excerpt"]=>
    array(3) {
      ["value"]=>
      string(94) "What difference does the RLC approach make? Chris Brown provides a model for assessing impact."
      ["matchLevel"]=>
      string(4) "none"
      ["matchedWords"]=>
      array(0) {
      }
    }
    ["content"]=>
    array(4) {
      ["value"]=>
      string(5650) "activities, support or resource (who/what was involved)? How it was envisaged participants should engage with them? What was the value to them of doing so? Relevance – how was this introduced/how was it perceived?Reaction to the activity – how was it hoped participants would respond/how did they respond [might replace engagement]How was it hoped that respondent’s attitudes might change?5) LearningWhat learning it was hoped would result/what participants learned from the activities? How?Have participants gained new knowledge or skills? Has understanding of the domain changed/have perspectives changed?What access to new people has been gained and how do these help with learning?What access to new resources has been gained (e.g. new tools, methods, processes)?Do participants have access to new sources of information? What?Do participants feel confident to do what is required?6) Changes in behaviourHow was it intended that participants use the intervention? Does this differ from how participants are actually using the intervention? What support was provided to facilitate changes to their behaviour?Has it successfully replaced any previous activity?Has it unintentionally replaced other activities?In what situations are they using it most frequently?How might their use be enhanced?7) DifferenceWhat effect did the implementation have?Are teachers more successful, how?Are pupils more successful, how? (e.g. improvements in attainment, wellbeing, confidence, attendance etc.)Did any of this affect metrics that are used to evaluate performance?What has participant’s organization been able to achieve as a direct result of their engaging with the implementation?8) Reframing valueHas the roll-out of the intervention changed perspectives on what is important in terms of effective teaching and learning?Are there new success criteria or evaluation metrics that emerge as a result of this changed perspective?Has this new understanding translated into institutional changes, frameworks or systems?Professional Learning Task: Making an impact!Impact needs to be considered before as well during and after the introduction of new initiatives. Thinking about an initiative, policy or strategy that was successfully introduced to your school and one that was unsuccessfully introduced, what do you know in relation to the following:Successful introductionUnsuccessful introductionWhat were the activities involved in introducing the initiative, policy or strategy to staff?What encouragement, support or resource was offered or provided?What learning it was hoped would result of these activities? Did staff feel confident to do what is required?What support was provided to facilitate changes to staff behaviour?What difference did the initiative, policy or strategy make to teaching and learning?How might thinking about these questions in advance improve impact in the future?References ■ Brown, C. (in press) Research Learning Communities: How The RLC Approach Enables Teachers To Use Research To Improve Their Practice And The Benefits For Students That Occur As A Result, Research for All.■ Brown, C. (2017) Measuring impact and the scale-up of educational innovations: a working paper (London, UCL IOE).■ Brown, C., Daly, A. and Liou, Y-H. (2016) Improving Trust, Improving Schools: Findings from a Social Network Analysis of 43 Primary Schools in England, Journal of Professional Capital & Community, 1, 1, pp. 69-91.■ Carley, K. and Krackhardt, D. (1999), Cognitive inconsistencies and non-symmetric friendship, Social Networks, 18, 1, pp. 1-27.■ Day, C. and Sammons, P. (2013) Successful leadership: a review of the international literature (Reading, CfBT Education Trust).■ Earl, L. and Katz, S. (2006) How Networked Learning Communities Work, Centre for Strategic Education Seminar Series Paper 155.■ Earl, L. and Timperley, H (2008) Understanding how evidence and learning conversations work, in: Earl, L. and Timperley, H. (Eds) Professional Learning Conversations: Challenges in Using Evidence for Improvement, (Springer).■ Earl, L. and H. Timperley (2015) Evaluative thinking for successful educational innovation, OECD Education Working Papers, No. 122, (OECD Publishing, Paris).■ Earley, P. (2013) Exploring the school leadership landscape: changing demands, changing realities, (London, Bloomsbury).■ Earley, P. and Porritt, V. (2013) Evaluating the impact of professional development: the need for a student-focused approach, Professional Development in Education, DOI: 10.1080/19415257.2013.798741■ Fielding, M., Bragg, S., Craig, J., Cunningham, I., Eraut, M., Gillinson, S., Horne, M., Robinson, C. and Thorp, J. (2005) Factors influencing the transfer of good practice, available at: http://webarchive.nationalarchives.gov.uk/20130401151715/  http://www.education.gov.uk/publications/eOrderingDownload/RR615.pdf.pdf, accessed on 21 May, 2016.■ Flórez, M. T. and P. Sammons (2013) Assessment for learning: effects and impact (London, CfBT).■ Flyvbjerg, B. (2001) Making Social Science Matter (Cambridge, Cambridge University Press).■ Fullan, M. (2001). The new meaning of educational change (3rd ed.), (New York and London, Teachers College Press and RoutledgeFalmer).■ Fullan, M. (2007) The New Meaning of Educational Change. Fourth Edition, (New York: Teachers College Press).■ Fullan, M. (2011) Change Leader, (San Francisco, CA, Jossey-Bass)■ Gusky, T. (2000) Evaluating Professional Development, (Thousand Oaks, CA, Corwin Press).■ Hall, G. (2013) Evaluating change processes Assessing"
      ["matchLevel"]=>
      string(7) "partial"
      ["fullyHighlighted"]=>
      bool(false)
      ["matchedWords"]=>
      array(5) {
        [0]=>
        string(7) "working"
        [1]=>
        string(9) "measuring"
        [2]=>
        string(3) "non"
        [3]=>
        string(9) "cognitive"
        [4]=>
        string(6) "skills"
      }
    }
    ["categories"]=>
    array(2) {
      [0]=>
      array(3) {
        ["value"]=>
        string(21) "'How to do it' Guides"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
      [1]=>
      array(3) {
        ["value"]=>
        string(24) "Professional Development"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
    }
  }
}

'How to do it' Guides

How to … assess the impact of evidence-informed interventions

20 Jun 2017