Are Your Interventions Working?: Measuring Motivation and Other Non-Cognitive Skills

We know that development of social and emotional skills underpins achievement and social mobility, but how can we know if what we’re doing is actually working? Rebecca Martin outlines some approaches to evaluating non-cognitive skills, making sure the support gets to the pupils who need it most.

Social and emotional skills, non-cognitive outcomes, soft skills—whatever you want to call them, they matter. Competencies such as motivation, self-efficacy, meta-cognition and conscientiousness have all been shown to underpin higher academic performance in pupils and lead to more successful outcomes later in their lives.¹

Create an account to read this article

£7.00+ VAT

One-off purchase

Purchase and Download today

Register for free

No Credit Card required

Register for free
Access to 3 free articles
Free TeachingTimes Report every month

array(8) {
  ["distinct_key"]=>
  string(10) "post#43680"
  ["title"]=>
  string(45) "Whole person Whole school Whole lot of impact"
  ["categories"]=>
  array(4) {
    [0]=>
    string(13) "Mental Health"
    [1]=>
    string(10) "Resilience"
    [2]=>
    string(13) "Special Needs"
    [3]=>
    string(9) "Wellbeing"
  }
  ["url"]=>
  string(63) "https://www.teachingtimes.com/ecj61-whole-person-school-impact/"
  ["post_date_timestamp"]=>
  int(1497962010)
  ["objectID"]=>
  string(12) "post-43680-3"
  ["_snippetResult"]=>
  array(2) {
    ["excerpt"]=>
    array(2) {
      ["value"]=>
      string(115) "Jennifer Shearman and Bea Stevenson write on developing a strategy for emotional health, resilience and well-being."
      ["matchLevel"]=>
      string(4) "none"
    }
    ["content"]=>
    array(2) {
      ["value"]=>
      string(422) "... York : Ballantine BooksEducation Endowment Foundation, https://educationendowmentfoundation.org.uk/Egalite, A. J., Mills, J. N. and Green, J. P. (2015). ‘The Softer Side of Learning: Measuring students’ non-cognitive skills’, Improving Schools, pp. 9th June 2016. doi: 10.1177/1365480215616313.Fair Education Alliance, www.faireducation.org.ukFamily Links, www.familylinks ..."
      ["matchLevel"]=>
      string(7) "partial"
    }
  }
  ["_highlightResult"]=>
  array(6) {
    ["title"]=>
    array(3) {
      ["value"]=>
      string(45) "Whole person Whole school Whole lot of impact"
      ["matchLevel"]=>
      string(4) "none"
      ["matchedWords"]=>
      array(0) {
      }
    }
    ["author"]=>
    array(1) {
      [0]=>
      array(3) {
        ["value"]=>
        string(13) "Bea Stevenson"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
    }
    ["excerpt"]=>
    array(3) {
      ["value"]=>
      string(115) "Jennifer Shearman and Bea Stevenson write on developing a strategy for emotional health, resilience and well-being."
      ["matchLevel"]=>
      string(4) "none"
      ["matchedWords"]=>
      array(0) {
      }
    }
    ["content"]=>
    array(4) {
      ["value"]=>
      string(4825) "answers.  We know that there is no perfect measure of key strengths.  If there was, it would be part of every government’s manifesto, every education minister’s targets, every setting’s Ofsted framework.  It would be used as a carrot for potential researchers and a stick to beat schools that didn’t quite ‘measure up’. In our report we will outline why measuring the positive attributes that are demonstrated in a broad range of children, in a systematic way, and over time, will highlight gaps that exist and more importantly will show why the work of our partners is making a positive impact.We believe that everyone can, and will, improve.  We know that there is a positive correlation between development of these key strengths and achievement.  We know that investing time and resources in developing and improving key strengths will improve educational attainment.  We also know that children from poor families are more likely to benefit from early years, in-school and beyond-school programmes. We want to directly celebrate instances of improvement and also highlight where improvement is not happening.  Only then can we dig deeper into whether the problem is one of access to interventions, or their effectiveness.Highlighting action does not in itself mean better outcomes for children.  We all know that.  But as a working group we are feeling more confident that we have the bedrock in place and are close to committing to a set of measures to which we will hold ourselves accountable.  After that, our effectiveness will be for all to judge.(The Fair Education Alliance (FEA)  established in 2012, is a national coalition, with representation from the education, business and the charities sector, united in closing the achievement gap between rich and poor.  Its 10-year mission is to identify and define the gap, and to see a measurable reduction in all measures by 2022 through campaigning, raising awareness and direct intervention.  In order to change this, the FEA has identified five ambitious Impact Goals).  Impact Goal Three Working Group Members: CCCU, Family Links, Allen & Overy, Barnardo’s, Debate Mate, ESU, Football Beyond Borders, Girlguiding, Hackney Pirates, NCB, Nurture Group Network, ReachOut, Right to Succeed, RSA, Step Up to Serve, UBS and Whizz KidzJessica Tanner is Research Coordinator at Family Links, a national charity dedicated to the emotional health of children, families and schools.Jennifer Shearman is Senior Lecturer at Canterbury Christ Church University. Bea Stevenson is Head of Social and Emotional Learning for Family Links.ReferencesBanerjee, R., Weare, K. and Farr, W. (2014), Working with ‘Social and Emotional Aspects of Learning’ (SEAL): associations with school ethos, pupil social experiences, attendance, and attainment. Br Educ Res J, 40: 718-742Canterbury Christ Church University, www.canterbury.ac.ukChapman, C., Muijs, D. and MacAllister, J. (2011). A Study of the Impact of Federation on Student Outcomes. Nottingham: National College.Dweck, Carol S.. (2008) Mindset: The new psychology of success New York : Ballantine BooksEducation Endowment Foundation, https://educationendowmentfoundation.org.uk/Egalite, A. J., Mills, J. N. and Green, J. P. (2015). ‘The Softer Side of Learning: Measuring students’ non-cognitive skills’, Improving Schools, pp. 9th June 2016. doi: 10.1177/1365480215616313.Fair Education Alliance, www.faireducation.org.ukFamily Links, www.familylinks.org.ukGoodman, A., Joshi, H., Nasim, B. & Tyler, C. (2015) Social and emotional skills in childhood and their long-term effects on adult life: A review for the Early Intervention FoundationJennings, P.A. & Greenberg, M.T. (2009). ‘The Prosocial Classroom: Teacher Social and Emotional Competence in Relation to Student and Classroom Outcomes’, Review of Educational Research, Vol.79, No.1. pp491 – 525.Jubilee Centre for Character and Virtues, http://www.jubileecentre.ac.uk/National Institute for Health Care and Excellence, (2015).Social and Emotional Wellbeing in Primary Education (NICE pathways). Public Health England, (2015). ‘Promoting Children and Young People’s Emotional Health and Wellbeing: A Whole School Approach’. Reach Out, www.reachoutuk.orgRicker, S. (2014). The Skillset more than 3/4 of employers want  Available at: http://advice.careerbuilder.com/ [Accessed 28th July, 2016]Weare, K. (2015). What works in promoting social and emotional wellbeing and responding to mental health problems in schools? National Children’s Bureau: Partnership for mental health and wellbeing in schools.Whatif Learning, www.whatiflearning.co.uk"
      ["matchLevel"]=>
      string(7) "partial"
      ["fullyHighlighted"]=>
      bool(false)
      ["matchedWords"]=>
      array(6) {
        [0]=>
        string(13) "interventions"
        [1]=>
        string(7) "working"
        [2]=>
        string(9) "measuring"
        [3]=>
        string(3) "non"
        [4]=>
        string(9) "cognitive"
        [5]=>
        string(6) "skills"
      }
    }
    ["categories"]=>
    array(4) {
      [0]=>
      array(3) {
        ["value"]=>
        string(13) "Mental Health"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
      [1]=>
      array(3) {
        ["value"]=>
        string(10) "Resilience"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
      [2]=>
      array(3) {
        ["value"]=>
        string(13) "Special Needs"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
      [3]=>
      array(3) {
        ["value"]=>
        string(9) "Wellbeing"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
    }
    ["tags"]=>
    array(4) {
      [0]=>
      array(3) {
        ["value"]=>
        string(19) "Emotional Wellbeing"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
      [1]=>
      array(3) {
        ["value"]=>
        string(23) "Fair Education Alliance"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
      [2]=>
      array(3) {
        ["value"]=>
        string(25) "Implementation Strategies"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
      [3]=>
      array(3) {
        ["value"]=>
        string(8) "Pedagogy"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
    }
  }
}

Mental Health

Whole person Whole school Whole lot of impact

20 Jun 2017

array(8) {
  ["distinct_key"]=>
  string(10) "post#59757"
  ["title"]=>
  string(64) "How to … assess the impact of evidenced-informed interventions"
  ["categories"]=>
  array(2) {
    [0]=>
    string(21) "'How to do it' Guides"
    [1]=>
    string(24) "Professional Development"
  }
  ["url"]=>
  string(91) "https://www.teachingtimes.com/how-to-assess-the-impact-of-evidenced-informed-interventions/"
  ["post_date_timestamp"]=>
  int(1558370816)
  ["objectID"]=>
  string(12) "post-59757-2"
  ["_snippetResult"]=>
  array(2) {
    ["excerpt"]=>
    array(2) {
      ["value"]=>
      string(118) "What difference does the Research Learning Community approach make? Chris Brown provides a model for assessing impact."
      ["matchLevel"]=>
      string(4) "none"
    }
    ["content"]=>
    array(2) {
      ["value"]=>
      string(309) "... Teachers To Use Research To Improve Their Practice And The Benefits For Students That Occur As A Result, Research for All.■ Brown, C. (2017) Measuring impact and the scale-up of educational innovations: a working paper (London, UCL IOE).■ Brown, C., Daly, A. and Liou, Y-H. (2016 ..."
      ["matchLevel"]=>
      string(7) "partial"
    }
  }
  ["_highlightResult"]=>
  array(5) {
    ["title"]=>
    array(4) {
      ["value"]=>
      string(73) "How to … assess the impact of evidenced-informed interventions"
      ["matchLevel"]=>
      string(7) "partial"
      ["fullyHighlighted"]=>
      bool(false)
      ["matchedWords"]=>
      array(1) {
        [0]=>
        string(13) "interventions"
      }
    }
    ["author"]=>
    array(1) {
      [0]=>
      array(3) {
        ["value"]=>
        string(11) "Chris Brown"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
    }
    ["excerpt"]=>
    array(3) {
      ["value"]=>
      string(118) "What difference does the Research Learning Community approach make? Chris Brown provides a model for assessing impact."
      ["matchLevel"]=>
      string(4) "none"
      ["matchedWords"]=>
      array(0) {
      }
    }
    ["content"]=>
    array(4) {
      ["value"]=>
      string(5651) "involved)? How it was envisaged participants should engage with them? What was the value to them of doing so? Relevance – how was this introduced/how was it perceived?Reaction to the activity – how was it hoped participants would respond/how did they respond [might replace engagement]How was it hoped that respondent’s attitudes might change?5) LearningWhat learning it was hoped would result/what participants learned from the activities? How?Have participants gained new knowledge or skills? Has understanding of the domain changed/have perspectives changed?What access to new people has been gained and how do these help with learning?What access to new resources has been gained (e.g. new tools, methods, processes)?Do participants have access to new sources of information? What?Do participants feel confident to do what is required?6) Changes in behaviourHow was it intended that participants use the intervention? Does this differ from how participants are actually using the intervention? What support was provided to facilitate changes to their behaviour?Has it successfully replaced any previous activity?Has it unintentionally replaced other activities?In what situations are they using it most frequently?How might their use be enhanced?7) DifferenceWhat effect did the implementation have?Are teachers more successful, how?Are pupils more successful, how? (e.g. improvements in attainment, wellbeing, confidence, attendance etc.)Did any of this affect metrics that are used to evaluate performance?What has participant’s organization been able to achieve as a direct result of their engaging with the implementation?8) Reframing valueHas the roll-out of the intervention changed perspectives on what is important in terms of effective teaching and learning?Are there new success criteria or evaluation metrics that emerge as a result of this changed perspective?Has this new understanding translated into institutional changes, frameworks or systems?Professional Learning Task: Making an impact!Impact needs to be considered before as well during and after the introduction of new initiatives. Thinking about an initiative, policy or strategy that was successfully introduced to your school and one that was unsuccessfully introduced, what do you know in relation to the following:Successful introductionUnsuccessful introductionWhat were the activities involved in introducing the initiative, policy or strategy to staff?What encouragement, support or resource was offered or provided?What learning it was hoped would result of these activities? Did staff feel confident to do what is required?What support was provided to facilitate changes to staff behaviour?What difference did the initiative, policy or strategy make to teaching and learning?How might thinking about these questions in advance improve impact in the future?References■ Brown, C. (in press) Research Learning Communities: How The RLC Approach Enables Teachers To Use Research To Improve Their Practice And The Benefits For Students That Occur As A Result, Research for All.■ Brown, C. (2017) Measuring impact and the scale-up of educational innovations: a working paper (London, UCL IOE).■ Brown, C., Daly, A. and Liou, Y-H. (2016) Improving Trust, Improving Schools: Findings from a Social Network Analysis of 43 Primary Schools in England, Journal of Professional Capital & Community, 1, 1, pp. 69-91.■ Carley, K. and Krackhardt, D. (1999), Cognitive inconsistencies and non-symmetric friendship, Social Networks, 18, 1, pp. 1-27.■ Day, C. and Sammons, P. (2013) Successful leadership: a review of the international literature (Reading, CfBT Education Trust).■ Earl, L. and Katz, S. (2006) How Networked Learning Communities Work, Centre for Strategic Education Seminar Series Paper 155.■ Earl, L. and Timperley, H (2008) Understanding how evidence and learning conversations work, in: Earl, L. and Timperley, H. (Eds) Professional Learning Conversations: Challenges in Using Evidence for Improvement, (Springer).■ Earl, L. and H. Timperley (2015) Evaluative thinking for successful educational innovation, OECD Education Working Papers, No. 122, (OECD Publishing, Paris).■ Earley, P. (2013) Exploring the school leadership landscape: changing demands, changing realities, (London, Bloomsbury).■ Earley, P. and Porritt, V. (2013) Evaluating the impact of professional development: the need for a student-focused approach, Professional Development in Education, DOI: 10.1080/19415257.2013.798741■ Fielding, M., Bragg, S., Craig, J., Cunningham, I., Eraut, M., Gillinson, S., Horne, M., Robinson, C. and Thorp, J. (2005) Factors influencing the transfer of good practice, available at: http://webarchive.nationalarchives.gov.uk/20130401151715/http://www.education.gov.uk/publications/eOrderingDownload/RR615.pdf.pdf, accessed on 21 May, 2016.■ Flórez, M. T. and P. Sammons (2013) Assessment for learning: effects and impact (London, CfBT).■ Flyvbjerg, B. (2001) Making Social Science Matter (Cambridge, Cambridge University Press).■ Fullan, M. (2001). The new meaning of educational change (3rd ed.), (New York and London, Teachers College Press and RoutledgeFalmer).■ Fullan, M. (2007) The New Meaning of Educational Change. Fourth Edition, (New York: Teachers College Press).■ Fullan, M. (2011) Change Leader, (San Francisco, CA, Jossey-Bass)■ Gusky, T. (2000) Evaluating Professional Development, (Thousand Oaks, CA, Corwin Press).■ Hall, G. (2013) Evaluating change processes Assessing extent of implementation (constructs, methods and"
      ["matchLevel"]=>
      string(7) "partial"
      ["fullyHighlighted"]=>
      bool(false)
      ["matchedWords"]=>
      array(5) {
        [0]=>
        string(7) "working"
        [1]=>
        string(9) "measuring"
        [2]=>
        string(3) "non"
        [3]=>
        string(9) "cognitive"
        [4]=>
        string(6) "skills"
      }
    }
    ["categories"]=>
    array(2) {
      [0]=>
      array(3) {
        ["value"]=>
        string(21) "'How to do it' Guides"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
      [1]=>
      array(3) {
        ["value"]=>
        string(24) "Professional Development"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
    }
  }
}

'How to do it' Guides

How to … assess the impact of evidenced-informed interventions

20 May 2019

array(8) {
  ["distinct_key"]=>
  string(10) "post#43667"
  ["title"]=>
  string(63) "How to … assess the impact of evidence-informed interventions"
  ["categories"]=>
  array(2) {
    [0]=>
    string(21) "'How to do it' Guides"
    [1]=>
    string(24) "Professional Development"
  }
  ["url"]=>
  string(59) "https://www.teachingtimes.com/pdt192-assessing-the-impacts/"
  ["post_date_timestamp"]=>
  int(1497961045)
  ["objectID"]=>
  string(12) "post-43667-2"
  ["_snippetResult"]=>
  array(2) {
    ["excerpt"]=>
    array(2) {
      ["value"]=>
      string(94) "What difference does the RLC approach make? Chris Brown provides a model for assessing impact."
      ["matchLevel"]=>
      string(4) "none"
    }
    ["content"]=>
    array(2) {
      ["value"]=>
      string(309) "... Teachers To Use Research To Improve Their Practice And The Benefits For Students That Occur As A Result, Research for All.■ Brown, C. (2017) Measuring impact and the scale-up of educational innovations: a working paper (London, UCL IOE).■ Brown, C., Daly, A. and Liou, Y-H. (2016 ..."
      ["matchLevel"]=>
      string(7) "partial"
    }
  }
  ["_highlightResult"]=>
  array(5) {
    ["title"]=>
    array(4) {
      ["value"]=>
      string(72) "How to … assess the impact of evidence-informed interventions"
      ["matchLevel"]=>
      string(7) "partial"
      ["fullyHighlighted"]=>
      bool(false)
      ["matchedWords"]=>
      array(1) {
        [0]=>
        string(13) "interventions"
      }
    }
    ["author"]=>
    array(1) {
      [0]=>
      array(3) {
        ["value"]=>
        string(11) "Chris Brown"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
    }
    ["excerpt"]=>
    array(3) {
      ["value"]=>
      string(94) "What difference does the RLC approach make? Chris Brown provides a model for assessing impact."
      ["matchLevel"]=>
      string(4) "none"
      ["matchedWords"]=>
      array(0) {
      }
    }
    ["content"]=>
    array(4) {
      ["value"]=>
      string(5650) "activities, support or resource (who/what was involved)? How it was envisaged participants should engage with them? What was the value to them of doing so? Relevance – how was this introduced/how was it perceived?Reaction to the activity – how was it hoped participants would respond/how did they respond [might replace engagement]How was it hoped that respondent’s attitudes might change?5) LearningWhat learning it was hoped would result/what participants learned from the activities? How?Have participants gained new knowledge or skills? Has understanding of the domain changed/have perspectives changed?What access to new people has been gained and how do these help with learning?What access to new resources has been gained (e.g. new tools, methods, processes)?Do participants have access to new sources of information? What?Do participants feel confident to do what is required?6) Changes in behaviourHow was it intended that participants use the intervention? Does this differ from how participants are actually using the intervention? What support was provided to facilitate changes to their behaviour?Has it successfully replaced any previous activity?Has it unintentionally replaced other activities?In what situations are they using it most frequently?How might their use be enhanced?7) DifferenceWhat effect did the implementation have?Are teachers more successful, how?Are pupils more successful, how? (e.g. improvements in attainment, wellbeing, confidence, attendance etc.)Did any of this affect metrics that are used to evaluate performance?What has participant’s organization been able to achieve as a direct result of their engaging with the implementation?8) Reframing valueHas the roll-out of the intervention changed perspectives on what is important in terms of effective teaching and learning?Are there new success criteria or evaluation metrics that emerge as a result of this changed perspective?Has this new understanding translated into institutional changes, frameworks or systems?Professional Learning Task: Making an impact!Impact needs to be considered before as well during and after the introduction of new initiatives. Thinking about an initiative, policy or strategy that was successfully introduced to your school and one that was unsuccessfully introduced, what do you know in relation to the following:Successful introductionUnsuccessful introductionWhat were the activities involved in introducing the initiative, policy or strategy to staff?What encouragement, support or resource was offered or provided?What learning it was hoped would result of these activities? Did staff feel confident to do what is required?What support was provided to facilitate changes to staff behaviour?What difference did the initiative, policy or strategy make to teaching and learning?How might thinking about these questions in advance improve impact in the future?References ■ Brown, C. (in press) Research Learning Communities: How The RLC Approach Enables Teachers To Use Research To Improve Their Practice And The Benefits For Students That Occur As A Result, Research for All.■ Brown, C. (2017) Measuring impact and the scale-up of educational innovations: a working paper (London, UCL IOE).■ Brown, C., Daly, A. and Liou, Y-H. (2016) Improving Trust, Improving Schools: Findings from a Social Network Analysis of 43 Primary Schools in England, Journal of Professional Capital & Community, 1, 1, pp. 69-91.■ Carley, K. and Krackhardt, D. (1999), Cognitive inconsistencies and non-symmetric friendship, Social Networks, 18, 1, pp. 1-27.■ Day, C. and Sammons, P. (2013) Successful leadership: a review of the international literature (Reading, CfBT Education Trust).■ Earl, L. and Katz, S. (2006) How Networked Learning Communities Work, Centre for Strategic Education Seminar Series Paper 155.■ Earl, L. and Timperley, H (2008) Understanding how evidence and learning conversations work, in: Earl, L. and Timperley, H. (Eds) Professional Learning Conversations: Challenges in Using Evidence for Improvement, (Springer).■ Earl, L. and H. Timperley (2015) Evaluative thinking for successful educational innovation, OECD Education Working Papers, No. 122, (OECD Publishing, Paris).■ Earley, P. (2013) Exploring the school leadership landscape: changing demands, changing realities, (London, Bloomsbury).■ Earley, P. and Porritt, V. (2013) Evaluating the impact of professional development: the need for a student-focused approach, Professional Development in Education, DOI: 10.1080/19415257.2013.798741■ Fielding, M., Bragg, S., Craig, J., Cunningham, I., Eraut, M., Gillinson, S., Horne, M., Robinson, C. and Thorp, J. (2005) Factors influencing the transfer of good practice, available at: http://webarchive.nationalarchives.gov.uk/20130401151715/  http://www.education.gov.uk/publications/eOrderingDownload/RR615.pdf.pdf, accessed on 21 May, 2016.■ Flórez, M. T. and P. Sammons (2013) Assessment for learning: effects and impact (London, CfBT).■ Flyvbjerg, B. (2001) Making Social Science Matter (Cambridge, Cambridge University Press).■ Fullan, M. (2001). The new meaning of educational change (3rd ed.), (New York and London, Teachers College Press and RoutledgeFalmer).■ Fullan, M. (2007) The New Meaning of Educational Change. Fourth Edition, (New York: Teachers College Press).■ Fullan, M. (2011) Change Leader, (San Francisco, CA, Jossey-Bass)■ Gusky, T. (2000) Evaluating Professional Development, (Thousand Oaks, CA, Corwin Press).■ Hall, G. (2013) Evaluating change processes Assessing"
      ["matchLevel"]=>
      string(7) "partial"
      ["fullyHighlighted"]=>
      bool(false)
      ["matchedWords"]=>
      array(5) {
        [0]=>
        string(7) "working"
        [1]=>
        string(9) "measuring"
        [2]=>
        string(3) "non"
        [3]=>
        string(9) "cognitive"
        [4]=>
        string(6) "skills"
      }
    }
    ["categories"]=>
    array(2) {
      [0]=>
      array(3) {
        ["value"]=>
        string(21) "'How to do it' Guides"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
      [1]=>
      array(3) {
        ["value"]=>
        string(24) "Professional Development"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
    }
  }
}

'How to do it' Guides

How to … assess the impact of evidence-informed interventions

20 Jun 2017

array(8) {
  ["distinct_key"]=>
  string(10) "post#36238"
  ["title"]=>
  string(47) "Does Thinking for Learning live up to its name?"
  ["categories"]=>
  array(1) {
    [0]=>
    string(32) "Creative Teaching & Learning"
  }
  ["url"]=>
  string(77) "https://www.teachingtimes.com/does-thinking-for-learning-live-up-to-its-name/"
  ["post_date_timestamp"]=>
  int(1205685471)
  ["objectID"]=>
  string(12) "post-36238-3"
  ["_snippetResult"]=>
  array(2) {
    ["excerpt"]=>
    array(2) {
      ["value"]=>
      string(184) "David Moseley and Hanneke Jones of the Newcastle University Research Centre for Learning and Teaching write about problems in assessing Thinking for Learning in Northumberland schools."
      ["matchLevel"]=>
      string(4) "none"
    }
    ["content"]=>
    array(2) {
      ["value"]=>
      string(391) "... at thebeginning of this paper, teachers were strongly persuadedthat pupils benefit from TfL-focussed work in terms ofmotivation, engagement and social awareness, as well asin cognitive and academic skills. It seemed important toseek confirmation of this from pupils themselves and in theNRAIS evaluation three different instruments for measuringattitude change were ..."
      ["matchLevel"]=>
      string(7) "partial"
    }
  }
  ["_highlightResult"]=>
  array(4) {
    ["title"]=>
    array(3) {
      ["value"]=>
      string(47) "Does Thinking for Learning live up to its name?"
      ["matchLevel"]=>
      string(4) "none"
      ["matchedWords"]=>
      array(0) {
      }
    }
    ["excerpt"]=>
    array(3) {
      ["value"]=>
      string(184) "David Moseley and Hanneke Jones of the Newcastle University Research Centre for Learning and Teaching write about problems in assessing Thinking for Learning in Northumberland schools."
      ["matchLevel"]=>
      string(4) "none"
      ["matchedWords"]=>
      array(0) {
      }
    }
    ["content"]=>
    array(4) {
      ["value"]=>
      string(5681) "high degree of  metacognition. Improved examination results and uptake.Table 2Levels of integration/infusion of teaching thinking into a curriculummeans of comparing the quality of teaching and learningin supported and non-supported schools and between theschool years 2004 and 2005.One practicable approach had been suggested at thestart of the evaluation, making use of Moseley’s integratedmodel for understanding thinking and learning (Moseleyand others, 2005). The idea was to ask pupils to identifythe kinds of thinking and learning in which they hadparticipated in particular lessons. This would have providedteachers with regular feedback which they could have usedto fine-tune their lesson plans and in some cases to makeradical adjustments to the cognitive demands made ontheir pupils. It would also have encouraged teachers toinfuse thinking-for-learning activities across the curriculum.However, the NRAIS consultants were not persuaded thatthis approach could work, so it was not used at all. Althoughthere are undoubtedly challenges for potential users, pilotwork elsewhere has suggested that there is considerablepotential in an assessment tool such as this. A suitable pro-forma which teachers are welcome to use and/or modifyis provided as an appendix to this article.Another way of approaching the problem would havebeen to set up a professional debriefing system to allowteachers to carry out self-assessment using an agreedframework, such as the following one drawn up by ateam of geography teachers (see Table 2: GeographicalAssociation Secondary Committee, 2004).A framework such as the above could prove invaluable infuture evaluation of TfL interventions, but would be most 23Teaching Thinking&Creativity  Vol 8:4www.teachthinking.co.uk   focus on research and assessmentlikely to work where a culture of coaching was a key feature ofprofessional development and if the teachers concerned werewilling for the information to be confidentially shared.Measuring attitude changeAccording to the survey evidence summarised at thebeginning of this paper, teachers were strongly persuadedthat pupils benefit from TfL-focussed work in terms ofmotivation, engagement and social awareness, as well asin cognitive and academic skills. It seemed important toseek confirmation of this from pupils themselves and in theNRAIS evaluation three different instruments for measuringattitude change were used. Unfortunately not enoughschools agreed to assist with the relevant data collectionto allow for thoroughgoing statistical analysis.Attitude change in first schools was measured overthe last year of the NRAIS initiative by using a 42-itemattitude questionnaire How I am Getting on at School(HIGO). Across 7 first schools which had been involved inBerwick RAIS and which continued to receive input fromNRAIS consultants after 2003, a significant improvementin academic self-concept was found. There were alsosignificant improvements on the following items:l Our teachers listen to what we have to say.l Our teachers often let us know how well we are doing.l Our teachers try to make our lessons fun.l I like finding out what other people think.H o w e v e r  s t i l l  m o r eimpressive improvementswere found across 3 non-supported schools, whereasno significant changes inattitude were found onany of the HIGO subscalesacross 11 other first schoolssupported by NRAIS.Pupil attitude changebetween 2002 and 2005cohorts was assessed inthree NRAIS and in twonon-NRAIS middle schoolsusing the Listening to YoungPeople survey devised at theUniversity of Northumbria.Comparisons with the non-supported schools did notsupport the idea that NRAIShelped bring about relativelymore positive pupil attitudestowards education or higher educational and aspirations.Beyond this, there were a few items where pupils in NRAIS-supported schools did better and a few where they did worsethan comparison group pupils. In one of the three NRAISschools the overall trend was upwards, in another it wasdownwards and in the third there was little overall change.Attitude changes were also assessed in three NRAIS andin two non-NRAIS high schools using the Listening to YoungPeople survey. Again, comparisons did not support the ideathat pupils in NRAIS-supported schools fared better thanelsewhere. Where improvements were seen, they tendedto occur in both groups of schools. In the personal-socialarea, pupils in NRAIS-supported schools did better on asmall number of items, but this was counterbalanced byadverse changes on others.Even though the number of schools which providedattitude data was limited, the evidence summarised aboveled us to infer that the main engine for improvement inpupil performance on national tests in 2004-5 did notoperate through processes of pupil attitude change or raisedaspirations in schools supported by NRAIS consultants.So is there a solution to the mystery?If there is a direct causal connection between a pedagogicalshift towards TfL approaches and improved performanceon national tests, one would expect to find correlationallinks between reported practice and test outcomes. Weshall look at this question again and at the related question 24focus on research and assessment   www.teachthinking.co.ukTeaching Thinking&Creativity  Vol 8:4ReferencesBell, S (2007) An investigation into the effect of thinkingskills activities on the ability of pupils to answer higherorder questions. At: www.thinkingforlearning.com/resources/_resources.htmBrown, L. and Hilsdon, L. (2007) Metacognition in theFoundation Stage; How can we help young childrendevelop the language"
      ["matchLevel"]=>
      string(7) "partial"
      ["fullyHighlighted"]=>
      bool(false)
      ["matchedWords"]=>
      array(5) {
        [0]=>
        string(13) "interventions"
        [1]=>
        string(9) "measuring"
        [2]=>
        string(3) "non"
        [3]=>
        string(9) "cognitive"
        [4]=>
        string(6) "skills"
      }
    }
    ["categories"]=>
    array(1) {
      [0]=>
      array(3) {
        ["value"]=>
        string(32) "Creative Teaching & Learning"
        ["matchLevel"]=>
        string(4) "none"
        ["matchedWords"]=>
        array(0) {
        }
      }
    }
  }
}

Creative Teaching & Learning

Does Thinking for Learning live up to its name?

16 Mar 2008