With this ‘HOW TO’ I show how the Dialogic Model of Impact (DMI) (Brown, 2017) can be used to measure the impact of evidence-informed interventions. DMI was designed with the specific purpose of evaluating the impact of Renfrew County Catholic District School Board’s roll-out and expansion of the Through their Eyes: Documenting Literacy and Learning in Kindergarten approach to teaching and learning. DMI has general applicability however, and can be used to measure the impact of any innovation.
Key components
Underpinning DMI is the idea that impact comprises three key components or characteristics:
- the content of an innovation;
- the processes involved as part of the innovation; and
- the context in which the innovation is situated.
The first of these, the ‘content’ characteristics, considers the ‘what’ of the innovation: for example, the new knowledge, skills or understanding that it is hoped the innovation will deliver. Second, ‘processes’ concern the ‘how’ – how will the innovation be planned, organized, delivered, continuously supported and so on. Finally ‘context’ characteristics are the ‘who’, ‘when’, ‘where’ and ‘why’ of the innovation – e.g. those innovating, the recipients of the innovation, as well as the systems, cultures and wider environment into which the innovation will be introduced. Simultaneously, evaluations should also consider the ‘quality’ of the innovation, the extent to which it has changed teacher knowledge and practice and, as a result, how the innovation has affected student outcomes (Guskey, 2000).
Measuring commitment to the RLC approach
Alongside these components it is also useful to consider how the recipients for a new innovation have engaged with it: in other words to examine scales or rubrics which examine the extent to which people feel invested in or concerned in relation to any new approach; the extent to which they will employ the new approach or even the ways in which they do so; and, as a result, the likelihood that any new approach will impact on practice. These highlight that innovators need to explicitly consider the following features:
- what specific problem the innovation is designed to address and the wider context encapsulating this problem;
- how recipients might come to learn about the innovation;
- the ways in which recipients might use the innovation and whether different interactions, engagement in different activities, or even different understandings of the problem or context, might impact on learning and understanding, as well as the levels of practice and pupil outcome change that are ultimately achieved.
By spotlighting the why and how of an innovation through the use of the engagement indicators, the logic of its operation is necessarily brought to the fore as well as an understanding of what might help or hinder its roll-out.