Abstract
The objective of the project ASIRPAreal time is to develop research management tools for real-time impact assessment based on a theoretical framework. These research management tools will help researchers and research project managers to design and conduct projects/programmes that will lead to transformative changes addressing societal challenges. Real-time assessment of “transformative research” will inform the stakeholders about their progression towards the goals intended by the projects and/or programmes. The challenges of ASIRPArt is to develop management tools based on a better understanding of the mechanisms that generate research impact (Joly et al. 2015) and to coproduce them with potential users (Robinson and Rip 2013).
Given the uncertainty and complexity that characterise the transformation processes linked to transformative research activities, we do not intend to design ballistic steering tools but to produce tools to foster learning processes, coordination and reflexivity of the actors involved. Our approach is founded on three broad streams of literature. Theory of Change approaches define goals and possible paths to achieve the intended changes (Weiss 1995). They take into account serendipity, complex contexts and situations, and flexibility (the possibility of revising the objectives). Depending on the level of dynamic monitoring, the theory of change will differ. This supposes to design a nested approach.
Processual analyses such as the Innovation Journey (Van de Ven et al. 1999) assume that transformation processes cannot be fully steered and planned because they generate new knowledge, new socio-technical associations and their effects depend on the progressive alignments of many heterogeneous elements.
Developmental evaluation (Patton 2016) and Strategic intelligence (Kuhlman, 2003) have developed tools (foresight, monitoring, etc.) that foster competences of the actors involved in transformative research activities and programming, as well as strengthen collective learning and coordination. This third set of approaches supposes to introduce non-linear thinking and to develop dynamic assessment criteria.
ASIRPArt will draw also on lessons learned from ASIRPA ex post and will mobilise the core concept of “impact pathway” (Joly et al. 2015). This ex post experience highlights that the phases of the impact pathway are qualitatively different and do not follow a linear sequence; “productive configurations” take into account both the organisational complexity of the research activities considered and their embedding in a wider context; the role of intermediaries play a key role in the dynamics of key translation processes; and the generalisation phase is often quite problematic.
The real-time evaluation process is based on an iterative model as follows:
1/ Target identification (anticipated transformation and impacts)
2/ Construction of the impact pathway by backward induction (key actors, intermediaries, blocking and facilitating factors)
3/ Identification of critical points
4/ Scenario construction
5/ Step by step decision and implementation
6/ Evaluation at each step and new loop
This sequential approach aims at conducting a process in which the main stages are analysed and scattered by internal or external information gains from the project. The principle is to identify the elements necessary to improve actors’ contribution in the present sequence, bearing in mind the uncertainty about the future.
Project Resources
fteval Journal for Research and Technology Policy Evaluation