Engaged universities: the view from Italian third mission national evaluation experience

brigida blasi
Italian National Agency for the Evaluation of Universities and Research Institutes (ANVUR)

LinkedIn profile Research Gate profile

sandra romagnosi
Italian National Agency for the Evaluation of Universities and Research Institutes (ANVUR)

In last decades universities have been subjected to strong pressure for change, triggering a discussion on the inner nature of academic institutions. Today’s university is recast as major player in the knowledge society and economic engine for communities; this has produced a global push for interacting externally and establishing new relationships with non-academic domains (Nedeva, 2008). Policy measures and dedicated incentives at various levels have spread out worldwide to support the development of what is generally referred as third mission of the universities, i.e. the set of activities concerned with “the generation, use, application and exploitation of knowledge and other university capabilities outside academic environments” (Molas Gallart et al., 2002: iii-iv).
However, third mission activities are by no means totally new phenomena (e.g. Liebig’s fertiliser venture in the mid-19th century) and the importance of this relationship was already noticed in many classical studies (among others: Clark, 1983; Etzkowitz & Leydesdorff, 1997). The emergent element is the societal push for interacting with non-academic domains that nowadays is leading institutions to systematize, reframe and better communicate these activities, in a word, to the institutionalization of university-industry linkages. The change is in the priority assigned to this type of activities and, therefore, the need to support, manage and organize them in a more effective way, moving from a craft production to an industrial production (Geuna & Muscio, 2009), with new organizational conditions and an upgraded governance system.
The new societal demand triggers also a growing demand for public accountability on this side. In many performance based research funding systems, such as the Britain and the Australian systems, incentives have been introduced on scientific results that benefit society, alongside quality of research and academic recruitment, what has been defined by Martin (2011) as an impact agenda. Also in Italy specific initiatives have been carried out aimed at assessing the activities of universities in interaction with non-academic domains. In particular, in recent years the Italian National Agency for the Evaluation of Universities and Research Institutes (ANVUR) has carried out two main evaluation programs focused on third mission within the framework of the national research assessment exercises, Evaluation of Research Quality (VQR) 2004–2010 and VQR 2011–2014. In particular, the participatory approach that has characterized the Italian evaluation experiences on this issue, has allowed to capture a cultural change ongoing in Italian universities and especially their recent effort in monitoring and governing the complex web of interactions that they have outside academia.
The aim of this paper is to put a spotlight on this interplay of cultural shifts, institutional rearrangements, and changes in incentives occurring in Italian universities that evaluation processes have brought to surface.
When the evaluation phase has been launched in 2012 with the first round of the research assessment exercise, VQR 2004-2010, a pioneering work has been done by ANVUR to consider the societal third mission alongside the technology one (Göransson et al., 2009)and to identify specific metrics for assessment, such as number of archeological excavations and presence of museums. Then ANVUR has promoted a long-lasting participatory process to set a perimeter around the concept of third mission and to depict the state of the art on the third mission evaluation method, data and metrics. The first step has been a consultation involving scientists from third mission related fields, experts from business sector and other stakeholders, in order to develop a wide evaluation model able to ensure fair comparison, suitable data and reliable metrics. The proposed evaluation model has been described in a Manual published by ANVUR (2015) and then tested by international experts with analogous evaluation experiences and it has been finally endorsed by a public consultation of academic community.
One of the main results of this process has been the design of a broad definition of third mission and the development of a quali-quantitative evaluation tools, that have then been used and tested in the second round of the research assessment exercise, VQR 2004-2010.
Besides activities related to valorization of research, i.e. by definition the transformation of goods supported by public funding (public research) into private goods (spin-off companies, patents, incubators …), also those “activities which have spillovers on society at large through the production of public good” have been taken into account, i.e. societal issues that have relevant impact, even though less visible, more heterogeneous and difficult to be standardized in metrics and measured, for example, throughout the creation and management of cultural heritage, Life Long Learning, clinical research and training, and the production of advice, expertise and communication of science.
As a matter of fact, as also shown by results of the OECD-EC study (forthcoming) on innovative potential of HEIs, universities in these years have developed a good awareness that third mission is more than tech transfer, stressing their cultural role and social mission and their contribution for Sustainable and Development Goals. Moreover, universities are showing an “appetite” for third mission evaluation and a demand for performance-based incentives; they are also asking for contributing bottom-up to the construction of the evaluation model and to propose their good practices (for example, music activities for the community) as standard.
Since the very first experimental evaluation processes up to now, ANVUR, through its participatory-type methodological approach, is reinforcing the institutionalization of third mission by triggering a process of surfacing of practices and standardization of monitoring systems in Italian universities.