Privacy Policy GDPR
0 0

Basket

No products in the basket.

Research Impact Toolkit

Planning, capturing, communicating, and monitoring the impact of research in higher education institutions

Liam Cleere
Senior Manager, Research Analytics and Impact, UCD Research and Innovation, University College Dublin

This article explores what ‘research impact’ means and the approaches that Irish higher education institutions and funding agencies are using to plan, capture, communicate, and monitor the impact of research on society. In particular it details what UCD has done in response to the research impact challenge.

What is research impact?

Research impact is the effect that research has on society. Research contributes to the world in many ways – it improves our health and well-being, creates economic prosperity, enriches our cultural lives, improves environmental sustainability, and contributes to government policymaking.

Approximately €800m per year of Irish tax-payers’ money goes to fund research in higher education institutions (HEIs),1 prompting an important question: What is the benefit of this investment for citizens in Ireland and further afield? This need for accountability of public spending is a key driver of the so-called impact agenda.

The research impact agenda

The research impact agenda has been gaining momentum in recent years as policymakers grapple with the need to prove the value of research conducted in HEIs. The idea of assessing the impact of research in a serious and systematic way originated in Australia, with the abandoned Research Quality Framework (RQF)2 in the mid-2000s. The RQF used case studies to describe the impact of completed research – a system eventually adopted in the UK’s 2014 Research Excellence Framework (REF).3 Countries and regions across Europe have taken different approaches4 to assessing research impact, but to date there appears to be no common taxonomy or consistent way of doing so.

In Ireland, research impact has historically been expressed through bibliometric analysis of research publications in peer-reviewed journals.5 In 2012, following the financial crisis, research prioritisation emerged as the government’s primary policy goal in science, technology, and innovation.6 This saw a concentration of the competitive research funding on areas deemed most likely to secure greater economic and societal impact, particularly in the form of jobs and foreign direct investment. The research prioritisation areas were refreshed in 2017 to ensure that Ireland ‘optimises the opportunities arising from new science and technology developments and disruptions’.

Launched in 2015, Innovation 2020 is Ireland’s current strategy for research and development, science, and technology.7 Using the Report of the Research Prioritisation Steering Group as an input, it calls for ‘excellent research to be performed in strategically important areas with relevance and impact for the economy and society’.

Research funding agencies have been quick to embed research impact into their strategies. Science Foundation Ireland (SFI) published its strategy document Agenda 2020 in 2013,8 setting out a vision in which Ireland will be the ‘best country in the world for both scientific research excellence and impact’ by 2020. To help implement this strategy, SFI developed an impact framework that provides guidance on how to prepare an impact statement and how metrics and narrative in support of impact should be reported.9 An impact statement is a short section in a research proposal that explains the significance of the research.

Similarly, the Irish Research Council’s (IRC) strategy identifies opportunities to further improve the impact and reach of the research it supports.10 The IRC strategy looks to establish and implement a new ‘broad-based, comprehensive impact framework for IRC-funded researchers’. The IRC also wishes to ‘regularly publish and disseminate quantitative and qualitative information on the impact of the awards they fund across all disciplines’.

Further afield, the European Union, through the Horizon 2020 research funding initiative, broadly defines impact as ‘the wider societal, economic, or environmental cumulative changes over a longer period of time’. Impact was first included as a selection and award criterion for research in the 7th Framework Programme (2007–2013). In Horizon 2020 (2014–2020), impact gained importance as one of three evaluation criteria, after ‘excellence’ and before ‘quality and efficiency of the implementation’. Horizon Europe (2021–2027), the 9th Framework Programme, continues Horizon 2020’s focus on impact assessment.

The reasoning behind the growing international move towards assessing research impact is complex, involving political and socio-economic factors. In the literature on impact, four critical justifications for assessing research impact are generally cited:

  1. Higher Education Institutions overview – To enable research organisations to monitor and manage their performance and understand the contribution they are making to communities.
  2. Accountability – To demonstrate the value of research to government, stakeholders, and the wider public.
  3. Inform funding – To understand the socio-economic value of research and subsequently inform funding decisions.
  4. Impact Journey – To understand the method and routes by which research leads to impacts, optimising the potential of research findings and developing better ways of delivering impact.11

Impact journey

The impact journey describes how research can lead to impacts on society and the economy. It traces research over time, distinguishing between different five stages on the pathway to impact:

  • Inputs include the human, financial, organisational, and community resources needed to perform the research.
  • Activities are the processes, tools, events, technology, and actions that are an intentional part of the research.
  • Outputs are products of research: scholarly publications (but not forgetting grey literature), products: prototype artefacts, research datasets, software; patents.
  • Outcomes are the results or consequences of the research activities and outputs on academia, society, or the economy.
  • Impacts are changes in society that result from outputs and outcomes.

The diagram below, based on the Kellogg Foundation logic model, demonstrates this pathway, with examples under each of the five stages.12

Although this a simple model – in reality, research journeys are non-linear and far more complex – it is a useful framework for thinking about what is needed for research to have an effect on the world.

The challenge

The methodological challenges in assessing research impact have been well documented.13 To plan, capture, communicate, and monitor impact, researchers need to think systematically about the various ways people can benefit from their work. This is more important than ever, as major funding bodies around the world now consider impact a fundamental aspect of almost all research programmes.

Although the impact of some research is apparent straight away, in other cases it can take years, even decades. These impacts may be the result of hundreds of factors, of which the research is just one. Research can affect all aspects of society,14 from culture to policy to the environment. A single project can have impact in many areas, and one impact may have knock-on effects elsewhere in society. These distant time horizons and tangled pathways can make it extremely difficult to plan, capture, communicate, and monitor impact.

What UCD has done

In response to the research impact challenge, a UCD ‘Beyond Publications’ steering committee was set up in 2013 to investigate the definitions, evidence, and systems for capturing outputs beyond publications, and the resulting impacts of research from the perspective of the university.

The committee’s report, published in 2014, recommended that UCD should strive to be a leader in the field of impact capture, measurement, and communication.15 In response, a research impact work programme was initiated in 2015 to develop capacity for research impact at UCD. It yielded many practical supports and resources for researchers:

  • A website aimed at researchers and research managers who are keen to understand how social media, and other online channels, can be leveraged to promote their research and its outcomes.16
  • A new researcher profiles system to record impact.17 The system is used to create researcher profile pages on UCD websites. The publicly available profiles are used by academics, industry, and organisations to identify potential collaborators for research projects, and by potential students to identify supervisors.
  • An annual research impact case study competition to build capacity throughout UCD in capturing research impact. It encourages all researchers, regardless of discipline, to consider and celebrate the impact of their work. The competition offers researchers at UCD an opportunity to develop a short, written case study, with illustrative images, highlighting the impact of their research to an external, non-specialist audience.18
  • A research impact portal website providing supports and resources to help researchers to plan, capture, communicate, and monitor their impact.19
  • A research impact website to showcase UCD’s research impact case studies in areas such as Agri-Food, Culture, Economy and Society, Environment, Health, and ICT.20
  • An impact seminar series bringing national and international research impact experts to UCD, offering valuable impact-related insights to researchers and the wider research community across UCD and Ireland.
  • A selection of tools to support research impact, such as the UCD Impact Planning Canvas, Elsevier SciVal (a bibliometric analysis tool), and Altmetrics.21 Altmetrics provide an indication of where academic papers are being used, by counting downloads, views, and references in policy documents, as well as mentions in social media, blogs, and so on. Altmetrics provide a new way of evidencing societal impact.
  • A dedicated Research Analytics and Impact team that promotes the university’s research reputation, embeds a culture of research impact throughout the university, and informs research strategy and performance through advanced analytics.22

What are we working on now?

The Higher Education Authority (HEA) recently awarded funding to UCD to develop and share its impact-related resources, such as an online toolkit and online training material. With support from the IRC, UCD is consulting Ireland’s wider research sector on this project, with the aim of helping the sector to speak with one voice and avoid duplication of effort.

A research impact working group has been formed to facilitate this collaboration, with representatives from Ireland’s universities, institutes of technology, funding agencies, and sectoral representative bodies. Using feedback from this group, UCD will update and expand its impact-related tools, resources, and definitions and share these across the sector. This work is due to complete in December 2020.

Through this project, UCD will develop new impact resources, refine its existing resources, and consolidate their place on the UCD website as a new Research Impact Toolkit resource. Existing resources can be found on UCD’s impact homepage.23 The supports and resources we will develop through this project include the following:

  • videos giving a short overview of research impact, associated terminology, and importance in the research ecosystem
  • definitions of impact and related terms
  • a taxonomy (or classification system) of types of societal impact
  • an updated Impact Planning Canvas
  • guidance on what makes a good case study
  • guidance on writing impact sections in funding applications
  • links to existing resources by other HEIs and third parties
  • exemplar impact case studies to demonstrate best practice.

Sharing the Research Impact Toolkit across the sector will increase efficiency, ensuring different institutions and organisations do not duplicate efforts. By working together in this way, the Irish research sector will be better able to demonstrate value for money of publicly funded research. Ultimately, with consistent messaging, researchers will be better able to plan for impact, yielding greater benefits for society both nationally and internationally.

ENDNOTES

  1. Department of Business, Enterprise and Innovation (2019) Research and Development Budget (R&D) 2018–2019. https://dbei.gov.ie/en/Publications/Publication-files/R-D-Budget-2018-2019.pdf.
  2. RQF website, archived at the Wayback Machine: https://web.archive.org/web/20081015194613/http://www.dest.gov.au/sectors/research_sector/policies_issues_reviews/key_issues/research_quality_framework.
  3. REF website: https://www.ref.ac.uk/.
  4. Jones, A. and Cleere, L. (2014) Furthering the Research Impact of UCD: Report of the Beyond Publications Committee. https://researchrepository.ucd.ie/handle/10197/7292.
  5. Forfás and HEA (2009) Research Strengths in Ireland – a bibliometric study of the public research base. https://dbei.gov.ie/en/Publications/Publication-files/Forf%C3%A1s/Research-strengths-in-Ireland-A-bibliometric-study-of-the-public-research-base.pdf.
  6. Research Prioritisation Steering Group (2011) Report of the Research Prioritisation Steering Group. https://dbei.gov.ie/en/Publications/Publication-files/Research-Prioritisation.pdf.
  7. Government of Ireland (2015) Innovation 2020. https://dbei.gov.ie/en/Publications/Publication-files/Innovation-2020.pdf.
  8. SFI (2013) Agenda 2020. www.sfi.ie/funding/sfi-policies-and-guidance/research-impact/AGENDA-2020.pdf.
  9. SFI Impact Framework. www.sfi.ie/funding/award-management/research-impact/.
  10. Irish Research Council (2019) Strategic Plan 2020–2024http://research.ie/assets/uploads/2020/01/Irish-Research-Council-Strategic-Plan-2020-2024.pdf.
  11. Penfield, T., Baker, M.J., Scoble, R., and Wykes, M.C. (2014) ‘Assessment, evaluations, and definitions of research impact: A review’, Research Evaluation, 23(1), 21–32. DOI: https://doi.org/10.1093/reseval/rvt021
  12. W.K. Kellogg Foundation (2004) Using Logic Models to Bring Together Planning, Evaluation, and Action: Logic Model Development Guide. www.wkkf.org/resource-directory/resources/2004/01/logic-model-development-guide.
  13. Stevens, H., Dean, A., and Wykes, M. (2013) Final Report of the DESCRIBE Project. Exeter University.
  14. UCD Research Portal. www.ucd.ie/research/portal/typesofimpact/.
  15. Jones, A. and Cleere, L. (2014) Furthering the Research Impact of UCD: Report of the Beyond Publications Committeehttps://researchrepository.ucd.ie/handle/10197/7292.
  16. Promote Your Research website. www.ucd.ie/promoteyourresearch/index.html.
  17. Find Researchers at UCD website. https://people.ucd.ie/.
  18. Impact Case Study Competition. www.ucd.ie/research/portal/ucdimpactcasestudycompetition/overview/.
  19. UCD Research Portal. www.ucd.ie/research/portal/outcomesandimpacts/impactplancapturecommunicate/#self.
  20. Research Impact website. www.ucd.ie/research/impact/researchimpact/.
  21. Research Impact Tools. www.ucd.ie/research/portal/impactsupportsresources/.
  22. Analytics and Impact Team. www.ucd.ie/research/portal/meettheteam/#9.
  23. Impact portal homepage. www.ucd.ie/researchimpact.

Copyright © Education Matters ®  | Website Design by Artvaark Design