Executive Summary
Ireland’s innovation productivity is declining, slipping from 6th to 11th place among EU Member States between 2016 and 2021. Metascience, the “science of science,” is being adopted by leading research agencies worldwide to maximise return on innovation investment. Taighde Éireann-Research Ireland (TERI), Ireland’s new state research agency, has a unique opportunity to implement metascience from its inception. Failure to adopt metascience now risks Ireland falling further behind in global innovation rankings and missing out on economic benefits.
TERI should:
- Establish a dedicated metascience unit
- Leverage US spending and expertise through international collaborations
- Build a national research database for evidence-based decision-making
- Future-proof against AI disruptions in research funding
The foundation of TERI is a special opportunity to boost innovation productivity
Around the world, the old strategies for funding science are being discarded. A new system, metascience, informs fresh models that maximise the return on innovation investment. The core idea is to “turn the scientific method on itself”. In practice, metascience generates data to inform evidence-based decisions. Just as researchers experiment in the lab, metascience involves studying science itself with trials and measurements.
Metascience replaces a funding system that largely lacked rigour and data. Little is known about how to get the most breakthroughs, start-ups, and highly-skilled scientists per euro funding. In the words of John Ioannidis, Prof. of Statistics at Stanford, “It is a scandal that billions of dollars are spent on research without knowing the best way to distribute that money.” In 2021, the Chair of UK Research & Innovation (UKRI), lamented that too much in science and research policy “tends to turn on the gut feel of the individuals involved.” According to Dr Heidi Williams, Prof. of Economics at Dartmouth University, “we know relatively little about what [innovation] investment strategies are most effective.”
Taighde Éireann-Research Ireland (TERI) is the new state research agency replacing Science Foundation Ireland (SFI) and the Irish Research Council (IRC). It is mandated “to promote the attainment and maintenance of excellence in the standard and quality of research and innovation”. Minister of Further & Higher Education, Research, Innovation, and Science (FHERIS), Patrick O’Donovan recently emphasised this mission. Excellent research and innovation is “vital to achieving multiple national policy objectives, including … climate action targets and the digital transition”, he said.
Without metascience, TERI risks flying blind on “gut feel”. Metascience is the best tool available to ensure research achieves national policy objectives. Irrespective of the specific target, metascience can optimise the process of science to achieve the desired outcome. Other national research agencies are using metascience to find the best strategies for them. They are driving regional development, increasing diversity in STEM, and accelerating the discovery of next-generation technology.
Establishing TERI is a once-in-a-generation opportunity to set Irish science on a new trajectory. By employing metascience from the start, TERI can deliver more for the same money. That means more scientific discoveries and more start-ups. It means a more competitive domestic economy, cleaner and cheaper energy, and better healthcare. This would be like finding huge additional investment for science, every year.
Failure to adopt metascience now will mean falling further behind. International funders are investing millions in metascience, and metascience requirements will likely feature in the next wave of European Innovation Council research support. If TERI doesn’t improve its science process, it’ll be further out of step with best-in-class institutions, and Ireland will slide down the innovation rankings.
TERI is a blank slate. Its establishment is a once-in-a-generation opportunity to drive a step-change in innovation. Here’s how.
Progress Ireland recommends that TERI:
- Establish a metascience unit to drive productivity gains in Irish science
- Leverage US spending and expertise to drive domestic innovation
- Build a research database for smarter, evidence-based decision-making
- Future-proof against AI disruptions in research funding
Why Innovation Productivity Matters to Ireland
Science shapes the daily lives of the Irish people. While scientific inquiry is essential philosophically, its practical impacts are considerable.
Research activity trains people in vital skills. Those who study science learn to think clearly and logically. Hands-on experimentation equips future Irish workers with the methods and mentality to solve complex technical problems. These skills are helpful in many jobs, from running firms to shaping policy.
This enriched labour pool drives job growth. In late 2023, 100% of the American Chamber of Commerce’s members agreed that “certainty regarding the availability of highly skilled talent is important to maintaining FDI employment in Ireland.” These companies bring work and wealth to Irish towns and cities. The IDA Annual Report 2023 states that “access to high-quality talent continues to be the top consideration for investment location decisions”. This influx of jobs lifts living standards across the country.
Research experience gives our skilled workforce the adaptability to engage in new types of work. The next generation of foreign direct investment will feature cutting-edge technologies and industrial methods. As industry and the corporate environment evolve in an AI era, workers must fall back on their capacity to adapt. Students develop expertise in assimilating and applying knowledge during undergraduate and postgraduate research. This expertise will be evermore vital as the job market shifts.
Irish science also helps us use global discoveries. When breakthroughs happen abroad, we need local experts to apply them here. Our science base lets us tap into world knowledge and use it to better Irish lives. With local experts, Ireland keeps pace with global innovations. This is known as “absorptive capacity” — the ability to recognise the value of new information, assimilate it, and apply it.
Domestic discoveries fuel the Irish economy and improve the standard of living. Our labs and universities breed new ideas that turn into valuable products. Take the Irish firm Wayflyer. Incubated at NexusUCD, it reached a value of €1.6 billion in 2022. Or researchers at RCSI working to commercialise a new way to fight sepsis. Their company, Inthelia, was backed by SFI, IRC, and Enterprise Ireland. With sepsis treatment in Ireland costing over €200m a year, Inthelia’s success could save lives and provide value to the exchequer.
Irish science tackles local issues, too. Our researchers work on problems that matter here. The Rooster, Ireland’s most popular potato variety, was first bred by Teagasc in 1990. Today, FarmZeroC in Cork is working to develop “climate neutral” processes for roll-out across the country. Some Irish problems will only be solved by Irish research and development.
Despite its importance to the Irish economy, Irish innovation is underperforming. A recent EU Commission report highlighted Ireland’s “unexpectedly poor” R&D investment as a “key factor holding back productivity.” The return on innovation investment is limited by many factors. These include delays in research funding, poor diversity, and a weak commercialisation pipeline.
Ireland is no longer one of the ten most innovative EU Member States. It slipped from sixth place in 2016 to eleventh place in 2021. Over the same period, Ireland’s position in the Global Innovation Index fell from seventh to nineteenth out of 132 countries.
The government has ambitious plans to ramp up spending. Impact 2030, the national strategy, outlines the Government's commitment to increase innovation investment to at least 2.5% of the domestic economy before the end of 2030. In 2023, the then-Minister of FHERIS, Simon Harris, called for “sufficient priority” to be given to R&D funding “to ensure Ireland remains a strong innovation leader”.
Increased funding is not enough to make Ireland an innovation leader. The strategies outlined in Impact 2030 can help us catch up with other countries. But, supercharged with metascience, Ireland could leapfrog international competition with the same investment.
The world's premier research bodies are implementing metascience
Research funders around the world are retrofitting their institutions to maximise impact. Most have established new teams dedicated to metascience problems. They tinker with existing funding structures, infrastructure, and databases. The insight gained is used to develop better strategies for funding science.
The UK Department of Science, Innovation, and Technology (DSIT) has listed metascience as a key area of research interest. An initial £10 million has been committed to a new metascience unit. Headed by Dr Ben Steyn, it will conduct live controlled experiments on the funding system. Dr Steyn is focused on “growing the academic community for metascience with grant calls, experimenting with partial lotteries, and developing a new set of metrics to measure research novelty”.
In late 2023, the NSF announced a new “science of science” partnership with the Institute for Progress (IFP). The partnership will design and conduct experiments to fine-tune innovation funding. NSF’s new Technology, Innovation and Partnerships Directorate (TIPs) leads the project. They are supported by the Federation of American Scientists and its Metascience Working Group. Together they’ll analyse NSF data to evaluate how efficiently the agency funds and supports research. The group has an ambitious target to conduct shadow experiments, retrospective analysis, and pilot studies.
According to the TIPs website, pilots involve defining a hypothesis, testing it, and assessing outcomes. Positive results lead to further investment and scaling.
The TIPs Directorate has recently piloted programmes:
- To help researchers launch a startup to commercialise technology.
- To maximise the chances of success for startups by making available novel curricula and support methodologies.
- To support the increased involvement and success of entrepreneurs from historically underrepresented groups.
A binding US Senate report has urged the National Institute of Health to examine how best to create or empower another metascience team. It would engage in NIH-wide experimentation with new ideas regarding peer review and funding models. The aim is to enhance NIH's operations and ultimately improve biomedical progress. As a start, the NIH has issued a public call for proposals to “strengthen the evidence base” for NIH decision-making.
Starting in 2018, the Swiss National Science Foundation (SNSF) experimented with drawing of lots to make funding decisions. In rare cases where funding applications receive the same score, research suggests that subsequent differentiation was affected by subconscious bias. To avoid this, SNSF used a lottery procedure. This was a piloted phase for the Postdoc Mobility career funding scheme. Following evaluation, the National Research Council decided to allow evaluators to draw lots if necessary in all of its funding schemes. As of 2021, randomisation decided all tiebreaker cases. Prof. Matthias Egger, President of the Research Council, said "in such cases drawing lots is the fairest solution because it is blind and rules out bias."
The results from this metascience experiment have travelled. Germany's largest private funder, the Volkswagen Foundation, and Austria’s public research funder have adopted randomised decisions. In 2022, the British Academy introduced a similar lottery to decide between equally scored funding applications. Other recent work includes a collaboration between the Innovation Growth Lab’ and Innovate UK. Together, they are testing the impact of innovation vouchers. Further collaboration with the European Commission is developing citizen-engagement experiments.
A pan-Europe initiative, AFIRE, brings together future-minded research agencies to design and implement funding experiments. These partners create “a structured pathway … to initiate, expand, and report [on] experimental interventions in priority-setting, grantmaking, review, and evaluation.” Switzerland, Austria, Luxembourg, and Norway have already joined.
Implementing Metascience at TERI
The foundation of a new research agency marks a crossroads for Irish science. TERI cannot keep a funding playbook that other countries are ripping up. Instead, TERI should adopt metascience and continuous improvement as a core mandate. According to the Innovation Growth Lab, incorporating the continuous improvement of metascience into an organisation's mandate from the beginning is the best means of reaping the benefits. Here are three recommendations to do just that:
Strategy #1: Establish a metascience unit to drive productivity gains in Irish science
Ireland needs the adsorptive capacity to benefit from metascience efforts around the world. Metascience is complex. Local expertise is required to quickly implement metascience strategies developed abroad. Doing so will put us at the innovation forefront. TERI needs a dedicated team to capture this value.
TERI should establish a dedicated unit that works on metascience full-time. SFI and the IRC currently employ capable project officers with expertise in grant design and assessment. Project officers in SFI’s strategy team could be given a new metascience mandate to work full-time to increase the return on innovation investment.
The cost of a new metascience unit would be minimal, particularly in comparison to the expected ROI. Going by metascience units in other agencies, one manager and four team members should be sufficient. Based on average SFI salaries, this unit would cost less than €300k a year. For argument’s sake, say that this team improves the effectiveness and efficiency of Irish science by a very modest 1 per cent. We spend roughly €1 billion a year on R&D. If that went 1 per cent further, it’d be like we had an extra €10 million for science every year. If each euro of science generates €70 in social value (as some studies suggest), this is like an extra €700m in value per year. All from an initial €300 thousand investment.
The first priority of the metascience unit should be to perform retrospective analysis of Ireland’s funding portfolios. Some of this work has already been conducted by the SFI Strategy Team. Expanded analysis should aim to, first, create a “snapshot” of SFI and IRC funding outcomes. The creation of TERI is a major change in the Irish research landscape. A snapshot of projects funded by SFI and IRC is a crucial point of comparison for assessing the impact of the amalgamation. Second, the analysis should inform the first phase of development for TERI by answering the question “What has been the highest impact and most successful funding projects to date?”.
The metascience unit should then re-run experiments from other countries. This would check that the results are valid for the Irish context. For example, the NIH is piloting a grant to reduce the admin burden on top scientists. The Maximising Investigators' Research Award (MIRA R35) aims to “increase the efficiency of … funding by providing investigators with greater stability and flexibility.” It hopes to enhance scientific productivity and the chances for important breakthroughs. A “MIRA for Early Stage Investigators” ensures the model benefits young, ambitious researchers. The TERI metascience team could copy the fundamentals of this pilot and try to replicate the positive results in Ireland.
The metascience unit should iterate on these findings and do something novel. New pilots could be based on concrete theoretical findings. For example, an experiment based an influential grant design paper could investigate different methods of paying out funds from the exact grant mechanism. After the grant is awarded, scientists would be randomly allocated into one of two funding delivery streams. Outcomes could be compared following a pre-specified period.
Involving just two dozen researchers in a pilot each year can yield significant data over time. By the end of a five-year experiment, over 100 data points can be collected, offering robust insights. One of the considerable advantages of conducting these pilots is gaining a deeper understanding of the entire research system, enabling more effective and targeted improvements.
The unit could draw heavily on The Experimental Research Funder’s Handbook (2nd Ed.) The Handbook synthesises insights from funders within the Research on Research Institute consortium. It collates information on trials with new approaches to funding review, allocation and evaluation. It details practical measures to design and implement experiments with funding processes. The handbook was supported by the Swiss National Science Foundation, Novo Nordisk Foundation, European Molecular Biology Organisation (EMBO) and by the Wellcome Trust.
Strategy #2: Leverage US spending and expertise to drive domestic innovation
The TIPs Directorate has initiated high-level collaboration talks with SFI. The US agency is ready to commit $12 million to a joint venture. The Irish government should match this funding and use the collaboration to enhance the impact of Irish funding and science.
Some of the partnerships that would boost Ireland’s metascience already exist in nascent form. The US-Ireland Research and Development Partnership, for example, is a unique initiative involving funding agencies in the United States of America, the Republic of Ireland, and Northern Ireland, focused on increasing collaborative R&D amongst researchers and industry. TERI should expand on SFI’s historic involvement in the Partnership by establishing a formal agreement with the TIPs Directorate of the NSF on metascience. Having already engaged in joint research funding, TERI should coordinate with NSF and UKRI to perform analysis on funding data, determine what has been successful, and pilot new schemes. Staff would up-skill on metascience in the process of gaining valuable insights from frontier organisations.
In particular, TERI should collaborate on specific NSF programs that address weaknesses in our innovation ecosystem:
- The Metascience Working Group (MSWG) is a forum and coordination platform for academics and policy practitioners to contribute and draw on key insights that could inform the design and evaluation of science funding programs. TERI could use this expertise in designing its own pilots.
- The NSF Innovation Corps (I-Corps™) program is an immersive, entrepreneurial training program that facilitates the transformation of invention into impact. The seven-week experiential training program prepares scientists and engineers to extend their focus beyond the university laboratory — accelerating the economic and societal benefits of basic research projects that are ready to move toward commercialization. SFI has already facilitated participation in this programme by some Irish researchers, but the collaboration should be expanded to bring commercialisation training to Ireland.
- The NSF Regional Innovation Engines (NSF Engines) program creates regional-scale, technology-driven, inclusive innovation ecosystems by accelerating key technologies, driving economic growth, creating and retaining quality jobs, expanding equitable pathways into careers, and strengthening national competitiveness and security. Ireland is the gateway to Europe for 950 US companies directly employing 209,000 people and a further 167,000 people indirectly. TERI should work with Amcham and DFHERIS to adapt this place-based innovation funding initiative to drive a step-change in Ireland’s competitiveness.
TERI should also formally collaborate with the Accelerator For Innovation & Research Funding Experimentation (AFIRE).
Strategy #3: Build a research database for smarter, evidence-based decision-making
If TERI is to be successful, high-quality data is needed to drive decision-making. A national R&D database would allow a comprehensive analysis of Ireland’s innovation performance for the first time. This would be like removing the blindfold from the agency’s new management.
TERI should establish and maintain a National Research Information Management System for all publicly funded research. The Barcelona Declaration on Open Research Information should inform the construction of such a database.
US Senator Cassidy recently said that one of our most significant “untapped resources” is granular data on funding applications. Whether accepted or rejected, the outcomes of funded projects provide valuable insight. An NRIMS should be established that includes the award type, duration, research question, institution, and rich detail on the academic and funding history. Tracking peer reviewers and funding scores is also necessary. The dataset should include information on grant applications that were rejected for funding.
Analysis of this data should play a vital role in the distribution of resources and the evaluation of researchers. TERI should use this information to set strategic priorities. An NRIMS that encompasses the full set of outcomes of Government Research funded projects would allow a holistic analysis of the Irish innovation ecosystem. For the first time, in-house retrospective analysis would have the data necessary to find waste in the funding ecosystem, identify redundancy, catalogue successes, and measure the impact of GBARD.
The Research and Development Budget 2022-2023 showed that the government’s investment in R&D surpassed €1 billion for the first time. DFHERIS was responsible for the largest share of total expenditure at 53 per cent, followed by DETE (24%), the DAFM (10%), Health (6%) and DECC (3%). Because almost half of Irish R&D will fall outside of TERI ’s direct remit, analysis of their grants alone won’t tell the whole story. The NRIMS should be managed by TERI but include data on all government-funded research.
Collating data across different government departments can be difficult without a unified research taxonomy. A neat solution might be to consider “R&D funding” whatever the government counts towards GBARD.
Data access is an unavoidably thorny issue. In other countries, external requests for research data can take years to navigate bureaucratic complexities. UKRI developed a new mechanism to provide top analysts access to data without compromising their legal responsibilities. Following an initial application, researchers were invited into UKRI for workshops to co-create a set of funded projects. The outcomes and required data access were agreed upon in advance, side-stepping the data-sharing issue with oven-ready deals. Implementing such a mechanism in TERI would be straightforward, particularly with expert guidance from UK DSIT.
A sophisticated NRIMS would facilitate collaboration with the UK DSIT Metascience Unit to jointly analyse and assess data. Ireland and the UK have broadly similar research structures and significant interaction between scientists, universities, and research agencies. Analysing Irish research funding data in the context of UKRI’s data would yield powerful insights into what strategies work at scale. Particular attention could be given to Northern Ireland, which receives funding from both UKRI and SFI.
The NRIMS should be informed by international best practices in research data. Signatories of the Barcelona Declaration on Open Research Information, including the Austrian, French, and Dutch national research agencies, have committed to
- make openness the default for the research information we use and produce,
- work with services and systems that support and enable open research information,
- support the sustainability of infrastructures for open research information, and
- support collective action to accelerate the transition to openness of research information.
Committing TERI to these principles at its inception, and allowing them to guide all activities, would catapult Ireland to the frontier of research excellence.
Strategy #4: Future-proof against AI disruptions in research funding.
AI is already widely (but not officially) used by applicants, reviewers, and agencies to generate and assess proposals. The result is that research proposals are written and read by an AI tool, completely undermining the peer review process.
The TERI metascience team could trial new ways of distributing research funding or adopt successful foreign solutions. For example, the US NSF is piloting an internal, protected, air-gapped database of research proposals, published articles, and patents. Applicants and reviewers will have access to AI tools in the database that operate on the closed system. Using AI, funders will be able to quickly find researchers who are working in a particular area and further sort by other criteria such as history of commercialisation. Tools such as this database may reduce the current overreliance on funding calls where applicants have to spend time and energy submitting a proposal. Instead, scouts could use AI to find the relevant researchers.
The Research on Research Institute (RoRI) has convened European funding agencies to collectively address this issue. RoRI’s annual conference will feature an in-depth discussion on the use of AI in research. Currently, no Irish Government or TERI representatives are registered to attend. The Government should prioritise meaningful engagement with the EU funding community as it solves these problems. Delegates should be sent to these conferences. To sit out these conversations risks Ireland becoming a negative outlier.
Closing Argument
According to the National Competitiveness and Productivity Council, “innovation is a fundamental driver of economic progress”. It is an important determinant of international competitiveness”. Innovation is crucial to maintaining current FDI, attracting the next generation of FDI, and developing the domestic economy with start-ups.
Research, funded by the new agency, plays an enormous role in generating this innovation. Despite its obvious value, it is underfunded and underperforming. Around the world, elite research agencies have implemented metascience to maximise innovation ROI. TERI’s incorporation presents an opportunity to do the same.
The first step is to build a dedicated “science of science” team. Enthusiastic international partners and a clear road map will ensure results. Armed with this new methodology, we can ensure Ireland remains a strong innovation leader.
