French Research Performance in Context

The CURIF (now UDICE) commissioned to SIRIS Academic the report French Research Performance in Context (2019) to analyze some worrying findings regarding the decline in the international competitiveness of the French research system.

‍

This report aims at contributing to the ongoing policy reflection on the French research system. It is structured in two main parts, each of which seeks to answer a simple question:

• Is the performance of the French research system satisfactory?
• If not, why is this not the case?

The full report is available here. What follows is an executive summary presenting the main analyses and findings.

Part 1. Contextualising French research performance

The first part provides a broad overview, comparing French research output across a wide set of indicators with that of a set of ten benchmark countries: Australia, China, Denmark, Germany, Japan, Netherlands, Spain, Switzerland, UK and US.

Global performance indicators

The first set of data compares benchmark countries in terms of (a) share of production and citations and (b) research focused university rankings. It enables us to identify three types of countries: high performing, emerging and low performing. 

Within continental Europe, the data clearly distinguishes:

• Denmark, the Netherlands and Switzerland, which are not only highly performing but are continuing to improve their performance;
• Spain, which has low performances but is catching up;
• France and Germany, whose performance is both low and declining.

The performance of France and Germany is worse in the case of more selective indicators (top 50% is better than top 10%; ranking in top 500 is better than ranking in top 100) and their decline is true not only when compared to emerging countries such as Spain but also when compared to high-performing countries.

Globally, the overall picture is similar:

• countries such as Australia (to which one could add Canada or Singapore) are performing similarly to leading European countries such as Denmark;
• a large and increasing number of countries are on a similar trend to Spain; amongst these, China is a case apart and should now be considered a global powerhouse with a research potential, which will soon be comparable to the US;
• Japan is performing even worse than France and Germany, with truly dramatic drops in global share of both production and citations;

Finally, although the US and the UK still dominate on size-dependent criteria such as total production and total citations, their overall performance is declining and they are below high-performing countries according to most size-independent criteria.

Excellence indicators 

The second set of data looks in greater detail at the performance of benchmark countries on excellence indicators. To do this, we have selected three groups of indicators:

• general bibliometric data with field-weighted citation impact, citations in PP Top1% and PP Top 10% and fine-grained performance across 251 fields;
• performance in cutting-edge fields such as biotechnology or fast evolving technological topics
• Indicators of individual performance such as highly-cited researchers and ERC awards

Despite the use of very specific indicators, the global results are perfectly aligned with those from the first set of data and confirm the three groups of countries, which we defined above. 

‍

Denmark, the Netherlands and Switzerland perform very well on all indicators (with a few exceptions for Denmark). The UK and US perform better on more selective indicators. Interestingly, China performs better in cutting-edge fields than in more traditional ones.

The French performance is particularly weak in cutting-edge fields, at an institutional level (rather than a country level) and on very selective indicators. The same is true of Germany but to a lesser extent.

‍

Comparing results in Horizon 2020

Our third set of data compares country performance in Horizon 2020.

In this case, France and Germany both stand out with cumulative losses of well over 1 billion Euros, whereas a country such as the Netherlands has gained close to 1 Billion Euros from H2020 over the same time period.

As in the case of other indicators, despite the fact that the results of France and Germany are already very poor, their market share is continuing to decline both when compared to the previous framework programme and on a yearly basis. On the contrary, countries such as Denmark, with excellent results, continue to improve.

‍

Reframing the problem

The “European Paradox” assumes that Europe’s research is strong, while its innovation is weak. It has been at the heart of European Research policy since 1995 and continues to guide framework programmes, as we move towards Horizon Europe.

This European paradox is clearly a myth. Europe does not have a comparative advantage in producing knowledge. It has difficulties turning knowledge into innovation and growth precisely because the proportion of truly world-class research it produces is low and its expertise in cutting-edge fields is poor.

However, the problem is not that of a “transatlantic gap” between Europe and the US, but one of intra-european divergence. And the main issue within Europe is not North versus South or East versus West but the weakness of Franco-German research performance.

Part 2. Why is the French research system underperforming?

The second part of this report analyses five families of hypothesis which could help explain why the French research system seems to be performing less well than it could:

• funding of the research system;
• connection to the global research system;
• structure of the research system;
• human resource model;
• autonomy, accountability and governance.

Each of these families of explanation is hypothetical and within each family of explanation, each specific explanation is also hypothetical. 

However, together they provide a coherent narrative which will hopefully make it possible to define a set of policy recommendations that will help make the French research system truly competitive. 

Hypothesis 1: Funding of the research system

Hypothesis 1 derives from a simple fact: without money there could be no research. This leads to two questions. 

1. Does the French research system have enough money?

The answer to this is clearly “no”. Two key arguments support this perspective:

• French investment in Research and Development (GERD) is below the OECD average, just above the EU average and clearly lower than key competing countries.
• Even more significantly, France is losing ground compared with nearly all our benchmark countries. Indeed, investment in research is increasing rapidly not only in emerging research countries like China or Spain, but also in both low performing research countries such as Germany and Japan and high performing ones such as Denmark and the Netherlands.

This trend has been clear for nearly twenty years and urgently needs to be addressed. 

2. Is the money well distributed?

This said, the quantity of money is not the only important parameter. Indeed, the way money is distributed is at least as important as the amount.

High performing countries share a number of characteristics:

• they concentrate research funding on research intensive universities;
• they have dual funding mechanisms for universities, which differentiate teaching and research funding and are usually combined with some degree of Performance-Based Research Funding;
• they privilege competitive funding mechanisms over block grants.

High performing national research systems thus have “vertically-segmented” systems that clearly distinguish between the missions that different kinds of higher education and research institutions are supposed to fulfill and allocate funding on this basis.

Unlike these countries, French universities are constrained by a budget allocation model that:

• Does not distinguish a Research stream from a Teaching stream;
• Depends far less on Performance indicators;
• Allocates only limited amounts on a competitive project-based basis.

In a world in which universities are the key hubs that ensure global visibility, this has major consequences on the performance of the French research system as a whole. 

Hypothesis 2: Connection to the global research system

For reasons outlined in the introduction, the scientific system essentially acts as a huge machine for internally filtering out interesting from less interesting endeavours. As a result, being performant in research supposes not only to be intrinsically good but also to be connected to the rest of the network in an efficient way: interesting science needs to be noticed in order to become effectively relevant. This is why being connected is so important.

Hypothesis 2 examines three key questions that measure the degree of integration of the French research system within the global research system:

1. Language and history

Science has always been global, but until the 1970s major national research systems continued to be reference points in terms of prestige. The emergence of a single global research system was comparatively easy for countries that had always looked abroad (Denmark, the Netherlands, Switzerland) and for countries whose system became the core of the global system (UK, US) but it was very hard for countries, which used to be major references in their own right such as France, Germany or Japan.

The difficulties that this implies continue to have an impact on performance (for example, attracting leading researchers implies switching to English not only for publishing, but also as a working language in the lab and for teaching).

2. Connections

Co-publication data clearly shows that French researchers are increasingly working with international colleagues. However, these co-publication networks are not correlated with excellence - the strongest scientific affinities of France are with Belgium, Italy and Spain, three neighbouring countries, two of which have relatively low research performances. Furthermore, France has the lowest co-publication rate of all our benchmark countries with China, the key emerging research powerhouse.

And, reinforcing the warning signs, the share of French researchers who have had short-terms stays in foreign institutions is surprisingly low compared to other countries. 

3. Hubs

However, to truly understand the impact of connections, it is necessary to look not just at raw data but at the networks behind these connections. These underline two facts:

• first and foremost, the research potential and visibility of hubs is key because it is there that new trends emerge, cutting-edge research is produced and connections at a global level are forged;
• secondly, the rest of the country needs to be well connected with these global hubs to enable global research results to be assimilated at a local level.

This is why most countries strive to reinforce a few key hubs rather than to promote the countries research potential as a whole. 

‍

Hypothesis 3: Structure of the French research system

Hypothesis 3 addresses the structure of national research system. To do this, we distinguish three main features:

• the presence or absence of large national research organisations ;
• the degree of integration of universities and national research organisations;
• the degree of institutional differentiation among research intensive universities and institutions with a more local and/or educational focus.

The French national research system can be characterised as (a) weakly segmented with a low degree of institutional differentiation and (b) hybrid, with large national research organisations partly integrated within large research and teaching universities.

We then analyse the impact of this structure on the performance of the French research system in two steps:

1. The impact of vertical segmentation on performance

The first part of the discussion is closely linked to hypothesis 2 and shows that weakly segmented systems in which research in general and top research in particular is being performed by a greater fraction of institutions, tend to perform less well because they do not benefit from the knock-on effect of strong hubs. 

2. The impact of strong national research organisations on performance

The second part of the discussion looks at the impact of strong national research organisations on the overall visibility and performance of a country when the global research system is structured around research intensive universities. It notably argues that:

• In a global system there is a premium for systems based on a “simpler” architecture. As a result, national research organisations have a measurable negative impact on:

• rankings and visibility of universities
• strategic decisions taken by potential international partners
• student choices

• National Research Organisations perform well in purely quantitative terms but not as well in qualitative terms. They are behind the world’s leading universities in per researcher performance and, in some cases, behind research-intensive French universities on indicators such as citations per paper.
• The mixed research unit system is structurally costly because it implies large transaction costs and inhibits strong strategic profiling and planning.
• Just as problematic, the two-track recruitment system differentiates the long-term mission and status of individual researchers for no good reason and on no strong basis.
• Finally, the size and geographic dispersion of large umbrella type national research organisations does not favour the emergence of cutting-edge fields.

Hypothesis 3 ends with a short description of some recent evolutions of national research systems, which could be interesting to consider in a French context.

Hypothesis 4: Human Resource model

At the end of the day, research performance depends on individuals: researchers publish articles, which are cited, they are awarded ERCs and become Highly Cited Researchers. Attracting talented researchers is thus key, which is why our fourth hypothesis explores the main features that makes a research system attractive.

France currently experiences a brain drain towards high performing countries, a balanced brain circulation with Germany, and a brain inflow from lower performing countries. This brain circulation is not only linked to working conditions but also to research performance: studies show that a researcher with an ERC will obtain better results if she chooses to move to a high performing country. 

Brain circulation has a major impact because research performance depends primarily on talent: fostering internal development of excellence is far harder and less efficient. This is why attracting talent is so important for national research systems. 

Studies show that the key factors that researchers are attracted first and foremost to:

• “outstanding faculty, colleagues or research team”;
• and “excellence/prestige of the institution”.

Better research infrastructure and access to research funds are important, but less so. The same is true of better salaries, quality of life and working conditions. 

In other words, leading researchers are attracted to vertically segmented research systems and perform better within them, which reinforces hypothesis 3. 

To attract the best researchers, high performing countries have launched specific funding programmes, or “talent schemes”. These aim to support the emergence of national lighthouses by providing long term research autonomy to successful applicants. 

However, the French research system is unable to compete with these countries because universities do not have the necessary autonomy and power to define their Human Resource policy. 

Close to half of the academic staff working in research intensive universities are still employed by national research organisations who define their own Human Resource rules. The university has no power to define their workload (balance between teaching and research) or incentives to foster strategic objectives. 

Within the university itself, staff promotion and hiring decisions depend on national agencies such as the CNU, that have strict national rules, which dramatically limit each university’s autonomy. Universities cannot even freely modulate the time an employee spends on research, academic and administrative duties. 

‍

Hypothesis 5: Governance model

All four previous hypotheses underline the fact that high performing countries tend to have segmented research systems in which a relatively low number of research intensive universities play a key role. 

This has deep consequences in terms of governance because it implies that the leadership of these institutions is able to define and implement an ambitious global strategy. And this, in turn, rests on a governance model, which is both autonomous and accountable.

Today, despite the important reforms of the last decades, French institutions are in the paradoxical situation of being accountable without having real autonomy. 

The 2017 EUA scoreboard on university autonomy shows that France still lags behind the rest of Europe on all indicators: financial autonomy, organisational autonomy, staffing autonomy and academic autonomy. Indeed, globally, France ranks last of all 29 research systems analysed. 

This said, the solution is not simply to increase autonomy on each indicator: autonomy cannot be isolated from governance as a whole. Indeed, for autonomy to be meaningful entails a minima 3 requirements:

• Autonomy must not be merely legalistic, but effective: this is not (only) about statutes, but about the factual political system of university governance.
• Governance must have true authority over all domains of activities.
• Accountability must ensure alignment with general sectoral policy objectives and foster professionalisation.

Reforms about governance and accountability are thus necessarily interrelated. There is no magic formula which demonstrates that the election or nomination of leaders is necessarily better, nor is there one, which defines the ideal proportion of external members in the governing bodies. However, there is a logical relation between (a) how resources come to an institution, (b) how much power the leadership has in terms of decision-making, and (c) how this leadership is appointed. 

This relationship helps explain why there has been a shift over the last 20 years from governance models built on internal elections to those built on appointment mechanisms or, in other words, from governance models in which the leadership is accountable primarily to the academic body, which elects them to governance models in which the leadership is accountable to the public bodies which finance them.

‍

Epilogue: getting to Denmark

Looking back through this report, one factor stands out: France does not trust its research intensive universities. 

Throughout the world, research Intensive Universities are the key hubs of the research system. They define research strategy, they host the best students, they concentrate leading researchers, they have dedicated funding mechanisms. 

France, on the contrary, combines a highly stratified higher education system in which the most prestigious institutions are not the main research centres with a weakly stratified research system in which the key research actors are not universities. This leads to a paradoxical result: France ends up with both the social stratification of elitist education systems, and the disappointing research performance of low performing research systems.

In a famous paper, Land Pritchett and Michael Woolcock quipped that the problem of getting to strong, reliable, transparent public institutions could be summed up as the problem of “getting to Denmark” (Fukuyama 2011). This image is particularly apt for the French research system: Danish, Dutch and Swiss systems of higher education are good examples of how to balance the competitive, and intrinsically elitist game of “world-class” research with the demand to provide a higher education and research system which promotes openness, inclusiveness and comprehensive social well-being. The fact that they increasingly outperform the UK and US on size-independent research indicators demonstrates that the anglo-saxon model is not necessarily the most performing one.

It is time to accept that the model already exists, time to rethink the role of national research organisations, time to end the distinction between grandes écoles and universities. It is time to differentiate universities, to create excellent university colleges and polytechnics, to reinforce research intensive universities. It is time, in other words, to go to Denmark.