Objectives

MERCI is a formative evaluation study exploring intended and non-intended effects of the European Research Council’s (ERC) “Starting Grants” program. In order to fulfil this aim, the MERCI project considers not only the successful applicants, but also the rejected ones. In doing so, we attempt to gather information on how the prestigious grant might influence for example the status and the working conditions of postdocs.
The overall objective of our study is to generate strategically relevant information which will facilitate the ERC in adjusting its activities in the context of fulfilling its tasks and attaining its goals. Hence, we do not only evaluate the program’s objectives, but to question the objectives themselves, respectively to assess if the applicants’ needs are effectively addressed.

Research design

A special feature of the project is the combination of different methods. MERCI goes beyond conventional multi-method approaches; it not only tests congruence and divergence of data from qualitative and quantitative method lines but intermediate results from individual methods are used to inform each other. The special gain of the approach comprises not only a parallel implementation of qualitative methods (semi-structured interviews, literature study), quantitative methods (online surveys in a panel design) and bibliometric analysis but also the functional interlacing of partial results, instrument development and interpretation of data (triangulation).

Semi-structured Interviews  
An exploratory pilot study was conducted in summer 2009, based on ten semi-structured interviews with Starting Grantees of the 2007 cohort. Data gathered from these interviews served as a first empirical input for addressing the research questions and also contributed to the identification of additional problems which were not detected in the pre-research period and have to be taken up and addressed in the following steps of the analysis. Results of these interviews were used for constructing the questionnaire of the online survey (first wave) and for compiling the guideline for semi-structured interviews of the qualitative main study. For this main study 40 applicants (accepted as well as rejected ones) of the 2009 call are interviewed in spring/summer 2011. For the purposes of triangulation, the results from these interviews will be used for constructing the standardized questionnaire for the 2nd wave of the online survey.

Standardized online survey
The investigation of the “Starting Grants´” impact on approved applicants requires a broad gathering of individual and aggregate data and their change over time. Thus, data on applicant demographics, as well as data on grant endowment, the appointment and relationship of the grantee with their host institution, and the influence of the grant on the young scientists’ career will be collected. For this purpose, two waves of standardized online interviews are conducted (panel design): The first wave will be conducted during the funding period (resp. for rejected applicants: one year after the application), the second wave at the end of the funding period. Furthermore, short intermediate surveys aiming to describe the project status will be conducted. Our first online survey (1st wave) was carried out between October 2010 and November 2010 among approved and rejected Starting Grants applicants of the 2009-cohort. We were constrained by the ERC to invite only those applicants who accepted the transfer of their personal data. Among those applicants, the response rate was high: 71% of the approved applicants and 61% of the rejected applicants participated in the survey. In autumn 2011, the second online survey (1st wave) will be conducted with approved and rejected Starting Grants applicants of the 2010-cohort.

Bibliometric analysis
To evaluate the ERC selection process and the scientific impact of the “Starting Grants”, a publication and citation analysis of funded and rejected applicants of the 2007 and 2009 StG-call will be carried out. The investigation of publications and citations has started in spring 2011 and is conducted jointly by the iFQ and the IWT Bielefeld. The analysis of data will be carried out mainly by the iFQ.

Cooperation

MERCI is carried out in cooperation with the Humboldt-University of Berlin (HU Berlin), the University of Dortmund (TU Dortmund) and the Institute for Science and Technology Research at the University of Bielefeld (IWT). The administrative supervision of the project partners and the cooperation with the European Commission is task of the HU Berlin. The iFQ is responsible for the project coordination, the report preparation as well as for the conceptualization and analysis of the online surveys and for bringing together and editing the results of the various cooperation partners. Prof. Michael Meuser (TU Dortmund) is responsible for the conceptualization and implementation of the qualitative study (semi-structured interviews). The bibliometric analysis will be conducted both by Dr. Winterhager (IWT Bielefeld) and the iFQ. Moreover, there is cooperation with PREST Manchester, coordinator of the EURECIA consortium, which carries out another evaluation study of the Starting Grants Program (“Understanding and Assessing the Impact and Outcomes of the ERC Funding Schemes“, EURECIA).

International Advisory Panel

MERCI is accompanied by an international group of experts (International Advisory Panel, IAP). The primary aim of this panel is to assure intercultural validity of the MERCI sub-studies and to bring in expertise about different national research systems. The first meeting of the IAP took place in Berlin in September 2009. The central question discussed was the attractiveness of the ERC funding for scientists against the background of the characteristics of the various national research systems and framework conditions for young researchers in Europe. In July/August 2010 the members of the IAP were asked to comment on the prepared country reports and the online questionnaire for the first wave of the online survey. The next meeting is planned for 2012.

MERCI in the context of iFQ-projects

MERCI is part of the iFQ subject area "young scientists and academic careers" and is based on the evaluation of the Emmy Noether program. With the aid of MERCI, the iFQ expertise in terms of “program evaluation”, “triangulation of methods” and “panel surveys” is to be developed further. Furthermore, the project significantly expands the data base for processing central topics of the institute. This refers in particular to the situation of young researchers in the postdoctoral period, the decision process in research funding, measurement of research services (qualitative and quantitative indicators), governance of science, etc.

Locating MERCI in the scientific debate

Long-term studies of instruments for postdoc funding are rare. In the course of further development of the ERA, the demand for (young) scientists will increase. It is expected that in this process the role and function of postdocs will change (Armbruster 2008). Postdocs are increasingly assume the role of Principal Investigator and they publish more and more independently.
The interest in information about the performance of young scientists funding programs in terms of their career effectiveness and their ability to actually promote top research will increase. In this respect, our project will not only deal with fundamental issues of the promotion of young talents but will generate input to an important science policy discussion. In particular, it helps to improve the empirical base in this area.

References

Armbruster, Chris, 2008:

The Rise of the Post-Doc as Principal Investigator? Policy Futures in Education, Vol. 6, No.4.