The origins of the DFG Expert Review System extend back to Fritz Haber, who won the Nobel Prize for Chemistry in 1918 and was decisively involved in establishing the predecessor to the German Research Foundation (Deutscher Forschungsgemeinschaft – DFG) in 1920: the Notgemeinschaft der Wissenschaft. The re-establishment of the DFG in 1951 largely adopted this system, whose structure then remained practically unchanged until 2003. The following year saw the system fundamentally reformed with the introduction of a new type of decision-making body: the Review Board (Fachkollegium).
1. The DFG Expert Review System prior to 2004
The DFG Expert Review System operated up to 2004 involved two groups of reviewers: Expert Reviewers (Fachgutachter) and Special Reviewers (Sondergutachter).
The 650 expert reviewers were elected for a period of four years by the academics and researchers working at universities and non-university research institutions. The DFG Statutes state that all research proposals should be reviewed by experts elected from the respective subject. Expert reviewers were responsible for reviewing the funding proposal. They presented substantiated recommendations on the award of funding to project proposals received by the DFG. If the expert reviewers additionally required specialist expertise or if the workload caused by the reviews was too great, it was possible, in exceptions, to consult so-called special reviewers. These are researchers with particular expertise in a special subject area (cf. DFG 2003: 73f.). Special reviewers are partly chosen by DFG Head Office and partly directly by the relevant expert reviewer(s). However, the assignment of special reviewers was neither governed nor approved by the Statutes of the DFG.
As the number of proposals continued to grow, so too did the number of special reviewers. When the pre-2004 system was adopted from the predecessor organisation in the early 1950s, the average annual workload was less than 10,000 proposals being vetted by 650 reviewers. Today, ten times as many are reviewed. The workload for the expert reviewers increased accordingly, while their number remained unchanged. This was additionally aggravated by the ever greater specialisation of science, which made it impossible for some expert reviewers to give a verdict on a proposal without consulting a specialist from the field in question. Consequently, the review of research proposals by special reviewers was no longer an exception, but rather the rule. Under the Individual Grants Programme, almost half the reviews were carried out by special reviewers in 1999. At just under 80%, the proportion of reviews carried out by special reviewers in the Coordinated Programmes was even higher (cf. Koch 2006: 23).
The introduction of so-called Priority Programmes in 1953 also saw the principle of reviews discussed on site introduced. This principle is now a fundamental part of every coordinated programme that has since been added. The participation of elected expert reviewers in these on-site sessions was not made compulsory, which is why only one third of all the coordinated programmes in 1999 were also reviewed without the participation of elected expert reviewers.
From 1999 to 2001, a total of just under 9,000 special reviewers supported the work of just under 1,000 expert reviewers in the two election periods 1996 to 1999 and 2000 to 2003 (cf. DFG 2003: 73f.).
2. The DFG Expert Review System after 2004
By reforming the DFG Expert Review System in 2004, the DFG responded to the development in the number of proposals and to the report by an international expert commission which had presented the results of a system evaluation of the DFG and the Max Planck Society in 1999. The central objective of these reforms was to "secure the efficiency and the practicability of the system in a meanwhile greatly changed research landscape." (Koch 2006: 25*). The specialist review of proposals, comparative assessment for funding recommendations and the decision on whether to approve funding were to be functionally separated.
The Review Board (Fachkollegium) was established as a new kind of decision-making body to ensure this. It took the place of the former Review Committees (Fachausschüsse). The members of the Review Boards, the so-called "Fachkollegiaten" are – like the former expert reviewers – elected by the scientific community for a period of four years. There are 48 such review boards that are each made up of one or more subjects. They are divided into 201 subjects in total, which, depending on the subject in question, are made up of 5 to 12 members. The central role for the 577 review board members in total lies in assuring the quality of reviews and in preparing funding decisions.
The number of review board members is based on the number and volume of proposals received from a subject area and on the degree of subject differentiation (DFG 2006a: 40). The greatest number of review board members (223) come from the field of Life Sciences. According to the DFG subject categories, this includes the Biology, Medicine, Agricultural and Forest Sciences. Medicine accounts for the largest group here, with 121 representatives. Since the Review Board on "Medicine" is a special case on account of its size, it already divided itself into four sections at its constitutive meeting.
The second largest subject area is formed by the Humanities and Social Sciences. With 132 members, this area accounts for 22.9% of all review board members. This is followed by the Natural Sciences (Chemistry, Physics, Mathematics, Geosciences) with 19.6% and Engineering Sciences (Mechanical Engineering and Production Engineering, Thermal Engineering and Process Engineering, Materials Engineering, Electrical Engineering, Computer Science and Systems Engineering, Construction and Architecture) as the smallest group with 18.9% (cf. Fig. 1).
HFG = Helmholtz Gesellschaft
FhG = Fraunhofer-Gesellschaft
MPG = Max-Planck-Gesellschaft
WGL = Wissenschaftsgemeinschaft
Gottfried Wilhelm Leibniz
Source: DFG 2006a
Just under 90% of the review board members work at universities. Of the non-university research institutions, the Max Planck Society, with 24 researchers, has the largest number of review board members, followed by the Helmholtz Association and the Leibniz Association (cf. Fig. 2).
The review board members are responsible for assessing the review process and results of each proposal and for issuing a decision proposal to the DFG Joint Committee, which makes the final decision.
While in the past the elected members of the review committees were directly responsible for the review, the assessment is now carried out outside the review boards. This structure aims to achieve a clear separation between the review and the evaluation of this review. This means that a new level of research evaluation and quality assurance has been introduced.
The role of DFG Head Office is now explicitly governed by the Statutes as well: "The Review Boards shall determine their own sets of internal rules and procedures, which shall require approval by the Senate." (Article 8, Clause 4; DFG 2002). Its task in this review process is to select reviewers, to obtain the written review report, and to produce a decision proposal for the review board. When selecting reviewers, priority is on subject/discipline suitability and on the reviewer's workload. Furthermore, any possibility of bias or conflicts of interest through cooperation or competition, teacher/student relationship, reciprocal reviews, etc., must be avoided.
Source: DFG 2006a
Today's reviewers correspond in function with the former special reviewers. In the period from 2002 to 2004, 65,556 paper-based reviews were carried out by 10,883 reviewers on 24,419 proposals. Each proposal corresponds to a mean of 2.7 review reports. Of these, the proportion of paper-based reviews carried out by reviewers working abroad amounted to 13% in the period under report, of which the greatest share of reviews came from Austria and Switzerland. These two countries account for 52% of all active DFG reviewers working abroad. Third place is taken by the United States (12%) ahead of Britain (10%), the Netherlands (8%) and France (5%).
Of the domestic reviewers, the largest proportion (87% of all reviewers) work at a university. Reviewers working at non-university institutions are mainly found at institutes of the Max Planck Society (MPG) (4%), the Helmholtz Association (HGF) (3%), and the Leibniz Association (WGL) (2%) (cf. DFG 2006a: 31).
The next round of elections for members of the review boards will be held in 2007. This will also mark the point at which the reformed system will once again be examined in detail, so that any modifications that might be necessary can be carried out. How the system is assessed by the players themselves was already discussed at a conference held in autumn 2005. This marked the first meeting of all the chairs of the review boards and enabled them to exchange ideas and opinions and to take stock of their experience, both among themselves and with representatives of the DFG. In the summer of 2006, the Institut für Forschungsinformation und Qualitätssicherung (iFQ) also carried out a survey among the review board members. On the one hand, the survey aimed to collect information on the experience gained with the newly-designed review system and, on the other, to collect ideas on possible changes to the review systems. A first experience report is expected in March 2007.
References
Deutsche Forschungsgemeinschaft (DFG), 1951: Satzung der Deutschen Forschungsgemeinschaft. Bonn, überarb. 2002. Online [Retrieved 02.07.2014]
Deutsche Forschungsgemeinschaft (DFG), 2003: Förder-Ranking 2003. Institutionen – Regionen – Netzwerke. DFG-Bewilligungen und weitere Basisdaten öffentlich geförderter Forschung. Bonn. [Retrieved 31.10.2006]
Deutsche Forschungsgemeinschaft (DFG), 2006a: Förder-Ranking 2006. Institutionen – Regionen – Netzwerke. DFG-Bewilligungen und weitere Basisdaten öffentlich geförderter Forschung. Bonn. [Retrieved 31.10.2006]
Deutsche Forschungsgemeinschaft (DFG), 2006b: Jahresbericht 2005. Aufgaben und Ergebnisse. Bonn. [Retrieved 31.10.2006]
Koch, Stefan, 2006: Die Deutsche Forschungsgemeinschaft und die Reform ihres Begutachtssystems. Zur Einführung der Fachkollegien. Wissenschaftsrecht, 39. Band, Heft 1, März 2006.
* „die Leistungsfähigkeit und die Durchführung des Verfahrens in einer inzwischen stark veränderten Forschungslandschaft zu sichern.“
