Illustration Valerie Geelen The new yardstick by which researchers and teachers are to be assessed is nothing short of a revolution, according to its creators. Resource spoke to two of the architects, Arnold Bregt (Education Dean) and Theo Jetten (Secretary of the PE&RC graduate school/WUR library).
The ink on the new Academic Career Framework is barely dry. And no, they don’t want to give themselves a grade. The WUR Council has yet to give its verdict on the paper. And the detailed elaboration of the evaluation in terms of precise guidelines is not yet finished. Nevertheless, Education Dean Arnold Bregt ventures to say that he is proud of the result so far. ‘The Recognition and Rewards Committee has been working hard over the past few years. We started back in 2020, just before the first lockdown. We gathered a lot of information from the organization and held discussions on all fronts. In the process, it once again became obvious how much this change was needed. I think we have managed to deliver a proposal that does justice to the culture we have, and yet starts off a shift towards broader recognition and rewarding of our academics.’
The new yardstick looks at performance in research, teaching, social impact and academic services. Does everyone have to shine in all these areas?
Bregt: ‘Not necessarily, but you are expected to make conscious choices. In your personal profile, you have to state what your vision and ambition are in those four areas. But how much time you devote to the various components is tailored to the individual, in consultation with the manager, the possibilities within the group and your personal interests.’
So are you also expected to have social impact?
Jetten: ‘You are expected to at least reflect on what your social impact is, how society benefits from your research, and to make that explicit. That you have a strategy on this matter.’
Bregt: ‘Ideally, as an organization, we want you to consider all these areas important. WUR believes in science for impact. If you’re not into impact, all you’ve got left is science. But there is some flexibility about the extent of that impact. There could be a trade-off: do a little less in one area and compensate with a little more in another.’
Jetten: ‘For example, you can dedicate a substantial amount of your time to socially relevant activities. That is a choice you can make, within the context of your group. Currently, the evaluation of research output is still strongly focused on publications in journals and books. But other kinds of activities could take their place.’
The focus of the evaluation process is shifting from quantity to quality. Will the points system for publications be called into question?
Jetten: ‘Yes. The current scoring system with a classification of journals based on impact factor is stringent and rigid. Most of that is going to go. The quality of the article will come first, rather than where you publish it. And the judgement on quality will be more in the hands of the evaluation committee. But of course it still matters where you publish your article or database, because it does have a certain status in your world. And the research portfolio must be of a certain size. We are trying to translate the current system into a measure of research output volume in the new system. And that output is allowed to be more diverse than it is now. It won’t be just articles that count, for instance, but also databases, book chapters and designs that meet specific quality criteria.’
The quality of the article will come first, rather than where you publish it
There will be no quantification for social impact. So how will that be evaluated?
Jetten: ‘We try to categorize it, depending on the career path you are on. That means for instance: as an assistant professor, this is roughly want we expect from you in terms of activities and products; as an associate professor, roughly this… And so on. The same goes for academic services. To put it simply, your responsibilities increase over the course of your career, and with them, the breadth of activities you’re involved in. You start out in your field as a team player, making a contribution, and you end up as a leader. It is up to the evaluation committee to assess that, based on supporting evidence provided by you.’
Bregt: ‘An example. As Dean of Education, I am on the evaluation committee for personal professors. One of the criteria for appointment is whether you are a leading light in your domain. If that domain has social impact, you are expected to play a leading role in the public debate and to help steer that debate – being asked to appear on the talk show OP1, for example. We are going to include a description of those expectations in the guidelines, but it is up to the individual to demonstrate with supporting evidence what level you have reached.’
One of the sore points in the current system is the large numbers of PhD students that associate professors and personal professors have to supervise. Will that number go down?
Jetten: ‘In the current system, supervising PhD students is a measure of earning capacity and of skill as a supervisor. Those two things will be separated out. PhDs are a money-maker and your line of research has to be funded. But that revenue can be obtained in more ways than through the PhD bonus. Competence as a supervisor will become a separate indicator in the evaluation.’
Bregt: ‘And yes, the lower limit will be lower, thereby removing the financial incentive to attract a lot of PhDs. You need to have some, of course, but not so much for financial reasons to prove your earning capacity.’
Won’t the greater focus on quality rather than demonstrable quantity make the evaluation process more subjective?
Bregt: ‘My first answer is: yes. But we think we can cover that by clearly defining what you have to provide in terms of supporting evidence.’
Jetten: ‘We try to guarantee an objective assessment by drawing up guidelines for evaluation, so you are not at the mercy of the whims of a particular committee.’
Bregt: ‘And attention will also be paid to the composition and training of evaluation committees. The same members will sit on several committees. This is already the case in the evaluation at the final step to becoming a personal professor. There are a lot of permanent members of that committee, and in my experience, its evaluations are very fair and consistent. This is because it includes many experienced evaluators who do it for all the science groups.’
The new system promises more flexibility within and between career paths. But that path has to tie in with the plans of the chair holder. Doesn’t that curtail your freedom considerably?
Bregt: ‘It might do. Each chair group has a strategic personnel plan and new developments have to fit into that. Still, if you’re on tenure track now and it doesn’t suit you, you’re stuck too, actually. This plan offers the possibility of switching to teaching. It will be easier to do that from now on. And that applies to all the paths. As things stand, quite a lot of people are not on a tenure track and they are not very happy about it. There are no evaluation moments and few development opportunities. In this system, all that will soon be in place.’
This plan facilitates switching between career paths
Jetten: ‘The new system also pays attention to postdocs, i.e. young researchers with a temporary (four-year) employment contract. We shall also try to give them supervision and advice. Postdocs are now an undervalued group.’
Bregt: ‘And often these are young researchers with a lot of potential who have just completed a PhD. If you can offer them prospects, that’s worth a lot to them and to the organization.’
Job done, then. What is the biggest challenge now?
Bregt: ‘That is working out the plan to arrive at precise evaluation guidelines. The devil is always in the details.’
Jetten: ‘This framework is part of a major culture change and that sort of thing always takes years.’
Evaluation, new-style
A mandatory evaluation will take place every five years. It can be done earlier if there is good reason to do so. The evaluation relates to promotion or is advisory in nature. Depending on what rung of the ladder you’re on, a formal committee may be called for or a lighter form is chosen, with the chair holder involved. The personal profile is central to the evaluation. In it, the ‘candidate’ describes their current situation, plans and wishes regarding their achievements in research, teaching, social impact and academic service. To structure this, 27 indicators have been proposed, many of which are already used to assess teachers and researchers. But many are also new. For instance, for researchers now things will count such as awards, the use of databases created by others or the editorship of scientific journals. A completely new feature is the evaluation of social impact: blogs, columns, lectures, interviews, and so on. Scoring on academic services can be done by looking at things like being active in committees or working groups (including internal ones) or as a peer reviewer. Not all the indicators are of equal importance to everyone. And as a general rule, the higher up the ladder you go, the higher the expectations of you.
Time
How much time are you expected or allowed to spend on social impact, for example? That is not stated, but there are limits. Teachers must devote at least 70 per cent of their time to teaching. The rest is available for other things. Researchers must spend at least 70 per cent of their time on research. On the professorial path, at least 70 per cent of the time must be spent on research and teaching, both roughly equally. Exceptions are possible, especially for part-timers. So everyone creates their own profile.
