I had an interesting chat with Björn Brembs this morning evoking the subject of currencies in the public research sector.
The public research sector is notorious for systematically incentivising undesirable behaviours from researchers. The incentives are established through the system’s way of rewarding researchers’ work. It is customary to think of this in terms of currencies.
A currency is a prevalent medium of exchange. With currencies in circulation in academia, one buys academic positions and funding. Having such means to easily acquire academic positions and funding easily is the academic version of being rich.
The richer you are
The more executive power you have
Your incentive as a researcher is to become rich
The problem is simple: in the public research sector, presently, becoming rich is different from doing good research.
Activities you do to get rich
Activities you do to make research progress
There is an overlap. There also are discrepancies.
Today there are several kinds of currencies in circulation in the public research sector:
 |
Number of articles: the more articles you have in your name, the richer you are in academia |
|
Number of citations of each of your articles: the more cited are your papers, the richer you are in academia |
|
Prestige of the journals in which you have published your articles: the more prestigious the journals, the richer you are in academia |
|
Number of co-authors of each of your articles: the less co-authors you have, the more your name stands out, the richer you are in academia |
|
Order of your name in the list of co-authors: the earlier it appears, the richer you are in academia |
All these currencies are based on peer-reviewed articles. If peer-reviewed articles didn’t matter, these currencies wouldn’t make sense the way they do.
I will distinguish between ‘currency’ and ‘currency basis’: at present peer-reviewed articles are the basis for the currencies listed above. I will call that “the old basis“.
I write this post to propose to build a new currency basis. I will call it “the new basis” and introduce some propositions and requirements for it.
Incentivisation of desirable behaviours in science: drop it already!
Björn insisted that if we want to incentivise a behaviour, we might first want to ask ourselves why this behaviour needs to be incentivised in the first place. Is it an unpleasant behaviour? Is it detrimental? If the behaviour was pleasant and self-serving, people would already be motivated and no further incentivisation would be needed. Incentivising new behaviours might not be the smartest thing to do, nor the easiest.
There are things to try before…
I’d add a cautionary “be careful what you wish for“. If you incentivise new behaviour among researchers, and you succeed, then you’ve just changed the profession a little, and possibly the reasons why people enroll in it. I find the idea of gamification (gamification of peer-review, gamification of open access publishing…) ostentatiously naive for this reason. If you change the rules of the game, you might change the name of the game too.
For now, the game is called science. It’s a good game. Many of its rules already make it simultaneously exciting and useful to play. The idea is to empower not to incentivise desirable scientific.
Elizabeth Gadd’s complementary arguments on incentivisation are worth considering as well → How (not) to incentivise open research.
The new basis does not change "the name of the game."
Incentives to do science are not tampered with.
The new basis supports behaviours that researchers already find pleasant and self-serving.
What’s the point in a new basis if it doesn’t change what people want to do? → To change what people can do.
How are we to change anything for the better without generating new behaviours, only promoting old behaviours? → By enhancing the old desirable behaviours, and making them matter more so that the old undesirable behaviours matter less.
By Björn’s remark, we hardly have a choice anyway. Other than temporarily forcing people to do what they don’t want to do and jeopardising the profession in the process of tinkering with it, we are left with encouraging scientists to do what they already want to do.
We all have ideas on what it is scientists want to do. We are looking for something more specific than that however: what scientists want to do that simultaneously serves a bigger picture such as scientific progress and society. Determining what that is, isn’t trivial. Let’s save the task for later. For now let’s complete Proposition 2:
The new basis supports behaviours that researchers already find pleasant and self-serving and that serve the bigger picture of science and society.
The new basis must not make things harder for researchers. It must not add to the workload of researchers.
The new basis is defined in terms of work that researchers already do.
The new basis must be institution-agnostic to allow for currencies to be portable from one institution/department/country to another.
The old basis is already like that. All research institutions in all countries are aware of the actual importance of article publications. The new basis should benefit from similar portability in order to allow researchers to keep moving. Institution-agnosticism is also important to recognise the contributions of non-affiliated researchers as well as researchers who make non-mainstream science with little institutional support.
Another remark Björn made this morning motivates the distinction between “currency basis” and currency . A single currency is not enough. Biologists have different scientific cultures than mathematicians. Scientific value is not assigned the same way in math and in biology.
At first sight, it looks like all sciences do actually agree on one currency, namely “the number of publications”. But there is nothing remotely scientific about this agreement (cf below: paragraph about “wrapping”).
An example of disagreement between scientific fields: In Björn’s field, one paper per year is good. Björn says “more than that, and you’re not thinking far enough, less than that and you’re not pushing yourself enough“. In my field, a paper a year is considered too much to allow for anything else than redundant or vacuous content. The old basis failed to make the rate of publication matter as well.
It must remain possible for different fields to assign value differently. As Björn said, we need multiple currencies.
The new basis must allow for multiple currencies.
The different things valued by different scientific fields can be matched by different currencies.
The example of differing rate of publication suggests that further than that, the old basis must also the same things to be valued differently… This requirement will be covered below.
The new basis must not be exclusively about articles.
The old basis is already almost exclusively about articles. For a reshuffle, the new basis has to be about something different.
Changing or removing some of the current article-based currencies is not enough. This is because those currencies aren’t universal, apart perhaps from the “number of article publications” currency. For instance, in my field, journal prestige hardly counts. Conference papers, which don’t count in other fields, do count in mine. The conferences tend to be rather small and focused on the work of a certain community. This bounds competition to a reasonable scale and scientific scope. It is clear that my field is hugely more comfortable than scientific fields in which journal prestige counts. Comparatively to life science fields, mine is a scientific Eldorado. However in absolute terms, my field is not a scientific Eldorado at all. The “number of publications” currency is enough in itself to ruin the profession. The inflation of this currency changes the name of the game. It promotes the optimisation of research output at the expense of the optimisation of research. And as argued further down, this annoying aspect of the “number of publications” currency is an intrinsic property of the old basis.
So indeed, changing or removing some currencies can make a huge positive difference. But if we pay attention to the different configurations represented by the different research fields of academia, we soon notice that attacking lone currencies will only get us so far and “so far” is insufficient and it’s already getting uncontrollably worse in some fields. It’s not only the worst case scenarios that should inform our solutions. The best case scenarios should inform them to.
Dynamics between the different scientific fields need to be considered. What happens to one field can propagate to other fields. An effect that is obvious in one field can be an underlying effect we underestimate in an other field.
The new basis must not be exclusively about data.
Otherwise the new basis would be a basis only for currencies that matter to some research fields, those that manipulate data. Many research fields don’t manipulate data including mine: ‘data’ is a word I never ever use as part of my research work. If academia starts rewarding researchers based on criteria defined in terms of data, I won’t get rewarded. As discussed elsewhere, if our solutions to academic dysfunctions, exclude some sciences by design, we might as well reconsider our common scientific identity and discuss the option of a schism in our profession.
The new basis must not be exclusively about data and articles.
Otherwise the change would only affect data-oriented sciences. Other sciences would continue operating on the basis of articles.
For now, let’s refuse the option of a schism. We want biologists and mathematicians etc to keep operating under the same roof, governed by the same institutions, to keep collaborating and considering each other as colleagues. Let’s also refuse the abolition of interdisciplinary collaborations. Then, as long as some sciences are still driven by article publications, I wager that all sciences will continue being driven by article publications. The continued interest of some sciences in the old article based currencies, will cause those currencies to continue being substantiated and general interest in them to continue being fueled.
At the start of my career, I decided not to engage in the publication race. The strategy has served me well, but it suffers a hard limit: not caring about article counts myself does not stop my collaborators from worrying about them. There always comes a point when I’m working with someone at the whiteboard and they bring up the subject of publication. With regards to the science we are making and have not yet finished making, I find the subject is usually brought up far too early. This discrepancy in our concerns interferes very concretely in the work we do at the whiteboard. Proving a theorem is not the same thing as preparing for a paper.
The new basis must be universal.
Just like the old basis of peer-reviewed articles, the new basis must be common to all fields of scientific research.
The system of currencies should avoid straining (interdisciplinary) collaborations with conflicting incentives /interests. For any pair of (interdisciplinary) collaborators working together at the whiteboard, this can mean either of two things:
1. There exists a common currency that both researchers can be rewarded in for the work they do together (probably different from the currency rewarding a different pair of collaborating scientists)
2. There exist different currencies to reward each of the two researchers’ contributions, and the currencies are compatible: the currencies recognise different aspects of the same collaboration, or different kinds of input to the same output.
The new basis must be science-agnostic to allow for currencies recognised in all scientific fields.
The new basis must allow for a continuum of compatible currencies.
Different scientific fields call for different competences in scientists. Further, different scientists have different qualities. Some are good at generalistic thinking, some are good at formulating questions etc. Further, one scientist may change what she is good at from one period of her life to the next. Perhaps even, she is be better at formulating questions before lunch and better at seeing the big picture while digesting her lunch. Arguably, a good currency is one that helps make the most of scientists by finely recognising and rewarding scientists’ different qualities/roles.
If none of those roles and qualities transcend disciplinary fields, a schism is probably in order.
I don’t think this is the case. If there were no role/quality/sort of input a scientist could contribute, that would be recognisable outside of her field of expertise, then interdisciplinary collaboration would be impossible. No mathematician could ever really collaborate with a biologist. It would be impossible for them to make a line of reasoning progress collaboratively, as neither would recognise what the other brings to the table. The relation would be purely transactional: one kind of scientist placing an order for the other kind of scientist to deliver (“Dear mathematician, please produce a theoretical model for my data“, “Dear biologist, please produce a justification of my theoretical work“). I have experienced enough true (/cybernetic) interdisciplinary collaboration to have no worries about that. It is possible for a computer scientist, biologically unsavvy, to formulate profound questions that will encourage a biologist to change their perspective on the biological reality. And a biologist can formulate fundamental questions that can inspire brand new avenues of thought for the computer scientist.
For any true scientific collaboration involving 2 fields of expertise, there should be currencies reflecting what each party in the collaboration contributes. Further than that. in order to substantiate a common scientific identity and justify the use of a shared basis, there should be universal currencies that work for all scientific disciplines not just pairwise (requirement 6). They don’t have to be the preferred currency of all fields. They don’t even have to be the preferred currency of any field. But they must exist to materialise interdisciplinary appreciation and to support a common scientific identity.
This relies on the variety of research fields sharing at least some commonality. Defining a new currency basis is an opportunity to highlight a meaningful commonality. Something more meaningful than the fact that we all write papers. Something more true than the fact that we all output data. It’s the whole point of a currency basis.
Currencies and their importance can vary from field to field, but the basis must be common.
Commonalities & Opportunities for relay
The previous requirements suggest we should take interest in:
- Commonalities shared by all sciences — eg all sciences make intensive use of questions; in all sciences, there are circumstances in which acknowledgment for the formulation of a good question is more fitting than acknowledgement for the publication of a good paper.
- Opportunities for sciences and scientists to relay each other — eg a biologist might give meaning to a formal system by turning it into a model of some biological system, thereby constructively framing the study of the model, while a computer scientist might frame the good use of the model by expliciting the formal system’s limited expression power.
What is a currency?
The dictionary definition works fine in this context. Let’s just add the following precision to narrow down to currencies of the public research sector:
Here are some more examples, to extend the list of article-based currencies above:
|
Number of collaborators |
|
Number of questions raised that have opened a line of inquiry |
|
Number of bridges built across otherwise independent research fields |
|
Number of indirect contributions to praised publications |
|
Amount of constructive feedback provided to peers |
|
Depth of scientific proofs constructed |
|
Distance of contributions to hot topics like the SDGs |
|
Degree of precision in the definition of the context of application of a model |
This says nothing about how those quantities could be measured.
This also says nothing about what is a good currency. Not all currencies are good, even when they represent something we want to capitalize on. We used to think more papers was necessarily a good thing until we realised there was a bug in our logic. As producing more science translated into producing more papers, we thought that conversely, producing more papers translated into producing more science. It turns out papers can be produced independently of science.
According, to Elizabeth Gadd, ‘measures of openness’ are another example of currencies that aren’t as good as they first seemed to be. And openness is thus another example of flawed currency basis.
Let’s leave for later the important discussions of how to define good currencies and how to measure them. For now we still have a new basis to find.
Björn highlighted the risk of inflation of currencies. This is the situation we have today with journal prestige and number of articles. Both these currencies have come to matter so much to researchers, that the profession has changed as a result of it. Research is now a profession dedicated to the writing of articles, preferably in the most prestigious journals.
Björn and I each came up with a solution to the problem of inflation:
| B: |
Multiply currencies so there are so many of them in circulation that traditional article-based currencies lose traction and no other takes over |
| M: |
Use a basis that is not in itself obviously countable (unlike articles which are easily countable) |
These must be two formulations of the same idea:
B → M : As long as we have a countable basis (like articles), we have a privileged currency (like the number of articles), and a risk of inflation.
M → B : A new basis that is not obviously countable must allow for multiple currencies for reasons detailed below.
Articles are easy to count. Further, articles are such that it is easy to say lots of things about them (how many of them there are, where they are published, who published them, how many pages they have…) without ever saying anything remotely scientific. Articles wrap up scientific content and make it perfectly natural to talk more about the wrapping than about the content. They are such that you either talk about the wrapping or you talk about the content but can’t talk about both at once. They make you choose between your position in the economy of public research and science. In that sense, they are a poor currency basis, bound to promote and entertain the notoriously bad situation academic research is in now.
The new basis mustn't wrap content in such a generic way that we can talk about the wrapping without consideration of the content. Content must be wrapped up in scientifically meaningful/useful containers.
This doesn’t say what scientifically meaningful/useful containers are. Just like the question of good currencies, the question of good containers is important and non-trivial. Let’s leave it for later. For now, let’s continue exploring the question of a good basis.
The following requirements follow from Proposition 3.
The new basis must not make the traditional way of publishing harder than it is. Ideally, it should facilitate it. At the very least, the new basis should be compatible with the old basis (coexist without competition).
The new basis must provide an optional bonus in addition to the old basis.
Putting together previous requirements for the new basis, especially:
- that it needs to relate to work researchers already do — work that is neither data nor article related —
- and that the new basis must not find itself competing with the old… :
The new basis must involve the daily scientific production of researchers, especially the neglected bits that don't end up in traditional publications.
What those neglected bits are has to be determined. For now let’s just say that we will be looking for bits:
- that researchers are happy to produce and already are producing
- that are useful and serve the big picture
- that we can build a new basis out of that satisfies the requirement of universality and science-agnosticism.
This is a demanding task. Let’s leave it for later.
Now, a new series of requirements which relate to the point made earlier about empowering existing desirable behaviours (rather than incentivising new ones) — and to propositions 1 and 2, and Corollary 1.
The new basis must not bother thinkers in their thinking, researchers in their research. The new basis must not interfere negatively with *scientific* customs, habits and practices in place in each field of scientific research.
Science is not broken. Science is something different from the environment in which science is made, the governance and administration of science, the incentivisation of science… Those things being broken don’t make science broken. Science has been successfully practiced for ages. Those who know how to practice it are highly specialised scientists. They might not know how to fix the other broken things. But they are the ones who know how to make science work.
Practicing science and managing science are two different activities. To know what interferes with a scientist’s progress, requires to know what the scientist is working on, to understand the approach she is taking and the scientific reasons why she is taking this particular approach. In other terms, it requires being the scientist herself or being scientifically very close to her.
To know how to not bother scientist thinkers in their work requires consulting with them. This is not a trivial task but like those mentioned above, it is an essential one.
The system is now designed to optimise the production of article publications. It is not designed to optimise the advancement of good science. Dysfunctions and undesirable behaviours are bound to happen. We should not let this situation make us blind to good science-making abilities where they can be found. Who is to blame is a different question. Irrespective of how we choose to answer this question, we should take care to not disregard scientific competences even when they are held by blamable individuals.
The new basis must be mess-friendly.
Mess is an integral part of science-making. Forbidding scientists from being messy — eg by forbidding them to say wrong or irrelevant things — is interfering with the process of science-making.
If this corollary requirement worries you, it might reassure you to know further down figures a requirement about quality control.
Just like the old basis, the new basis must be very flexible.
Science is characterised by its continual self-updating.
Rigid standards risk inhibiting precious changes of perspectives and jeopardizing scientific breakthroughs.
Flexibility is needed to ensure the new basis accommodates highly specialised scientific cultures.
Checking the new basis is flexible enough will require testing before validation.
The new basis must not get in the way of science evolving continually. Ideally, it must facilitate the continual evolution of science. It must mitigate the risk of today's science getting in the way of tomorrow's science.
The new basis must allow for what we believe to be relevant and true today, to not come in the way of brand new perspectives in the future. Ideally, it must be designed to facilitate relevant changes of perspectives while simultaneously supporting scientific explorations according to actual perspectives.
→ This relates to the concept of “information in the dark” i.e. information that is overshadowed by information we already have.
In relation to Requirement 12, Requirement 14 also concerns scientific approaches. They too evolve. And as they do, scientific fields evolve: how we define and delineate scientific fields. In defining a new basis we need to take care not to rigidify the system by unnecessarily crystallising in it some current views that could change.
The new basis must be an extension of the old basis. It must not exclude data- and article-based currencies. It must be compatible with all currencies in use.
This is needed to avoid forcing researchers to chose at their own expenses between capitalising on new currencies (what we want researchers to want) and capitalising on old ones (what researchers actually want). If there were a competition between currencies, the old ones would likely win because they already are currencies i.e. prevalent. A competition would risk making the lives of researchers harder. Researchers would be forced to choose between giving up on academic wealth and mild gilt.
Utilisation of anything new should not rely on researchers renouncing anything old they presently capitalise on.
Since the new basis must extend the old, the new basis must involve some form of publishing just like the old basis:
The new basis must involve some form of publishing.
Open Science has been operating as if it were obvious that a better future for public research entailed changes to the scholarly publication system. I have never been convinced of this.
As I already mentioned, data publication doesn’t concern me. As for articles, I spend a tiny amount of my time at work thinking about article publications. I can easily go half a year without even thinking of my next publication. I also don’t read articles that often.
In between the moments were my work brings me to read a paper or think of writing one, my work has nothing to do with papers. There is a huge amount of communication that happens between me and my peers. It just doesn’t happen through articles on a daily basis. Communication in my work only marginally overlaps publication.
Our understanding of the term publication has drifted. It could mean “offering a piece of information to the public and offering to take responsability for it as an author“. But instead it rather means “claiming a form of ownership over a piece of information, in public“.
Before I can offer the public my responsability over a piece of information, I must test that I am able to take responsability for it facing peers who speak my scientific language. What I produce is meant to be both of quality and useful. My peers, working with me at the whiteboard, are the key partners in ensuring the quality of my work. I don’t necessarily have to communicate with them through an article. And often, an article wouldn’t be appropriate. Also, my work can only be used by people who understand the highly specialised language in which it is expressed. Writing an article is a way to reach these people but there other ways, some more direct.
The new basis must have scientific value in itself. Since it extends traditional publishing, it must also extend traditional peer-review correspondingly. The new basis must add to scientific quality control.
The new basis must involve some form of scientific quality control that adds to traditional article-based peer-review.
The old basis has gotten us used to thinking official quality control of scientific content only happens through the peer-review process at the submission of an article. We need to ensure that the new basis comes with an integrated quality control system equivalent to what peer-review is for the old basis.
This is more trivial than we think once we stop thinking in terms of the old basis and acknowledge that in reality, scientists spend their days doing quality control. Quality control is too big a part of science for it to wait for article submission.
Quality control is part of the “neglected bits” mentioned earlier in Corollary 2: the elements of science that don’t, per se, add to academic wealth as long as wealth is counted with the old currencies. Thus adding the task of ensuring quality control on the new basis has not added to the challenge introduced earlier which is to identify the “neglected bits“.
Conclusion:
The starting point of the discussion was that despite some overlap, there are discrepancies between academic enrichment and academic research. There are two approaches to address the notorious consequences of this. I propose the second:
| 1. | Try to deincentivise enrichment (very hard given that enrichment = acquiring the means to do research) |
| 2. | Mitigate the discrepancies towards: |
The 2nd approach requires work and thought. — cf the icon in the right margin above. This icon corresponds to questions which now require our dedicated and methodical attention. To take care of those question appropriately, we need a plan. I have a something to propose as a starting point (topic of a next post).
Recap:
The new basis does not change the name of the game. It must ensure the profession of scientific research is indeed about scientific research.
PROPOSITION NO 2:
The new basis supports behaviours that researchers already find pleasant and self-serving and that serve the bigger picture of science and society.
PROPOSITION NO 3:
The new basis must not make things harder for researchers. It must not add to the workload of researchers.
COROLLARY NO 1:
The new basis is defined in terms of work that researchers already do.
The new basis must be institution-agnostic to allow for currencies to be portable from one institution/department/country to another.
REQUIREMENT NO 2:
The new basis must allow for multiple currencies.
REQUIREMENT NO 3:
The new basis must not be exclusively about articles.
REQUIREMENT NO 4:
The new basis must not be exclusively about data.
REQUIREMENT NO 5:
The new basis must not be exclusively about data and articles.
REQUIREMENT NO 6:
The new basis must be universal.
REQUIREMENT NO 7:
The new basis must be science-agnostic to allow for currencies that are recognised in all scientific fields.
REQUIREMENT NO 8:
The new basis must allow for a continuum of compatible currencies.
REQUIREMENT NO 9:
The new basis mustn't wrap content in such a generic way that we can talk about the wrapping without consideration of the content. Content must be wrapped up in scientifically meaningful/useful containers.
The new basis must not make the traditional way of publishing harder than it is. Ideally, it should facilitate it. At the very least, the new basis should be compatible with the old basis (coexist without competition).
REQUIREMENT NO 11:
The new basis must provide an optional bonus in addition to the old basis.
The new basis must involve the daily scientific production of researchers, especially the neglected bits that don't end up in traditional publications.
The new basis must not bother thinkers in their thinking, researchers in their research. The new basis must not interfere negatively with *scientific* customs, habits and practices in place in each field of scientific research.
COROLLARY NO 3:
The new basis must be mess-friendly.
REQUIREMENT NO 13:
Just like the old basis, the new basis must be very flexible.
REQUIREMENT NO 14:
The new basis must not get in the way of science evolving continually. Ideally, it must facilitate the continual evolution of science. It must mitigate the risk of today's science getting in the way of tomorrow's science.
The new basis must be an extension of the old basis. It must not exclude data- and article-based currencies. It must be compatible with all currencies in use.
REQUIREMENT NO 16:
The new basis must involve some form of publishing.
REQUIREMENT NO 17:
The new basis must have scientific value so it must involve some form of scientific quality control.
One Reply to “Currencies in academia”