I have been playing tennis and publishing papers long enough that I have found something in common.
So here is what happened during the tennis tournament my son played over the weekend. (For background information, my son has been playing competitive tennis since the summer of 2020, and these days he plays every day, plays friendly matches every week, and plays in tournaments every month.) The opponent wins the first set 6-4. My son wins the second set 7-6. The game goes into a 10 point tiebreak. My son hits a service ace. The opponent is discouraged. But then the opponent’s mom sends hand signs to her son claiming that the ball was out. After a long pause the opponent says the serve was out. Both kids are upset, because my son got cheated and the opponent was forced to cheat by his mom. The game moves on, the opponent wins the point and also the tiebreak at 11-9. (But at one point my son was leading 9-8, so without being cheated he would have won 10-7.)
The problem with low level competitive tennis is that we self-judge. We are supposed to be honest, but when the match is close, there is a temptation to cheat and there is nothing we can do against cheaters. (This is not the first time I witness cheating. Unfortunately there are many kids who do, and they tend to be boys around age 10, but there are also adults of both sexes.)
Cheating in tennis is not such a big deal because if you are a professional tennis player, there will be officials and hawkeye, and if you are a recreational tennis player, the outcome of a match does not have any material impact on your life and you can cut off cheaters from your friends list.
Unfortunately, the problem of self-judge also arises when we try to publish our research. In many fields, including economics, whether to publish a paper or not is determined by peer review. Referees (who are experts in the field) recommend to the editor whether the paper is worth publishing or not. Of course, referees are supposed to be honest and neutral, but because they are anonymous, there is lots of room for corruption. For instance, suppose researcher A submits a paper to a journal and gets anonymously reviewed by researcher B. Since both A and B are experts, there is a good chance that A and B are competitors, and B might recommend rejection by coming up with whatever reasons. And publishing in top journals is so competitive that editors have no choice but to reject the paper if a referee complains about whatever.
Here are typical examples of complaints by the referees, all of which happened and continues to happen to me when I send a paper to top 5 economics journals.
- If you prove a theorem, the referee will say the paper is theoretical and unsuitable for a general audience.
- If you prove a theorem, the referee will say it is obvious.
- If you prove a theorem, the referee will say it is well known without evidence.
- If you don’t prove a theorem, the referee will say the paper lacks a theorem.
- If you don’t illustrate the theory with a numerical/empirical application, the referee will say the paper is purely theoretical and unsuitable for a general audience.
- If you illustrate the theory with an application, the referee will say the application is stylized or unrealistic.
- If you present a more serious application, the referee will say the application is not serious enough.
And so on. Whatever you do, there is no way to escape from arbitrary criticisms.
The peer review system can also be exploited to your advantage if the pool of potential referees is small, for instance when the number of high profile people in a particular field is relatively small. Let’s say there are two high profile researchers A and B. They may explicitly or implicitly agree to rubber stamp each other’s papers at top journals. Because this is a repeated game, A and B need not be friends to sustain cooperation. (This repeated game is identical to the repeated prisoner’s dilemma; (reject, reject) is the unique equilibrium of the stage game, but the Pareto efficient outcome (accept, accept) is sustainable if players are sufficiently patient.) Obviously I have no direct evidence that this is what is happening, but the fact that people go to conferences and socialize is an anecdotal evidence.