Alternative maths reviews

Here’s another idea for a wiki-style website, one that might bring closer the day when mathematicians ceased to bother about print journals. It’s a site where people can post reviews of mathematical papers. Such a site, if it did what I have in mind, would have one disadvantage and two advantages over Math Reviews. The disadvantage, which is also one of the advantages actually, is that by no means every paper would be included. If you want a list of all published papers in mathematics, then Math Reviews (or Zentralblatt) does the job very well. However, it’s not really a site where one would browse for fun, and part of the reason is that all papers are given equal status, so if one is looking for an interesting paper one has to look amongst a whole lot of uninteresting ones. With a bit of skill and prior knowledge one can find interesting things of course, but that’s not really what I’d call browsing, in the sense of just having a look at what’s there and finding all sorts of gems.

But the main point of the new site would be to be a forum for telling people why papers were interesting. It would of course include things like the papers of Wiles and Taylor/Wiles on Fermat’s last theorem, but the most useful entries would be on papers that were not world famous in that way. Rather, if you wanted to contribute to the site, you would choose a paper in your area that you particularly like and write a little essay about where it fits into the area, what the ideas inside it are that so appeal to you, why its results are useful (if they are), and so on. If the paper doesn’t have a nice long introduction, you would give it one—much longer than would ever be published in a journal. Ideally, such a review should be written in a way that a new entrant to the area in question could understand: the imagined audience would be a beginning graduate student who had not necessarily taken advanced courses in the area.

To avoid the site filling up with junk, one obvious ground rule is that people should not write reviews of their own papers: part of the idea is that if a paper was reviewed then it would be an indication that it was of genuine interest to other people. Perhaps another rule might be that papers were not included until they had been around for a little while (at least in preprint form). But perhaps that’s a bad idea, as it could be quite useful to know why very recent papers are significant. Another decision would be whether to have a rating system, either for the reviews or for the papers. For instance, it could be quite useful for the reader to know in advance that the reviewer (or rather, the average reviewer—I imagine people making edits to reviews) considers the paper to be a gorgeous result that doesn’t actually open up new avenues of research, or an innocuous-looking lemma that turns out to deal with a difficulty that occurs all over the place, or a set of definitions that makes an entire area of mathematics easier. This could be contained in the text, but perhaps also in some kind of grading system for the help of people who want to browse quickly. As with the tricks wiki, this general idea seems as though it ought to be fairly easy to implement, and could lead to a very useful resource. But again I’d be interested to know other people’s ideas about the details of how precisely it should work.

Thinking about it slightly more, I notice a difficulty that needs to be addressed. It’s not enough to stop people reviewing their own papers—they must also be stopped from editing other people’s reviews. In the end I think a modification of Amazon’s system of book reviews might be best: authors control reviews; they also give ratings; there can be several reviews of the same paper; there is a facility for others to say whether they found a review helpful; there is also a facility for people to suggest changes to reviews, which the authors are encouraged to implement if they are sensible ones.

Just to finish, let me explain why I think it could hasten the end of print journals, at least for the majority of papers. It’s that if it became very successful, and if appropriate safeguards were in place (but I’m not sure what they would have to be), then it might be more impressive on a CV that a paper had been given a mega-important rating by seven people, whose reviews (which helpfully explained to a job committee why the paper was interesting and important) had been found helpful by many others, than that it had appeared in Inventiones. Possible problems with that of course, but I throw it out anyway.

16 Responses to “Alternative maths reviews”

  1. Isabel Says:

    This sounds like a good idea to me. As it stands, the only real way one has of judging the “quality” of a paper (whatever that means) is by judging the quality of the journal that it’s printed in, which is probably a significant part of what keeps the journals in business even though the “real” work is getting done in the preprints.

    I’m not sure how accurate it is to say that a paper’s already out of date by the time it’s in a journal (we don’t move as quickly as, say, computer scientists, who seem to have basically abandoned journals for conference proceedings — but then there’s still the problem of good conferences vs. bad conferences), but it seems like the “original” purpose of journals — to disseminate new results — is kind of gone.

    I do agree that an amazon-like setup would be most useful, because here there’s real value in not allowing people to review their own papers; this is in contrast to the tricks wiki, where you do want people to write about tricks they have used.

  2. Carol Says:

    Suggest you look at the site which offers comments on biological papers, Faculty of 1000. An element missing from your suggestion is whether the people allowed to write to the site are vetted.

  3. ulfarsson Says:

    One way to around a system like that is the following: Two naughty mathematicians each write a bad paper. Then they give each other a great review and upload the paper. But if an amazon-like setup is implemented these papers would get other (hopefully bad reviews).

    But I really like the idea of setting up a site like this – it would be an awesome resource for beginning graduate students who are looking for a field to specialize in.

  4. Top English WP Blogs « Hành trang 8X Says:

    […] Alternative maths reviews Here’s another idea for a wiki-style website, one that might bring closer the day when mathematicians ceased to […] […]

  5. Andy P. Says:

    One problem I see is that writing good reviews of papers is extremely hard work. For instance, it takes me approx. 3-4 hours to write a good MR review. Now, I agree to review papers for MR because I view it as an important service to the community, but I know many people who simply don’t have the time to do it. What I suspect would happen if a system were set up like the one you propose is that in some areas a few very energetic people would write lots of reviews and in other areas it simply would not happen.

    Another problem is that it would require you to publically say mean things about other people’s work. Of couse, one says these things in private and semi-private settings, but I certainly would be loath to make negative comments about anyone in public. It would be career suicide for a young person! I suppose one could make the reviews anonymous, but that seems even more open to abuse then the current anonymous referee system (after all, no one but you and the editor knows that your paper was rejected…).

  6. gowers Says:

    I agree that the first problem could well be a problem, but perhaps, if there were enough reviews for the site to be incredibly useful to, say, harmonic analysts, appropriately energetic people in another area would be insipired to write enough reviews to get their area involved too.

    As for the second, I don’t see it as a problem at all: the site would not be a forum for making negative comments. The idea, which I think goes some way towards answering your too-much-work objection too, is that people would submit a review if, for their own private reasons, they had made the effort to understand a paper, felt that the effort was worthwhile, and wanted to transmit some of what they had learned so as to make life easier for other people. So the extra effort of writing a review would not include the very considerable effort needed to get to grips with the paper in the first place, and people would write about papers only if they felt some positive enthusiasm for them. This is related to my suggestion that by no means all papers would be reviewed, even in an ideal world: if your paper appeared there it would be a feather in your cap.

    I entirely agree, by the way, that it would be best for these reviews to be signed.

  7. Adam Says:

    This sounds like a groundbreaking idea. I am not sure though if getting
    rid of journals should be one’s goal. I am a bit uneasy about purely
    electronic dissemination – if there ever come times of upheaval (e.g.,
    due to the scarcity of oil and running of world affairs by nutters),
    all information stored in purely electronic way could become lost.
    A similar worry applies to private university repositories. E.g., I would
    be more relaxed if the NSF was the owner of arxiv.
    Also, I think the AMS citation database goes some length towards
    downgrading the practice of publishing trivial papers – given enough
    time, this might by itself clean things up.
    A collection of classical, yet previously hushed-up results, could serve
    as a warning to editorial boards to perform their duties with integrity.
    E.g., it is beyond comprehension how one of most prestigious maths journals can keep a paper refereed for 10 years, as Terry pointed out.
    I think that the introduction of works and their Amazon-style rated reviews should be signed by people with PhDs and having academic affiliations –
    if anything, it would serve to improve the social standing of scientists in
    some countries. Unrated anonymous reviews could also be allowed,
    in case a paper needs to be criticized e.g., if one finds an embarrassing error and doesn’t want to end up like Hippasus.
    How then does one go about setting the whole thing up?

  8. Trustees of human knowledge at Freedom of Science Says:

    […] Gowers proposes a “wiki-style website” to bring closer the day when “mathematicians will cease to […]

  9. Nicola Ciccoli Says:

    A job similar to what is currently done for books by
    There a system of “review value” is maintained allowing any user to judge a review as “useful” or unuseful”, which would avoid overwriting (you add a review only if you think something relevant was missed by other reviews, otherwise you vote one – if you wish).

    The main difference should be an “excellent” browsing system bringing you as
    close as possible to the informations you wish (papers should be tagged according to their MSC and “classified” according to reviews).

  10. Marie Farge Says:

    I like this proposal: you write reviews only about papers you like, to share your enthousiasm with others. If you do not like a paper, you should not waste your time explaining why you don’t like it. As a result, there will be no negative review and, since dull papers will not be reviewed, they will fade away without any action being needed. Concerning papers where you find some mistake, the gentleman’s practice is to contact the author(s) and keep the debate private.

    The burden today is the huge number of papers which are published and that no one (or few) takes the time to read (besides the referees who are bound to do so). Developing the practice of review at large scale and in an open way is certainly an excellent direction where we should go. This practice has a long history in arts and literature, known as ‘la critique littéraire’ (literary criticism). The beauty of the present proposal is that, instead of being critical, it is supportive. Let us call it ‘la recommandation mathématique’ (‘the mathematical recommendation’ may be an appropriate translation).

    It is time to take this very seriously: the number of publications increases while the time avalailable to sit quietly and read them (without being interrupted) decreases, therefore we will soon reach a point where the time spent for reading the papers published in our field will tend towards a set of measure zero. The practice of the ‘mathematical recommendation’ may be a way to overcome this obstruction, and I do not see any objection for not trying to work it out.

  11. gowers Says:

    Dear Marie,

    Good to hear from you, and I’m glad you like the idea — I’m quite serious about trying to do it when I have just slightly more time (and a bit of help, which has already been offered). Watch this space …

    Best wishes,


    PS I’ve taken the liberty of removing the first version of your message (the second was almost identical, but with a typo removed).

  12. Paul Says:

    Why not ask amazon if they would be prepared to set it up and run it? You just use the amazon system with paper titles replacing book titles. What incentive for them? Surely prestige and a small ‘powered by amazon button’ would be enough incentive? And the threat to approach Google 🙂 To avoid cranks, real names and institutions would need to be used for reviewers names & profiles. Note amazon’s ‘report the crank feature’!

  13. Roland Bacher Says:

    The interest of alternative reviews could perhaps be increased if it included some kind of reading lists “for beginners”. There are hundreds of more or less specialized subject in Mathematics and entering a new domain is generally not easy. You can of course use the library or Google but they don’t give advise of the style “Start reading this and then proceed with …”. My favourite method is bothering office-neighbours my but even they are not always helpful. An easily available list of good textbooks (or introductory review articles) sorted into different more or less specialized topics (and reviewed briefly
    mentioning necessary prerequisites) could be a convenient place to to decide what to do first.

  14. Timothy Chow Says:

    I like the idea in principle, but would like to sound a cautionary note. Mathematical Reviews for a time had something they called a “Featured Review.” A reviewer would be asked to write a somewhat longer and more accessible review than average. I think the audience they had in mind was the person who wanted to browse the literature for gems.

    “Featured Reviews” were eventually discontinued. As far as I know, no official reason was given, but through the grapevine I heard that one reason was that MR found that people were treating the list of Featured Reviews as a list of the “best” papers. And MR did not want to set itself up as a judge of which papers were “best.”

    If such a site were set up and were successful, then I do not doubt that people would start treating the highly-ranked papers on such a site as the “best” papers. But we need to think carefully about whether that is really a good thing. The ranking on such a site would be governed by many different factors (some of which other people have mentioned) that would not necessarily correlate with how “good” the paper was. If you doubt this, ask yourself if the “best” books ever written are those which get the most five-star reviews on Amazon.

    This is not to say that such a site should not be set up, but that we should be wary of thinking of it as a *replacement* for print journals. For example: It is always difficult to work in an unfashionable field, but at least in the current system you can (usually) get at least one fair hearing of what you have to say by submitting it to a conventional journal. A new world order in which there is even more pressure to be fashionable, lest *nobody* give your work a fair hearing, is probably not a good thing.

  15. Doing mathematics online « Motivic stuff Says:

    […] blogs, and lots of online books and lecture notes. Tim Gowers once proposed a site with alternative maths reviews. There are various useful databases, like the Sloane’s Online Encyclopedia of Integer […]

  16. Sergey Says:

    Similar ideas are being discussed on the Algebraic Topology Discussion List, in the thread starting with

    Recently, related issues were also discussed in the “ru_math” community (in Russian)

    leading to the creation of an actual website of comments to papers posted on the arXiv, see

    Alas, apart from this one thread there seem to be not many substantial comments so far. There is an “advanced search” feature which pops up on doing simple “search”, but I was unable to make it find any nonempty reviews. The creator of the website seems to be very open to feedback, however.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: