Comments on: Alternative maths reviews

By: Sergey

Sergey — Sat, 26 Sep 2009 19:58:43 +0000

Similar ideas are being discussed on the Algebraic Topology Discussion List, in the thread starting with

https://lists.lehigh.edu/pipermail/algtop-l/2009q3/000570.html

Recently, related issues were also discussed in the “ru_math” community (in Russian)

http://community.livejournal.com/ru_math/720263.html
http://community.livejournal.com/ru_math/723781.html

leading to the creation of an actual website of comments to papers posted on the arXiv, see

Alas, apart from this one thread there seem to be not many substantial comments so far. There is an “advanced search” feature which pops up on doing simple “search”, but I was unable to make it find any nonempty reviews. The creator of the website seems to be very open to feedback, however.

By: Doing mathematics online « Motivic stuff

Doing mathematics online « Motivic stuff — Sun, 12 Apr 2009 01:19:11 +0000

[...] blogs, and lots of online books and lecture notes. Tim Gowers once proposed a site with alternative maths reviews. There are various useful databases, like the Sloane’s Online Encyclopedia of Integer [...]

By: Timothy Chow

Timothy Chow — Thu, 04 Sep 2008 00:29:07 +0000

I like the idea in principle, but would like to sound a cautionary note. Mathematical Reviews for a time had something they called a “Featured Review.” A reviewer would be asked to write a somewhat longer and more accessible review than average. I think the audience they had in mind was the person who wanted to browse the literature for gems.

“Featured Reviews” were eventually discontinued. As far as I know, no official reason was given, but through the grapevine I heard that one reason was that MR found that people were treating the list of Featured Reviews as a list of the “best” papers. And MR did not want to set itself up as a judge of which papers were “best.”

If such a site were set up and were successful, then I do not doubt that people would start treating the highly-ranked papers on such a site as the “best” papers. But we need to think carefully about whether that is really a good thing. The ranking on such a site would be governed by many different factors (some of which other people have mentioned) that would not necessarily correlate with how “good” the paper was. If you doubt this, ask yourself if the “best” books ever written are those which get the most five-star reviews on Amazon.

This is not to say that such a site should not be set up, but that we should be wary of thinking of it as a *replacement* for print journals. For example: It is always difficult to work in an unfashionable field, but at least in the current system you can (usually) get at least one fair hearing of what you have to say by submitting it to a conventional journal. A new world order in which there is even more pressure to be fashionable, lest *nobody* give your work a fair hearing, is probably not a good thing.

By: Roland Bacher

Roland Bacher — Mon, 14 Apr 2008 15:43:37 +0000

The interest of alternative reviews could perhaps be increased if it included some kind of reading lists “for beginners”. There are hundreds of more or less specialized subject in Mathematics and entering a new domain is generally not easy. You can of course use the library or Google but they don’t give advise of the style “Start reading this and then proceed with …”. My favourite method is bothering office-neighbours my but even they are not always helpful. An easily available list of good textbooks (or introductory review articles) sorted into different more or less specialized topics (and reviewed briefly
mentioning necessary prerequisites) could be a convenient place to to decide what to do first.

By: Paul

Paul — Mon, 24 Sep 2007 14:42:57 +0000

Why not ask amazon if they would be prepared to set it up and run it? You just use the amazon system with paper titles replacing book titles. What incentive for them? Surely prestige and a small ‘powered by amazon button’ would be enough incentive? And the threat to approach Google To avoid cranks, real names and institutions would need to be used for reviewers names & profiles. Note amazon’s ‘report the crank feature’!

By: gowers

gowers — Sun, 23 Sep 2007 22:30:15 +0000

Dear Marie,

Good to hear from you, and I’m glad you like the idea — I’m quite serious about trying to do it when I have just slightly more time (and a bit of help, which has already been offered). Watch this space …

Best wishes,

Tim

PS I’ve taken the liberty of removing the first version of your message (the second was almost identical, but with a typo removed).

By: Marie Farge

Marie Farge — Fri, 21 Sep 2007 23:50:30 +0000

I like this proposal: you write reviews only about papers you like, to share your enthousiasm with others. If you do not like a paper, you should not waste your time explaining why you don’t like it. As a result, there will be no negative review and, since dull papers will not be reviewed, they will fade away without any action being needed. Concerning papers where you find some mistake, the gentleman’s practice is to contact the author(s) and keep the debate private.

The burden today is the huge number of papers which are published and that no one (or few) takes the time to read (besides the referees who are bound to do so). Developing the practice of review at large scale and in an open way is certainly an excellent direction where we should go. This practice has a long history in arts and literature, known as ‘la critique littéraire’ (literary criticism). The beauty of the present proposal is that, instead of being critical, it is supportive. Let us call it ‘la recommandation mathématique’ (‘the mathematical recommendation’ may be an appropriate translation).

It is time to take this very seriously: the number of publications increases while the time avalailable to sit quietly and read them (without being interrupted) decreases, therefore we will soon reach a point where the time spent for reading the papers published in our field will tend towards a set of measure zero. The practice of the ‘mathematical recommendation’ may be a way to overcome this obstruction, and I do not see any objection for not trying to work it out.

By: Nicola Ciccoli

Nicola Ciccoli — Tue, 18 Sep 2007 15:22:33 +0000

A job similar to what is currently done for books by http://www.anobii.com.
There a system of “review value” is maintained allowing any user to judge a review as “useful” or unuseful”, which would avoid overwriting (you add a review only if you think something relevant was missed by other reviews, otherwise you vote one – if you wish).

The main difference should be an “excellent” browsing system bringing you as
close as possible to the informations you wish (papers should be tagged according to their MSC and “classified” according to reviews).

By: Trustees of human knowledge at Freedom of Science

Trustees of human knowledge at Freedom of Science — Tue, 18 Sep 2007 01:32:00 +0000

[...] Gowers proposes a “wiki-style website” to bring closer the day when “mathematicians will cease to [...]

By: Adam

Adam — Mon, 17 Sep 2007 15:24:19 +0000

This sounds like a groundbreaking idea. I am not sure though if getting
rid of journals should be one’s goal. I am a bit uneasy about purely
electronic dissemination – if there ever come times of upheaval (e.g.,
due to the scarcity of oil and running of world affairs by nutters),
all information stored in purely electronic way could become lost.
A similar worry applies to private university repositories. E.g., I would
be more relaxed if the NSF was the owner of arxiv.
Also, I think the AMS citation database goes some length towards
downgrading the practice of publishing trivial papers – given enough
time, this might by itself clean things up.
A collection of classical, yet previously hushed-up results, could serve
as a warning to editorial boards to perform their duties with integrity.
E.g., it is beyond comprehension how one of most prestigious maths journals can keep a paper refereed for 10 years, as Terry pointed out.
I think that the introduction of works and their Amazon-style rated reviews should be signed by people with PhDs and having academic affiliations –
if anything, it would serve to improve the social standing of scientists in
some countries. Unrated anonymous reviews could also be allowed,
in case a paper needs to be criticized e.g., if one finds an embarrassing error and doesn’t want to end up like Hippasus.
How then does one go about setting the whole thing up?

By: gowers

gowers — Sun, 16 Sep 2007 09:21:06 +0000

I agree that the first problem could well be a problem, but perhaps, if there were enough reviews for the site to be incredibly useful to, say, harmonic analysts, appropriately energetic people in another area would be insipired to write enough reviews to get their area involved too.

As for the second, I don’t see it as a problem at all: the site would not be a forum for making negative comments. The idea, which I think goes some way towards answering your too-much-work objection too, is that people would submit a review if, for their own private reasons, they had made the effort to understand a paper, felt that the effort was worthwhile, and wanted to transmit some of what they had learned so as to make life easier for other people. So the extra effort of writing a review would not include the very considerable effort needed to get to grips with the paper in the first place, and people would write about papers only if they felt some positive enthusiasm for them. This is related to my suggestion that by no means all papers would be reviewed, even in an ideal world: if your paper appeared there it would be a feather in your cap.

I entirely agree, by the way, that it would be best for these reviews to be signed.

By: Andy P.

Andy P. — Sun, 16 Sep 2007 08:45:56 +0000

One problem I see is that writing good reviews of papers is extremely hard work. For instance, it takes me approx. 3-4 hours to write a good MR review. Now, I agree to review papers for MR because I view it as an important service to the community, but I know many people who simply don’t have the time to do it. What I suspect would happen if a system were set up like the one you propose is that in some areas a few very energetic people would write lots of reviews and in other areas it simply would not happen.

Another problem is that it would require you to publically say mean things about other people’s work. Of couse, one says these things in private and semi-private settings, but I certainly would be loath to make negative comments about anyone in public. It would be career suicide for a young person! I suppose one could make the reviews anonymous, but that seems even more open to abuse then the current anonymous referee system (after all, no one but you and the editor knows that your paper was rejected…).

By: Top English WP Blogs « Hành trang 8X

Top English WP Blogs « Hành trang 8X — Sun, 16 Sep 2007 00:27:49 +0000

[...] Alternative maths reviews Here’s another idea for a wiki-style website, one that might bring closer the day when mathematicians ceased to […] [...]

By: ulfarsson

ulfarsson — Sat, 15 Sep 2007 15:41:48 +0000

One way to around a system like that is the following: Two naughty mathematicians each write a bad paper. Then they give each other a great review and upload the paper. But if an amazon-like setup is implemented these papers would get other (hopefully bad reviews).

But I really like the idea of setting up a site like this – it would be an awesome resource for beginning graduate students who are looking for a field to specialize in.

By: Carol

Carol — Sat, 15 Sep 2007 14:22:01 +0000

Suggest you look at the site which offers comments on biological papers, Faculty of 1000. An element missing from your suggestion is whether the people allowed to write to the site are vetted.

By: Isabel

Isabel — Sat, 15 Sep 2007 11:53:20 +0000

This sounds like a good idea to me. As it stands, the only real way one has of judging the “quality” of a paper (whatever that means) is by judging the quality of the journal that it’s printed in, which is probably a significant part of what keeps the journals in business even though the “real” work is getting done in the preprints.

I’m not sure how accurate it is to say that a paper’s already out of date by the time it’s in a journal (we don’t move as quickly as, say, computer scientists, who seem to have basically abandoned journals for conference proceedings — but then there’s still the problem of good conferences vs. bad conferences), but it seems like the “original” purpose of journals — to disseminate new results — is kind of gone.

I do agree that an amazon-like setup would be most useful, because here there’s real value in not allowing people to review their own papers; this is in contrast to the tricks wiki, where you do want people to write about tricks they have used.