Polymath1 and open collaborative mathematics

In this post I want to discuss some general issues that arise naturally in the light of how the polymath experiment has gone so far. First, let me say that for me personally this has been one of the most exciting six weeks of my mathematical life. That is partly because it is always exciting to solve a problem, but a much more important reason is the way this problem was solved, with people chipping in with their thoughts, provoking other people to have other thoughts (sometimes almost accidentally, and sometimes more logically), and ideas gradually emerging as a result. Incidentally, many of these ideas are still to be properly explored: at some point the main collaboration will probably be declared to be over (though I suppose in theory it could just go on and on, since its seems almost impossible to clear up every interesting question that emerges) and then I hope that the comments will be a useful resource for anybody who wants to find some interesting open problems.

The sheer speed at which all this happened contributed to the excitement. In my own case it led to my becoming fairly obsessed with the project and working on it to the exclusion of almost everything else (apart, obviously, from things I absolutely had to do).

But how does what happened compare with my initial fantasy about what might happen? Looked at from that point of view, it was more successful in some ways and less in others.

On the plus side, the mathematical result of the project has far exceeded what I thought would be possible in a mere six weeks. I deliberately set a rather modest aim: to explore just one approach to DHJ(3). In retrospect, this seems not to have been the right decision, though it may have been quite good as a starting point, since in the end we moved off into other directions that were more fruitful (not that I completely rule out a proof along the lines first envisaged, especially given some of the tools that we have now developed). Anyhow, these initial restrictions were quietly abandoned, and it looks as though we have proved a stronger result than seemed remotely feasible then. (More precisely, if we had managed to get my initial suggestion to work, it would probably have been unpleasant, though not impossible, to generalize.)

Also on the plus side, the project has been genuinely collaborative, and has led, to a remarkable extent, to the kind of efficiency gains that I was hoping for. To give one example, Randall McCutcheon made some very useful comments, but they were in the language of ergodic theory, which I understand only in a very limited way. But Terence Tao is a master at translating concepts back and forth between combinatorics and ergodic theory, so I was able to benefit from Randall’s contributions indirectly.

But something I found more striking than the opportunity for specialization of this kind was how often I found myself having thoughts that I would not have had without some chance remark of another contributor. I think it is mainly this that sped up the process so much.

I could go on, but from the point of view of discussion I am more interested in the way that the project fell short of my expectations, and there is one way that stands out. There seemed to be such a lot of interest in the whole idea that I thought that there would be dozens of contributors, but instead the number settled down to a handful, all of whom I knew personally. (Actually, Randall, I know of you so well that I feel as though I know you but I can’t quite remember whether we have met — hope to do so soon.)

This raises two questions. (<rant> By the way, it doesn’t beg any questions at all. </rant>) Why did it happen like this, and does it matter?

I would be very interested to hear from anyone who thought that they might like to contribute but ended up not doing so. I have spoken to one or two people like that, so I know of at least one reason, which I suspect may be the most important: it’s that the number of comments grew so rapidly that merely keeping up with the discussion involved a substantial commitment that not many people were in a position to make. I definitely intend to start another polymath project, but next time I think we may have to have some policy such as writing up all useful insights on the corresponding wiki before we allow ourselves a new comment thread, so that anybody who wants to join the discussion can read about the progress in a condensed and organized form. Or perhaps we should just artificially slow ourselves down. Or perhaps it will just naturally be slower second time round.

Another possible reason is that the problem I chose lent itself more naturally to a smaller collaboration, since in order to be well placed to think about the density Hales-Jewett theorem it was a huge help to be familiar with, and to have thought hard about, other related results. So the pool of potential collaborators was not as large as it might have been (though it was still substantially larger than the number of people who did contribute). The next problem I have in mind is less like that: it should be possible to contribute with virtually no prior knowledge.

A third possible reason is that there are a lot of experts out there who could in principle have contributed but who just aren’t part of the blogosphere in any serious way. Amongst them are probably several people who would not in any case feel comfortable about airing their thoughts so publicly.

Does it matter? In a way no: this smaller collaboration has worked very well, and, as several people have commented, it has provided, for possibly the first time ever (though I may well be wrong about this), the first fully documented account of how a serious research problem was solved, complete with false starts, dead ends etc. It may be that the open nature of the collaboration was in the end more important than its size. I can even imagine solo polymath-type projects, where somebody thinks online just to interest anyone who might be interested.

Another thing I have found good about the project is that it has made it possible to work hard without having the sensation of working hard. There are some people who have brains that they seem to be able to split into three or four parts that can work independently, one part solving problems, another digesting the literature, another blogging, another giving lecture courses. My own brain works in series rather than in parallel, but now that I have found a way of simultaneously blogging and carrying out research I can do two things at the same time by identifying them.

However, there is still a part of me that would like to see whether a much bigger collaboration might be possible. I think a lesson of this one is that a big collaboration would need an extra level of organization, so that it was possible to work on part of the project without keeping track of the whole of it. But finding a good way of doing that would be quite a challenge: when we tried to split this discussion into separate threads, it didn’t really work, except for the thriving thread on DHJ numbers on Terry’s blog, which worked because it was a more or less disjoint enterprise from the one here.

There are some other questions that need discussing, such as the best way to write results up, what appropriate conventions should be for referring to work here, and so on. Comments on these and other practical questions are also welcomed.