In this post I want to discuss a combinatorial problem that is very appealing in its own right, but also important as a potential first step towards solving a central open problem in Ramsey theory. It is the second in a series of three posts (I may add to this number later, but three is what I have written or semi-written so far) that give further details about possible Polymath projects. This one was number 2 in the post in which I first discussed these projects. In that post, I said nothing beyond the fact that the project had close connections with the density Hales-Jewett theorem. Unlike the origin-of-life suggestion, this project is a straightforward mathematical one.

Let me very briefly indicate what the central open problem in Ramsey theory is. The density Hales-Jewett theorem, which has been discussed at great length on this blog, is the assertion that every dense subset of contains a combinatorial line (provided that is large enough in terms of and the density). This implies Szemerédi’s theorem.

Now there is an amazing generalization of Szemerédi’s theorem, due to Bergelson and Leibman, known as the polynomial Szemerédi’s theorem. This is the following assertion. For any and any choice of polynomials with integer coefficients and constant terms equal to zero, there exists such that every subset of size at least contains a subset of the form with . To see that this generalizes Szemerédi’s theorem, just let (unless you feel that is more natural).

Another special case of this theorem is when and . Then one is trying to find a subset of the form , or in other words a pair of elements of that differ by a perfect square. This was proved independently by Sarkozy and Furstenberg, though my favourite proof is a more recent argument due to Ben Green that follows more closely the general structure of Roth’s proof of Roth’s theorem.

An obvious question now arises: is there a generalization of DHJ that implies both DHJ and the polynomial Szemerédi’s theorem? So far, the answer is no, and that is the central problem I was referring to earlier. However, we do at least know the colouring version: that is, there is a colouring statement that simultaneously generalizes the Hales-Jewett theorem and van der Waerden’s theorem. This is a result of Bergelson and McCutcheon — alternative proofs have been given by Mark Walters (who has the shortest and simplest argument) and Saharon Shelah (who has produced a primitive recursive bound).

A result of this work is that we know what the correct statement of polynomial DHJ ought to be. It is slightly complicated, so first let me give one special case that is already enough to convey the flavour of it.

Conjecture 1. For every and every positive integer there exists such that if is any subset of of density at least , then contains a combinatorial line such that the wildcard set is of the form for some subset .

By considering the map that takes each element of to the sum of all its values, we find that the conjecture above implies the following special case of the polynomial Szemerédi theorem.

Theorem 2. For every and every there exists such that every subset of of density at least contains an arithmetic progression of length whose common difference is a perfect square.

The deduction is not quite trivial, but it is not too hard either. The main reason it holds is that the cardinality of is a perfect square.

Here is (one version of) the full polynomial density Hales-Jewett conjecture. It is a bit like a set-theoretic equivalent of trying to find and such that belongs to a given set whenever for every , just as Conjecture 1 resembles Theorem 2.

Conjecture 3. For every and every pair of positive integers and there exists such that if is any subset of of density at least , then contains a -dimensional combinatorial subspace such that the wildcard sets are of the form for some subset (where is understood to be a subset of the that appears in the product).

As far as I am aware, the polynomial version of DHJ is the only interesting statement of the form “Does every dense subset of such and such a structure contain a substructure of such and such a type?” where the answer is not known, even if one asks nothing at all about bounds. (I’m not saying that there are definitely no other interesting purely qualitative unsolved density conjectures, but I can’t think of any. I’d be interested to hear of others.)

One might expect Conjecture 1 to become hard when , but it is in fact not known even when . On reflection, this is not quite so surprising, since it implies the theorem of Sarkozy and Furstenberg, which is far from trivial. On the other hand, the theorem of Sarkozy and Furstenberg is significantly easier than the full polynomial Szemerédi theorem, so perhaps there is some hope of tackling the case of Conjecture 1.

A small variant of this case is particularly appealing combinatorially, and is what I would actually suggest as a good Polymath project.

Conjecture 4. For every there exists such that if is any collection of at least graphs with vertex set , then contains two distinct graphs and such that and is a clique.

From this conjecture one can deduce the special case of the polynomial Szemerédi theorem where one is looking for a pair of numbers of the form . (The polynomial does not have integer coefficients, but that is a minor technicality.) Bergelson and Leibman proved their theorem by first proving the colouring version (the polynomial van der Waerden theorem) and then using general ergodic machinery to deduce the density version. The colouring version followed by a sophisticated induction scheme (which they called PET induction), and in that scheme the polynomial was “easier” than the polynomial . Thus, there are two reasons for looking at this question about dense sets of graphs: it has a nice combinatorial formulation that does not require one to understand the terminology connected with the Hales-Jewett theorem, and it ought to be (trivially equivalent to) the easiest unsolved case of polynomial DHJ.

Here is a looser way of stating the conjecture: a dense set of graphs must contain two that differ by a clique. How might one go about proving this?

Here is an approach that doesn’t work, but it is still worth thinking about it in order to get a reasonable understanding of the difficulties of the problem. We can think of Conjecture 4 as a strengthening of Sperner’s theorem: we can reformulate Sperner’s theorem as the assertion that if you have a dense set of graphs, then you can find distinct graphs and in your set such that , and Conjecture 3 adds the requirement that should be a clique. (Of course, if we do not add some such requirement then the resulting statement is not really a statement about graphs at all.)

The “correct” proof of Sperner’s theorem, if we pretend that it is a statement about graphs, proceeds by considering a random ordering of all the edges of the complete graph on vertices. If is a dense set of graphs, and if we start with the empty graph and build up to the complete graph by adding random edges one at a time, then the expected number of times we produce a graph in is equal to times the equal-slices density of (which can be defined as the probability that a graph belongs if you first choose an integer uniformly at random from the set and then choose with edges uniformly at random from all such graphs). It is easy to show that if is dense, then it has equal-slices density greater than , and this completes the proof of Sperner’s theorem.

Can we adapt this idea to prove a properly graph-theoretic version of Sperner’s theorem? If we could find a way of randomly building up a sequence of graphs that started with the empty graph and ended with the complete graph and had the property that any two graphs in the sequence differed by a clique, then we would be in business. However, a moment’s thought shows that that is not even close to being possible: indeed, a union of two cliques cannot be a clique, so even a very short sequence of such graphs does not exist.

But it is not necessary to construct a sequence with such a ridiculously strong property. For example, suppose that one could build a sequence of graphs (where ) such that has edges, and is a clique whenever for some (which is clearly a necessary condition). That wouldn’t immediately lead to a proof, but it would be quite encouraging, because if ever intersected such a sequence densely, then by the polynomial Szemerédi theorem (in fact, this case follows also from the work of Sarkozy and Furstenberg) it would have to contain two graphs that differed by a clique.

Even this property is far too strong, however. For instance, it would require to be a triangle for every , and it doesn’t take long to convince oneself that that is impossible.

Nevertheless, these discouraging observations do not completely rule out covering almost all graphs by sequences that are sufficiently rich in clique differences for it to be possible to reduce the problem to a problem about sets of integers. One possible line of enquiry would be to see whether all such proofs can be ruled out. For example, perhaps it is the case that for every chain of graphs one can find a set of at least graphs in the chain such that no two differ by a clique. That would completely kill off all hopes of a Sperner-type proof, and it is also quite a nice problem in itself (though it may turn out to be easy).

Let me state that formally as a problem. It strikes me as an almost compulsory question to try to answer if one wishes to prove Conjecture 4.

Problem 5. Is it true that for every there exists a sequence of distinct graphs such that for every subset of size at least there exist such that and is a clique?

I suspect that this question is not too hard. In fact, I think Mark Walters probably already knows the answer. (No logical relationship is intended between those two sentences.) If the answer is yes, then one could set about trying to decompose the set of all graphs on vertices into a union of such sequences (or do something more general like approximating the uniform measure on the set of all graphs by a weighted sum of such sequences). However, I think it will be more interesting if the answer is no. Then one would be faced with the following question: what could a proof of Conjecture 4 conceivably look like?

One difficulty we would face with this problem if we adopted it as a Polymath project is that we do not have as wide a variety of model proofs to work with as we had with DHJ. The obvious theorem to try to generalize is the theorem of Sarkozy and Furstenberg, but the only two proof techniques that have so far given proofs of that theorem are Fourier analytic techniques (used by Sarkozy, and also in subsequent proofs such as that of Green) and ergodic techniques. It seems to be hard to develop an analogue of Fourier analysis in a Hales-Jewett setting—we had a try at that with DHJ(3), and although we did have some ideas we did not manage to use them to prove DHJ(3) itself. Having said that, we did find a Fourier-ish proof of Sperner’s theorem, so perhaps it would be worth revisiting those ideas.

An alternative approach might be to try to find a more combinatorial proof of the theorem of Sarkozy and Furstenberg. Given the way that the PET induction scheme works, it might be sensible to assume Szemerédi’s theorem. If we could find a density increment argument that made use of Szemerédi’s theorem, then we might be able to generalize it in a way that was similar to the way the modified version of Ajtai and Szemerédi’s proof of the corners theorem was generalized to yield DHJ(3). That is quite a long shot, but it still seems worth asking the following question.

Problem 6. Is there a combinatorial proof, using Szemerédi’s theorem as a black box, that shows that if is a dense subset of that contains no pair , then there is an arithmetic progression with length tending to infinity such that the density of inside is at least ?

How might Szemerédi’s theorem be useful, even in principle? I’m not sure, but the kind of thing I vaguely imagine is, say, a clever trick that associates a two-dimensional set with , finds a density increment in some set , where is an arithmetic progression and is a translate of , and does something clever with the diagonals.

That’s probably so vague as to be useless. One might get something better by examining the colour-focusing argument of Walters. It is admittedly a colouring argument, but it would at least give some ideas if we wanted a proof of the following kind (which is the basic form of the DHJ proof).

Step 1: Find a configuration of a certain kind in which is dense.

Step 2: Argue that many numbers cannot be in as a result, and that these numbers form a set with a certain structure to it. (This is where a close examination of Walters’s argument might give us some clues.)

Step 3: By averaging, show that has a density increment on a structured set.

Step 4: Show that a structured set can be almost entirely partitioned into arithmetic progressions with square common difference. (It’s plausible that Szemerédi’s theorem might be useful for this step.)

Step 5: By averaging, obtain a density increment on an arithmetic progression of square common difference.

Step 6: Iterate.

***********************

I am writing this section of the post after having been in contact with Mark Walters. It seems that part of what I wrote above may be wrong (in as far as a matter of judgment can be wrong). It turns out that he does not know the answer to Problem 5 above, and despite what I said above that is good evidence that the problem is after all hard. His instinct is that such a sequence of graphs does exist, and that if it didn’t exist then the entire theorem would be false.

That suggests another question one could think about.

Problem 7. Suppose that for every there is a dense set of graphs with vertex set such that no two differ by a clique. Does it follow that for every nested sequence there is a dense subset such that if then is not a clique?

In case that sounds a bit far fetched, let me indicate how one might go about proving it (not that I have any particular reason to suppose that this proof idea works). Given a nested sequence of graphs one would choose some that is much much bigger than and then cover the set of all graphs with vertex set as evenly as possible with “copies” of this sequence. Then one would take a dense set of graphs with vertex set and no clique difference, and by averaging one would find a copy of that contained at least elements of this dense set. This would show that the copy of had a dense subset with no clique difference, and hence that itself had a dense subset with no clique difference.

Since almost all graphs contain almost all small graphs as induced subgraphs, and do so with roughly the same frequency, it could be that the above sketch is basically correct, but I’d need to check. If it is, then Problem 5 is, as Mark thought, equivalent to the whole problem. In that case, attention would focus on the following question: how does one produce a nested sequence of graphs that is “rich” in clique differences?

To make that question very slightly more precise, let us define a class of graphs as follows. Given a nested sequence of , let be the graph with vertex set where is joined to if and only if is a clique. Let us call a graph constructed in this way a clique-difference graph. Problem 5 can be reformulated as follows.

Problem 5A. Is it true that for every there exists a clique-difference graph with vertices and independence number at most ?

(The independence number of a graph is the largest possible number of vertices you can find such that no two are joined, so this is equivalent to saying that you cannot find graphs in the corresponding nested sequence without two of them differing by a clique.)

Having reformulated Problem 5 in this way, we are in a position to ask weaker questions, and therefore to give ourselves some hope of getting started. For instance, one could ask the following general question.

Problem 8. What properties do clique-difference graphs have?

To give some idea of the kind of result one might hope for here, let me make the simple observation that clique-difference graphs are all triangle free, since a disjoint union of two cliques is never a clique. So we are searching for a triangle-free graph with small independence number (and hence large chromatic number). This might sound a bit worrying, but the difficulty already arises in the number-theoretic case. For instance, if you join two integers when they differ by a cube, you get a triangle-free graph (by a special case of Fermat’s last theorem, proved by Euler), but all dense sets of integers contain a difference equal to a cube (so there’s yet another solution to the nice problem of finding triangle-free graphs with arbitrarily large chromatic number).

Of course, the fact that clique-difference graphs are triangle free is working against us, but it may be that if we could understand the restrictions that clique-difference graphs have to satisfy, we would then be in a better position to work out whether clique-difference graphs with small independence number exist.

If constructing good clique-difference graphs seems to be too difficult, then another line of attack might be a probabilistic one. The following question encapsulates what I mean.

Problem 9. Let and define a nested sequence of graphs with vertex set by taking to have no edges and letting be the union of with a random edge not contained in . What can we say about the resulting clique-difference graph?

Looking at that after having written it, I’m not sure it’s such a great question. It looks to me as though if you add edges completely randomly, then there will be almost no clique differences in the resulting sequence. But perhaps one can do something more ingenious, such as adding edges in a way that favours vertices of high degree. (Such processes have been studied in detail as a way of modelling how the internet grows.) So let us interpret “random edge” above to mean “edge chosen according to some clever procedure that involves at least some randomization”. Even then, it is by no means clear that randomization will work better than a careful deterministic choice of graph sequence that creates many clique differences.

This is the final post in the series about complexity lower bounds. It ends not with any grand conclusion, but just with the three characters running out of steam. The main focus of this final instalment is the Gaussian-elimination problem mentioned in earlier instalments (find an explicit nonsingular matrix over that needs a superlinear number of row operations to turn it into the identity). The discussion follows a familiar pattern, starting out with some ideas for solving the question, understanding why they are hopelessly over-optimistic, and ending with some speculations about why even this problem might be extremely hard.

You know, since we started thinking about the Walsh matrix, I’ve begun to have my doubts about the burst of enthusiasm I had for the natural-proofs barrier when you first told me about it.

How do you mean?

Well, I started applying it to pretty well everything with no evidence to back up what I was doing. I gave a model for a random formula that turned out to be pretty hopeless, though it seems I was lucky and that pseudorandom generators with low formula complexity are known to exist (subject to the usual hypotheses). But I also said that it was plausible that no polynomial-time algorithm could distinguish between a random non-singular matrix over and a random matrix obtained from the identity by row operations. We’ve subsequently seen that that statement is false for an obvious reason: the second matrix will be sparse. It is possible to get round that problem, but I don’t see a strong argument for its being hard to give a superlinear lower bound, particularly given that the obvious way of producing a Walsh matrix takes time I think the point at which the problem becomes hard could just as easily be at around which is also the point at which a random sequence of row operations ought to produce a random-looking matrix.

So do you have some idea of how you might go about proving a lower bound of ?

I do actually. Not a properly thought through idea, so it’s very unlikely to work. But it’s just one of those ideas that you have to think about before dismissing it.

I think I know the kind you mean …

Ha ha. Anyhow, my thought was that when you build the Walsh matrix in the obvious way, you start with the identity matrix. Then you make that into a block diagonal matrix with copies of going down the diagonal. Then you pair those up to make copies of and so on. You have to do steps, and each step takes you row operations.

Now perhaps there is some parameter that you can associate with matrices, with the following property. Let be the block diagonal matrix that consists of copies of We would like to show that the number of row operations you need to apply to a matrix with in order to obtain a matrix with is at least

I also have in mind an obvious parameter to try.

Oh yes?

Yes. It’s quite similar to the norm, and comes from the theory of quasirandom graphs. You take your matrix, and convert it into a matrix in the usual way. If the converted matrix is then the parameter is

If is 1 everywhere, then you obviously get 1 for this expression. Also, since you can rewrite the expression as

and since when it follows that the value taken by the parameter is at least the probability that which is of course the reciprocal of the size of the set that and come from. In our case this size is

Now what we are basically doing in calculating this parameter is taking the inner products of all pairs of rows, squaring them, and taking the average. Since the rows of a Walsh matrix (in its version) are orthogonal, we see that it has exactly the minimal value of the parameter I’ve just defined.

I think that means you want to take the reciprocal. Earlier you said you wanted it to take at least row operations to get the parameter to increase by a certain amount.

Yes, you’re right. Or we might find that we wanted to go for some kind of dual definition to this one, as we did when we were looking at norms, in which case we’d have an increasing parameter.

Do you mind saying exactly what you mean?

Not at all. If and are two real matrices of the same size, define to be And then, if is the parameter I defined above, define a dual parameter by

If is a matrix, then taking gives us the inequality So the Walsh matrix has parameter at least (and in fact equal to)

So the fourth power that you put in there was just to make the maximum equal to rather than

Sort of. Without the fourth power we get a norm, but the parameter was the fourth power of a norm and somehow it feels nicer to reflect that by making the fourth power of a norm as well.

Anyhow, what I’d very much like is for the value of the parameter at to double (or possibly halve if we decide to decrease) when is replaced by It seems that is not going to do that for us: the problem is that the identity matrix becomes a diagonal of s in a sea of s, which is very unbalanced. Indeed, all the s in their versions are unbalanced.

But being unbalanced is not so much of a defect when we take dual parameters, so perhaps we’ll have better luck there. What is of the identity?

Hmm …

You’ve gone all quiet.

Sorry, I had to do some calculations. I’m coming to the conclusion that it is a nuisance to have matrices here. I want to define the parameter to be where is defined exactly as above but is a 01 matrix rather than a matrix. And now, if some hasty and unchecked guesses and calculations are to be believed, of the identity is something like

Just as a matter of interest, how did you arrive at that figure?

I guessed that the Walsh matrix would be a good matrix for norming the identity. I suppose I should have tried the identity itself. What do we get there? Well, of the identity is so Since it follows that

I must apologize. That is now my revised guess. Indeed, I would guess that is always worked out by hitting it with the appropriate multiple of

Well it looks easy to work out The probability that and all come from the same block is and the expectation of the thing you average over given that they do is So you end up with So the right multiple of should be The inner product of with itself is, up to a constant factor, Putting in the normalizing factor we get Finally, raising this to the power 4 gives us as the value, up to a constant, of Note that this isn’t a proof, because we haven’t proved that is self-norming. But it seems to be at least potentially true.

Right, and if it is true, then we could hazard a guess of the following kind. Perhaps in order to double the value of you have to apply at least a linear number of row operations.

I’m afraid I’m starting to feel very suspicious again.

Why?

Because the matrices are very sparse when is small, and could therefore be giving you a misleading impression. If we go back to the idea of creating an upper triangular matrix and then randomly applying row operations, we might well find that we could get in linear time to a matrix with In fact, I wouldn’t be surprised if you could go all the way.

I’m not saying that you can get the Walsh matrix this way: it seems quite plausible, as you say, that the obvious way of producing it is the best. However, I think the parameter you are looking at may be too close to natural to work.

Ah yes. If you think about eigenvalues, I think you find that of a random matrix will be within a constant of the maximum, so if the proof worked it would certainly distinguish between random matrices and matrices that can be produced in operations.

Maybe to get a handle on this we should consider the following model for a random low-complexity matrix. Given any matrix and any permutation of the numbers from to let stand for the matrix you obtain from by adding row to row then row to row and so on all the way up to So row of is the sum of rows up to of

And now, to produce a matrix in linear time, choose permutations and take the matrix

This can of course be thought of in another way: let be the matrix that is 0 below the diagonal and 1 on and above it; then take matrices obtained by randomly and independently permuting the rows of ; then multiply them all together.

That’s more like it. Now we have a matrix that looks as though it could be hard to distinguish from a purely random (non-singular) matrix.

It seems to me that if this is a good model, then even should be pretty scrambled. Indeed, without loss of generality, is the identity permutation. That means that if and if Writing for and writing for we then find that equals the parity of the set of all such that

If we now take and add rows and (mod 2), then we get the parity of the set of all such that and This will be some random variable (depending on ), and as long as and are not too close to each other and and are also not too close to each other (a sufficient condition is that is significantly bigger than ) there will be very little correlation between the value we get at and the value we get at It would seem to follow that of this matrix will be very small, which does I think disprove the lemma that I hoped might show that you can’t get the Walsh matrix in linearly many operations. I’d have to check the details, but I definitely don’t believe in that approach any more. In fact, I think I’d now want to suggest that this new model of random low-complexity matrices could be polynomially indistinguishable from purely random matrices and go back to thinking that there cannot be a “natural” solution to the Gaussian-elimination problem.

Of course, this suggestion is backed up by some serious hard thought, heuristic arguments, etc.?

No, I’m just making it off the top of my head, and am ready to abandon it if you can think of a clever way of working out what and are if you are given the matrix (Right at this moment I don’t even have a way of working out from but I think there might be enough residual non-randomness to do that. But should clear that away I think.)

Can I just share with you a thought that’s so naive-seeming that it really does seem as though it couldn’t possibly work?

Wow, that’s quite a plug. Let’s hear it!

The thought I had was that the number of row operations you need to get from the identity to the Walsh matrix is the same as the number of row operations you need to get from the Walsh matrix to the identity.

Trivially. What good does that do?

Well, as you pointed out, it is initially tempting to base arguments on the false assumption that the intermediate matrices have to be quite sparse. Unfortunately, ’s new model shows that no such argument can work. But what if we argued instead that when you work backwards from the Walsh matrix, it is very hard to get rid of all the 1s that you need to get rid of in order to end up with just of them down the diagonal? If you do some random scramblings, you certainly won’t make them any sparser.

There’s a pretty quick answer to that. If you’re trying to get from to the identity, why should you want to get rid of 1s as quickly as you can? Perhaps that greedy algorithm, so to speak, is far from optimal. Perhaps the best way to get the identity is to do some very clever sequence of operations that gives you a sequence of matrices that look pretty random but are secretly getting simpler (according to some complexity measure that we cannot calculate in polynomial time), and suddenly, ta da, we end up with a matrix that is 1 on and below the diagonal and 0 above it, which we can easily convert into the identity in a further steps.

Basically, the property you were considering was “multiply by the Walsh matrix and see how many zeros you have,” which is far too natural a property to be likely to work.

OK, I thought it was too much to hope for.

By the way, I think it’s easy to work out from Indeed, let be the lower-triangular matrix that’s 1 everywhere on and below the diagonal. Then each row of the matrix is a sum of rows of More precisely, the th row is a sum of the rows of But that means that the difference between rows and of is the th row of I’m fairly sure all you have to do now is search for pairs of rows that have differences that look like this: That will give you a directed graph, inside which you need a directed path of length I think the graph will already be such a path, but haven’t checked this.

What happens if you no longer assume that is the identity permutation?

Ah, perhaps for this algorithmic problem you do lose generality when you take I see your point: it’s not that obvious what row-differences to look out for. But there are a few extremes. For example, you know that there will be a row difference that’s all 1s and another that has just one 1 in it. But I’m not sure that’s enough information to allow for an inductive proof or something like that.

Another question. Suppose it really is the case that no polynomial-time algorithm can distinguish between a random matrix of the form and a purely random matrix. (I’ve gone for three steps to be on the safe side.) What does that tell us about the Gaussian-elimination problem? Is there any conceivable argument that could solve it?

Well, we’d be in a similar situation to the one we are in for circuit complexity. We’d be wanting to find a property that is not given by a polynomial-time algorithm (and if we were being pessimistic we might even guess that the best algorithm took time ), but that is somehow not “trivial”, so that it really does split the original hard problem into two easier parts.

Can you say more precisely what you mean by a “trivial” property?

That is something I would very much like to be able to do. As things stand, I just feel as though I know one when I see it. For example, here the obvious “trivial” property is “Can be obtained using at most row operations”. This distinguishes beautifully between matrices of the form and purely random matrices, but to prove that any given matrix does not have the property is trivially equivalent to the original problem.

So perhaps we should focus on what would in principle count as a non-trivial property.

I have a necessary condition, but it’s very very far from sufficient.

What’s that?

Simply that the property should be something that is shared not just by low-complexity functions but also by other functions as well.

Great!

OK, let me try to say something a tiny bit more interesting. I’d like to focus on properties in NP, since the contrast between “trivial” and “potentially useful” properties shows up already here.

Suppose, then, that we have a property in NP that we want to use to distinguish between low-complexity Boolean functions and random Boolean functions. We can obtain this as follows. Let be a polynomially computable set of Boolean functions defined on where is the set of all Boolean functions defined on and is the set of all Boolean functions defined on Then let be the set of all Boolean functions such that there exists with I want to distinguish between properties that say that in some way includes a computation of and more “natural” properties

How do you propose to do that?

I don’t know, but I have a small idea about a possible way of getting started. It would be to use some logic. Roughly speaking, I’d want to introduce a language that allowed me various notions such as correlation between Boolean functions as primitives, and then I’d want to say that any formula that could be built in that language would give a property that could not give rise to an interesting lower bound.

For example, the property “correlates better with a quadratic than a typical random function does” would be something like “there exists such that is quadratic and “.

But how would you express “ is quadratic” in your language?

I’m not sure. Possibly I would allow all polynomially computable properties of and just place restrictions on how can relate to . But perhaps I’d insist on saying something like

for every which is a nice first-order formula concerning the values of

But ultimately your property is second order because you say “there exists ” and is a function.

Yes. But I just do that once to make a property in NP. Perhaps then one could go further than NP and quantify more than once, obtaining a whole class of properties that cannot work. So in one sense they would be more general than NP, but in another sense they would form a smaller class, because we would be replacing “polynomially computable” by something much more restricted — but perhaps still general enough to be interesting and rule out all the attempts I have made so far. I would find this interesting even if I couldn’t actually prove it rigorously, as long as I could come up with a formulation that was not obviously false. I’d be even happier if I could show that the property in some sense had to involve encoding a computation of , but I don’t know how to make that idea precise.

But if you could do that, wouldn’t you have shown that there was no proof that PNP?

Probably not. I agree that it sounds a bit like that — as though the only property that can distinguish between low-complexity functions and random functions is a property that essentially says “ has low complexity” — but that still leaves open the possibility of a property that fails the largeness condition. In other words, there might still be a property that holds for random functions but is carefully designed to fail for some specific function in NP.

So where do all these conversations leave us?

Sadder but wiser, just as predicted.

This is the first of a few posts I plan (one other of which is written and another of which is in draft form but in need of a few changes) in which I discuss various Polymath proposals in more detail than I did in my earlier post on possible projects.

One of my suggestions, albeit a rather tentative one, was to try to come up with a model that would show convincingly how life could emerge from non-life by purely naturalistic processes. But before this could become a sensible project it would be essential to have a more clearly defined mathematical question. By that I don’t mean a conjecture that Polymath would be trying to prove rigorously, but rather a list of properties that a model would have to have for it to count as successful. Such a list need not be fully precise, but in my view it should be reasonably precise, so that the task is reasonably well defined. It would of course be possible to change the desiderata as one went along.

In this post I’d like to make a preliminary list. It will undoubtedly be unsatisfactory in many ways, but I hope that there will be a subsequent discussion and that from it a better list will emerge. The purpose of this is not to start a Polymath project, but simply to attempt to define a Polymath proposal that might at some future date be an actual project. For two reasons I wouldn’t want this to be a serious project just yet: it seems a good idea to think quite hard about how it would actually work in practice, and someone who I hope will be a key participant is very busy for the next few months and less busy thereafter.

As a starting point, let me mention two ideas that are already out there and have attracted a lot of attention. One is the idea of cellular automata. A fairly general type of cellular automaton can be defined as follows. You have a graph (usually something like an infinite two-dimensional lattice), and at some points you have 1s and at other points you have 0s. You then let the system evolve in rounds according to some simple rule that is usually the same for every vertex. It might be something like this: if at least two of my neighbours are 1s then I will become a 1, and otherwise I will become a 0. It turns out that very simple rules can lead to extremely complicated and interesting behaviour.

What counts as complicated and interesting? Well, perhaps it is better to say what counts as dull. One possible form of dullness is if a system evolves to some state such as the all-1s state, or perhaps a big rectangle full of 1s with 0s outside, or an oscillation between two configurations. Another form of dullness is a system that tends to disperse the 1s until they form some fairly random looking bunch of 1s that never stops looking fairly random. But in between, there are systems that tend to evolve towards some kind of criticality, where you get fractal structures with organization at many different distance scales. One thing that interests people about cellular automata is that there are very simple rules that seem to want to evolve towards these nice “edge of chaos” patterns.

The second idea is self-organized criticality, which is a phenomenon exhibited by certain models in statistical physics — notably the so-called sandpile models. These are supposed to model what happens if you drop grains of sand one by one on to a pile. They will start to build up into a conical shape, but if the sides get too steep there are avalanches. The sizes of these avalanches vary, and if you plot the frequency of avalanches of various sizes, you find (experimentally at least) that they obey a power law. And power laws get people excited because they are what you find associated with critical phenomena. A typical sandpile model is something like this. You have a big square divided into a grid of small squares. You then set all squares equal to 0 except a few randomly chosen ones that you give small integers to. You then add 1 to the central square (let’s assume there is one), and after you have done so you have a rule that says that if any square has value at least 4 it must give 1 to each of its neighbours. This procedure you iterate until no square has value at least 4. (It can be shown that the order in which you do these operations doesn’t matter.) You then add 1 to the central square again, and keep going.

It turns out that the sizes of the “avalanches” that take place here (that is, how many iterations you have to do of the simple rule before all squares have value 0 to 3) also obey a power law, and also that systems such as these have a tendency to evolve towards interesting (that is, not too random and not too structured) configurations. That is, you can get critical phenomena without having to fine-tune some parameter. Again, this has got people excited as it seems to promise an explanation of how the complexity in nature could have started.

In the above description, I made the starting configuration random but after that the way the model evolved was deterministic. There are of course many different possible models, and in some of them the new “grains of sand” are dropped in random places. Again you get interesting critical behaviour.

Now as far as I know, with both cellular automata and sandpile models you get nice critical phenomena appearing, but while they give you pretty patterns they do not give you anything resembling an ecosystem. Yes, Conway’s game of life gives you glider guns and configurations that can reproduce themselves, but you have to set them up carefully in advance, and they don’t seem to do anything all that exciting. They also support universal computation, but again if you want to program the game of life to create an artificial-life simulator, you might as well use a much more powerful computer to do so. What a Polymath project would be looking for is a very simple system with the property that, regardless of the starting configuration, it would tend to develop and eventually produce something that looked like a complex ecosystem.

This brings me to a point that is worth making. The idea of this Polymath project would not be to produce yet another artificial life program, fascinating though those programs can be. One could think of it more like this: can one come up with a very simple model that almost always “self-organizes” and produces something that looks a bit like what you get with artificial life programs? In other words, we would be trying to model abiogenesis rather than evolution.

After that discussion, I think I can have a stab at saying what the properties are that would make a truly interesting and new model. (I am much less sure about the “new” part, and would be interested to hear from people with more knowledge about this kind of topic what the state of the art is.) Some of the properties below seem to be more important than others, but for now I won’t bother to distinguish between those that I regard as essential and those that are merely desirable.

1. It should be a dynamic model that evolves according to simple rules.

2. It should have a tendency to evolve towards patterns with a “critical” character — not too random and not too simple, with interesting features at many distance scales.

3. Probably it should be a somewhat randomized model (to give it a certain robustness). Here I am referring to the rules by which the model develops rather than the initial conditions, but perhaps the initial conditions should be randomized as well.

4. It should have a tendency to produce identifiable macroscopic structures.

5. It should be possible to classify these macroscopic structures in interesting ways. (That is, we would like to be able to say that certain structures look more or less the same as certain others, and ideally this similarity would be a bit more flexible than one just being a translation of another.)

6. These structures should interact with one another, and the interaction should sometimes be destructive (thereby providing some selection pressure).

7. With high probability, self-reproducing structures should eventually emerge. (Before posting this I showed it to Michael Nielsen, who made some interesting points. One of them is that experience in the actual universe suggests that perhaps there should be some fine tuning of parameters before the probability becomes high: after all, life does not evolve on all planets.)

I could go on, but the idea is that once you’ve got 6 and 7, and perhaps a few other properties (for instance, one might decide to have major environmental changes from time to time just to stimulate the development of the system), then natural selection can begin to operate.

Of course, the major challenge is 7. The most plausible route I can see to 7 is a purely probabilistic one: almost all configurations are not self-reproducing, but if a self-reproducing one ever does arise, then it will reproduce itself and start appearing all over the place. But in that case 5 is also a huge challenge. The kinds of structures one would ideally like are not things like the bullets from Conway’s glider guns, but larger configurations that can move about and that are defined more topologically. Indeed, that could be a huge and general problem: the geometry of just isn’t the same as the geometry of , but a continuous model would be very difficult to design and simulate (or would it?). But perhaps there could be some cleverly chosen simple rule that would tend to protect “clumps” of 1s and allow them to move, and to do complicated things like rotating (whatever that can be made to mean in ). Or perhaps a complicated ecosystem could develop that was more -like than -like.

Here, incidentally, is a paragraph from the Wikipedia article on Conway’s game of life, which shows that it is not already an example of what I am talking about:

From a random initial pattern of living cells on the grid, observers will find the population constantly changing as the generations tick by. The patterns that emerge from the simple rules may be considered a form of beauty. Small isolated subpatterns with no initial symmetry tend to become symmetrical. Once this happens the symmetry may increase in richness, but it cannot be lost unless a nearby subpattern comes close enough to disturb it. In a very few cases the society eventually dies out, with all living cells vanishing, though this may not happen for a great many generations. Most initial patterns eventually “burn out”, producing either stable figures or patterns that oscillate forever between two or more states; many also produce one or more gliders or spaceships that travel indefinitely away from the initial location.

We would be looking for something a bit like the Game of Life, possibly randomized, with the important difference that it almost always got more and more complicated and more and more interesting.

Physics

There is one other property that I think would make a model more convincing as an argument for the probability of life arising out of non-life without any magic processes operating. I partly owe this thought to Michael Nielsen, who included the following two questions in a comment he made on the post where I originally mentioned this problem.

(1) How would you go about recognizing self-replicating beings?

(2) What sort of models are “reasonable”, in the sense of both reflecting what we know of physics, and being simple enough to be tractable? The Game of Life isn’t very physical, in that it disobeys many basic physical principles, like conservation of energy, conservation of mass, conservation of momentum, and so on.

One of the things that people often say about life, evolution, biological systems and the like is that they are ways of locally combatting the second law of thermodynamics. So perhaps one could add the following property as one that it would be very nice to have.

8. The general tendency for the model is to become more and more disordered, and eventually to end in heat death, but for there to be many local increases in order.

Of course, one would need to be clear what that meant. The other physical principles that Michael mentioned would also be good to have.

Here is a subproblem that occurs to me as I am writing this. It is connected with the thought that one would like macroscopic structures to have some tendency to survive. In the Game of Life, it seems that structures that survive do so almost by accident — they settle down into some sort of periodicity, say. But structures in the biological world are held together by physical forces, and they have identifiable boundaries and things like that. So one might try to develop a model that captures just this behaviour. As with the main problem, I’m not sure how to formulate this subproblem precisely, but let me have a go. Does there exist a model with the following properties?

(i) If you draw some large-scale shape (think of the 0s and 1s as black and white pixels, say, so the shape is on a much larger distance scale than the distance between two neighbouring points of the grid), it has a tendency to move “continuously”.

(ii) There is a tendency for mass and momentum to be conserved.

To give an idea of the kind of thing I mean here, let’s suppose that “mass” is represented by 1s, and you take a large annulus, place it over , and put a 1 at every grid point that lies in the annulus. Then in the interior circle of the annulus put a random scattering of not too many 1s. And then slightly move the annulus part, and slightly move all the little particles inside. If the first position of the annulus represents where some very simple structure is at time 1 and the second where it is at time 2, then conservation of mass and momentum would tell us to expect it to continue moving in the same direction (so it would be more sophisticated than a cellular automaton of the kind described earlier because its behaviour would depend not just on how it behaved an instant earlier), and to stay the same size. We might also have “forces” between neighbouring 1s that encouraged them to stay together somewhat, and so on.

Of course, as with Conway’s Game of Life, the idea would be to devise the simplest possible set of rules that did what one wanted (in this case preserve macroscopic shapes at least to some extent and allow them to move about reasonably flexibly but without distorting themselves too much). It would not be to try to create the most realistic model one could of the actual world.

Since writing the above paragraphs I’ve found out the following relevant facts. First this from the Wikipedia article on Life-like cellular automata:

Larger than Life is a family of cellular automata studied by Kellie Michele Evans. They have very large radius neighbourhoods, but perform `birth/death’ thresholding similar to Conway’s life. The LtL CA manifest eerily organic `glider’ and `blinker’ structures.

RealLife is the “continuum limit″ of Evan’s Larger Than Life CA, in the limit as the neighbourhood radius goes to infinity, while the lattice spacing goes to zero. Technically, they are not cellular automata at all, because the underlying “space” is the continuous Euclidean plane R2, not the discrete lattice Z2. They have been studied by Marcus Pivato.

Secondly, here is the paper by Marcus Pivato mentioned above.

Chemistry and the problem of scale.

By far the most famous contribution to our understanding of how life started is the Miller-Urey experiment, in which Miller and Urey attempted to simulate the chemical conditions that might have prevailed early on in the life of the Earth. They used electrodes to create lightning-like sparks that passed through a vapour that was formed of water, methane, ammonia and hydrogen, and found that they produced complex amino acids, which are essential building blocks of life.

What relevance would this experiment have for a computer simulation? My view is that one should not necessarily try to produce a virtual Miller-Urey experiment (complete with virtual lightning, virtual ammonia, etc.) but that the experiment does raise a couple of questions that it is essential to address.

A fundamental fact about life as it exists in the physical world is that it is carbon based. The great virtue of carbon is that its particular bonding properties allow it to combine with other atoms to form molecules that are large and complicated enough to encode highly sophisticated information. So an obvious question is this.

Question 1: Should one design some kind of rudimentary virtual chemistry that would make complicated “molecules” possible in principle?

The alternative is to have some very simple physical rule and hope that the chemistry emerges from it (which would be more like the Game of Life approach).

This is just one example of a general tension. The more features you design into a model, the less “universal” it becomes and the less convincing it is as a demonstration of the inevitability of life. However, one can also argue for at least some designed features. After all, if we want to explain the origin of life, it is not necessary to start with a virtual Big Bang and get from there to the possibility of complex molecules. It may be that designing rules to make complex molecules possible (and then arguing that with probability 1 this possibility is actually realized) is attacking the problem at the correct level.

I do not have a strong view about what the right answer to this question is. Obviously I would prefer the chemistry to emerge as if by magic, but that may be an unrealistic hope.

The second question does not arise directly out of the Miller-Urey experiment, but it is related.

Question 2: How large and how complicated should we expect “organisms” to be?

A real-world organism, even a micro-organism, is made out of more atoms than one could hope to simulate on a computer. (I am not certain that that last sentence is correct, but I would be very surprised if it wasn’t. Added later: Michael Nielsen tells me that there are rudimentary organisms that are so small that they could perhaps be simulated in full.) Moreover, although it has many levels of complexity, there will also be distance scales at which it is relatively simple. For example, if I look at my hand from a distance of about a yard, my skin looks smooth. Similarly, if I were to look through a powerful microscope at one of the cells of my hand, then the boundary of that cell would be reasonably smooth, rather than fractal-like. In general, it seems that if you look at a typical organism, it is not equally complicated at all distance scales, but is more like this: you take some small objects and put them together in a reasonably simple way to form bigger objects; you then use these bigger objects as building blocks for yet bigger objects; continuing this process for eight or nine (??) levels (perhaps if I knew more biology I would revise this number up considerably) you end up with a complex organism.

If that picture is roughly correct, then the number of “atoms” in a complex multicellular organism might be prohibitively large for a simulation. Is this a problem?

I think it shouldn’t be too problematic. Just as we are not trying to start with the Big Bang, neither are we trying to end with mammals. The main aim is to get to the point where evolution can take over. In particular, if a readily identifiable micro-organism appeared that could reproduce itself with small modifications, then the simulation would surely be declared a success.

Nevertheless, the question of scale remains. Would we want such a micro-organism to consist of a small handful of “pixels” that by some magic local rule gives rise to a copy of itself? Or would we want something much larger that had “smooth boundaries” at some distance scales and was composed of “complex molecules”? My inclination at the moment is to prefer the second for two reasons: it is less like the Game of Life (and therefore more likely to be novel and interesting) and it is closer to the life forms that we actually observe.

Added later: I haven’t quite made clear that one aim of such a project would be to come up with theoretical arguments. That is, it would be very nice if one could do more than have a discussion, based on intelligent guesswork, about how to design a simulation, followed (if we were lucky and found collaborators who were good at programming) by attempts to implement the designs, followed by refinements of the designs, etc. Even that could be pretty good, but some kind of theoretical (but probably not rigorous) argument that gave one good reason to expect certain models to work well would be better still. Getting the right balance between theory and experiment could be challenging. The reason I am in favour of theory is that I feel that that is where mathematicians have more chance of making a genuinely new contribution to knowledge.

This instalment has a brief discussion of another barrier to proving that PNP, known as algebrization. I don’t fully understand it, and therefore neither do my characters. (I’m hoping that maybe someone can help me with this.) But even a fuzzy understanding has some consequences, and the characters are led to formulate a simpler (and almost certainly already considered by the experts) problem that has the merit that when trying to solve it one is not tempted by proof techniques that would run up against the algebrization barrier. However, for all the usual reasons, this “simpler” problem looks very hard as well.

**************************************

I’m afraid I’m not yet ready to tell you what basic 3-bit operations do to quadratic phase functions.

In that case, can I instead mention something I read that looks relevant?

Sure, go ahead.

Well, you may remember my mentioning that Scott Aaronson and Avi Wigderson have a paper in which they introduce another barrier to lower bound proofs, which they call “algebrization”. If a proof can be algebrized, then it can’t prove that PNP.

What does “algebrized” mean?

Unfortunately, my understanding of it is rather hazy. But it refers to a technique where in order to prove complexity results, you try to approximate low-complexity functions by low-degree polynomials over It turns out to be helpful to look at low-degree field extensions of so one can talk of results holding in various extensions.

A proof that PNP would algebrize if you can show that there is a function in that is not in . Here, is any oracle, and is a low-degree extension of

What’s an oracle?

I’m a bit hazy on oracles too. Basically, it means that you can take some fairly complicated function and assume that your computer takes only one step to calculate it. (The word “oracle” is used for obvious reasons: it’s as though when the computer wants to know the value of this very complicated function, it just goes and asks the oracle.) One then writes and for the sets of functions you can compute/verify in polynomial time if you have access to the oracle It is already known that there are oracles such that and other oracles such that It follows that any proof that PNP must not relativize: that is, must not remain valid even if you throw in an oracle.

Why would a proof remain valid for an arbitrary oracle?

The kinds of proofs we’ve been thinking about wouldn’t, but there is a very different class of techniques that would. The observation about oracles shows that you can’t hope to prove that PNP by means of some clever counting or diagonalization argument.

What do you mean by “diagonalization argument” in this context?

Well, one might try to run through all functions in P and construct a clever function in NP making sure that it disagrees with each function in P in at least one place. That’s the kind of thing you do to prove the insolubility of the halting problem. But for proving that PNP it is known not to work.

Why is that called “algebrization”?

It isn’t. That’s relativization. A proof that PNP must not relativize, meaning that it must not be valid “relative to an arbitrary oracle”. By the way, you might like to check out a post by Terence Tao about the relativization barrier.

I definitely would!

Anyhow, algebrization is a further twist, I think, where you have two oracles, and one is allowed to be a low-degree field extension of the other. Roughly, they show that you cannot prove that PNP if your proof would also show that for every oracle and every low-degree field extension of

But surely that’s already included in the relativization result? Just take

Hmm, that is quite confusing I agree. It may explain why I find this concept quite hard to grasp.

Aha, in the paper, which, incidentally, may be found here (or just Google “algebrization”), they say that they are referring to algebraic oracles, which they define as oracles that can work out not just the value of a function but also its value in any field extension. Since that doesn’t seem to make much sense for an arbitrary Boolean function, I think it must mean that the oracles are not arbitrary—since then you would surely be right—but only allow you to work out the kinds of functions, such as polynomials, that can be interpreted in a larger field. So their result is stronger in one sense—a proof is ruled out if it yields similar results for a rather small class of oracles—but weaker in another—those “similar results” have to show not just that for every algebraic oracle, but the stronger statement that for every algebraic oracle and every low-degree field extension of

But surely every Boolean function can be written as a polynomial.

Well, yes, but the polynomial will depend on which doesn’t really count. Or does it? I’m afraid I’m not sure what the right response is to that question.

What if you have two different polynomials that take equal values over one field but not over an extension?

I don’t think that’s a big problem. You’d just give some algebraic formula to the oracle and the oracle would be able to tell you the answer. It wouldn’t matter if some other algebraic formula happened to agree with it.

I’m sorry my understanding of this is less than perfect, but what I wanted to draw your attention to was the following few sentences from the Aaronson-Wigderson paper:

Can we pinpoint what it is about arithmetization that makes it incapable of solving these problems? In our view, arithmetization simply fails to “open the black box wide enough.” In a typical arithmetization proof, one starts with a polynomial-size Boolean formula , and uses to produce a low-degree polynomial But having done so, one then treats as an arbitrary black-box function, subject only to the constraint that is small. Nowhere does one exploit the small size of except insofar as it lets one evaluate in the first place. The message of this paper has been that, to make further progress, one will have to probe in some “deeper” way.

Ah, I see why you’re saying that. Even if we don’t fully understand the algebrization barrier, we can still be a little sceptical of any proof attempt that makes use of polynomials without also making use of how those polynomials are put together. I think we’ve slightly run up against this thought in some of our earlier discussions actually. It seems to suggest that it is unlikely that we would be able to prove that applying 3-bit operations to arbitrary linear combinations of degree- polynomial phase functions gave you linear combinations with coefficient sums that were not much bigger. That seems to tie in with my experiences with quadratics.

However, having said that I’d like to mention that the degree is by no means the only interesting parameter one can associate with a polynomial over

Why, what else is there?

There’s also its rank.

What does the rank of a polynomial mean? I thought ranks were things that linear maps had.

Well, let me first say what the rank of a quadratic is. Suppose you have a quadratic polynomial We can associate with the symmetric matrix that takes the value if and otherwise. What’s more, this is quite a natural thing to do, because you get precisely the matrix of the bilinear form as you will quickly see if you check it. And there is a very close association between quadratic forms and bilinear forms—though not quite as close over as it is over fields of higher characteristic.

We can now define the rank of the original quadratic to be the rank of the associated bilinear form (or equivalently of the matrix). The rank has a direct impact on the norm of the quadratic phase function that you build out of the quadratic. Indeed, if we write for the quadratic and for the Boolean function then we get

Using the fact that addition and subtraction are the same over one then finds that this is Therefore, the fourth power of the norm of is Now for any fixed the expectation over is unless the function is identically zero, in which case it is In other words, you get unless belongs to the kernel, so to speak, of This shows that the norm is equal to the density of the kernel, which is where is the rank. In other words, the rank of is just

So if you’re talking about quadratics, then the rank is exactly what you are interested in, rather than the degree. Sorry, that was slightly silly—if the degree is fixed then obviously you aren’t interested in it.

But what you are saying is still basically right—it’s the rank that tells you all about the norm, and in fact the dual norm as well because for quadratic phase functions it turns out that is equal to 1 rather than just at least 1.

So presumably for cubics you do the same thing, except that this time you get a trilinear map. Is that right?

Yes and no. Yes you do get a trilinear map, but now it isn’t obvious what you mean by the word “rank”.

Why not?

Because there are several natural candidates for a definition, and they do not agree. The result is that there is no consensus about what the “right” definition should be for a multilinear map.

What are some of these natural candidates?

Well, let be a trilinear form on an -dimensional vector space. One could define its rank to be , where is the dimension of the space of all such that for every Or one could take it to be the smallest such that it is possible to write as Or one could take it to be the smallest such that we can write as a sum where each is a product of a linear function in one of and a bilinear function in the other two. But actually I like a definition that was proposed in a recent and not yet published (or even arXived, it seems) paper of Gowers and Wolf, and is also implicit in work of Green and Tao. It’s to forget about the algebra and simply define the rank of to be (That’s the definition when the vector space is over There is a similar definition for and also in some other contexts.)

That sounds a little tautological.

I know, but Gowers and Wolf observed that one could prove that this “analytic” rank had a number of useful properties. For example, they showed that if and are -linear, then the rank of is at most times the rank of plus the rank of I can’t quite remember whether that proof worked in low characteristic—I’ll have to check.

So their result is weaker than what you’d get in the bilinear case from algebraic methods?

Yes, that’s odd. I expect it would be possible to improve the factor to by tweaking their argument a bit. But that’s not my main concern here. I just want to point out that this definition exists and is quite useful. Also, and I think I mentioned this in one of our earlier conversations, Green and Tao showed that a low-rank polynomial phase function could be decomposed into phase functions of lower degree. To give a simple example of this, an arbitrary Boolean function that depends on just variables has at most non-zero Fourier coefficients, so in particular a quadratic phase function that depends on at most variables, and therefore has rank at most can be decomposed into at most linear phase functions.

Is there an algebraic interpretation of this notion of rank?

Sort of. In the trilinear case, for instance, is if the linear map is not identically zero and is if it is identically zero. So you could say that the rank is related to the size of the “kernel” of However, the set of all such that for every is not a linear subspace but rather some bilinear structure. And its density does not have to be a power of 2, so the rank of is not necessarily an integer.

I’m not sure why you think that any notion of rank is going to be of any use in proving lower bound results.

Why not?

Because the rank is never going to be more than since if then belongs to the “kernel”. Also, it goes up in a sort of subadditive way. So it seems to me that after a linear number of scrambling operations you’ll have reached something with maximal rank—or at least, it won’t be possible to use just the rank to prove that you haven’t. I think you could perhaps extend what Aaronson and Wigderson said: you have to use more about your polynomials than just their degree and their rank.

That does sound plausible, alas. I wonder if some principle like that follows from their results on algebrization. I feel like asking an expert about this.

Meanwhile, perhaps we should return to the question of what basic operations do to quadratic phase functions?

Oh yes, thanks for reminding me. But I’m afraid I still don’t have any progress to report. But I have had a little thought about it that I’d like to float past you.

OK.

I was a bit worried by your point that it is unlikely that a basic operation applied to a polynomial phase function of high degree will have correlation with another polynomial phase function of the same degree. And I’m also worried by your point about the combination of degree and rank probably not being a refined enough tool for proving complexity lower bounds. But it now occurs to me that it might just conceivably be possible to define a set-valued complexity measure using these sorts of ideas.

Ah, that sounds potentially interesting if you really can do it.

Well, the idea I had was this. If one is trying to prove that a function of low complexity is a linear combination, with coefficients that aren’t too big, of functions that are in some sense structured, such as low-degree polynomials and the like, then perhaps one can argue that the set of functions you use for is not much bigger than the union of the set of functions you use for and the set of functions you use for That’s not quite what I mean because it makes no mention of the coefficients, but you get the idea: if you’ve got a straight-line computation of then perhaps the set of functions you need to express all the functions that occur in the computation as efficient linear combinations is not too vast.

I sort of get the idea, and it sounds interesting in a way, but I worry that much of its interest may derive from its being rather vague and may vanish as soon as you try to express the idea more precisely. Can you give me a precise statement that says something like, “If there exists a set of functions with such and such a property, then there is a function in NP with a superlinear circuit lower bound”? Of course, in itself that’s not enough because some such statements are useless, such as “If there exists a set of functions that contains all functions of circuit complexity at most and does not contain the clique function, then there is a function in NP with a superlinear circuit lower bound.”

Let me tell you the kind of thing I had in mind. If you don’t mind I’ll call the set of functions instead of Now suppose that we have two Boolean functions and and that we can write them as and with the functions and belonging to

What can we then say about ? I was hoping that we would be able to write as where each was either a or a or one of a small number of “error” functions that came in. And also there would have to be some condition about the sum of the not being too big in terms of the sums of the and

I can see a number of problems with this. To start with, if the coefficient sums go up by even a small constant factor each time you do a Boolean operation, then after a linear number of steps they will be exponentially large. But if your set is reasonably balanced, it will be possible to write every Boolean function as a linear combination with coefficients adding up to at most So you won’t have distinguished between functions of linear complexity and random functions.

But even if you can solve this problem (which is conceivable—for instance, there might be parameters other than the sum of the coefficients that could give you a measure of the “size” of the linear combination), there is another that seems to me to be more fundamental. Recall from an earlier conversation of ours that Boolean operations felt more like products than sums. Now what happens if you take the product of and ? You get Now there’s good news and bad news here. The good news is that if and are degree- phase functions then so is so we’ve got another linear combination of functions in But the bad news is that we have used all the functions (and we have multiplied together the coefficient sums, but we’re ignoring that aspect now). So it seems that the set of functions we use for is going to be something like the product set of the sets we use for and for Therefore, after linearly many steps of the process it seems that we will end up with exponentially large sets. But no function needs more than exponentially many functions from , since the dimension of the space we are talking about is

As ever, the water you pour over my ideas is well and truly cold. But there’s one point you made that I think needs to be thought about slightly harder.

As ever, you carry a little immersion heater around with you.

Something like that. What I wanted to say was that at a certain point you made the assumption that the cardinality of the set of functions was equal to the product of the number of and the number of Obviously this is an upper bound, but perhaps there are coincidences. For example, if is the set of all quadratic phase functions, and and , then so if and are the sets of quadratics used by and then the set of quadratics used by is the sumset

Now in general can have size but if the sets and have additive structure (an extreme example would be when they are parallel affine subspaces of the space of quadratics over ) then can be much smaller.

OK, but now you’ve introduced another idea into the picture. Your hypothesis would be not so much that you can write low-complexity functions as a linear combination of few structured functions, and more that you can write them as a linear combination of structured sets of structured functions. But where is your set-theoretic complexity measure now?

I’ve thought of a question that might be worth considering. It just feels as though it could clarify the discussion a bit.

OK, what is it?

Well, it seems to me that, motivated by the remarks of Aaronson and Wigderson, you are trying to distinguish between low-complexity and high-complexity polynomials, even when they have the same degree. This seems worth thinking about even for quadratics: a simple counting argument tells us that most quadratics have superlinear (in fact, almost quadratic) circuit complexity. So what is the difference between a high-complexity quadratic and a low-complexity quadratic? If you force yourself to think about just this question, then maybe you won’t keep coming up with properties that fail for known reasons.

That sounds like an excellent idea, but potentially also a very difficult one, because it might be possible to define “weird” quadratics by devising some clever low-complexity calculation that involves functions that are nothing like quadratics but that magically produce a quadratic at the end. But perhaps one could have a restricted model of computation that allowed more gates but insisted that all functions at every stage were quadratics.

That is sounding very like arithmetic complexity to me.

What is arithmetic complexity?

It’s very like circuit complexity, but you replace the Boolean operations and by arithmetic operations such as and I think a good illustration of the basic idea is the function This has arithmetic complexity because you can get it in steps by starting with and repeatedly squaring. (I’m now talking about arithmetic complexity of polynomials over the reals, by the way.) This shows that the arithmetic complexity of a polynomial can be far smaller than the degree. And in general it seems to be hard to prove lower bounds for arithmetic complexity.

A very important fact in cryptography is that the arithmetic complexity of raising a number to a power modulo a large index is small. If you want to work out mod then you can do the repeated-squaring trick to work out for all and then multiply together the ones you need to make This shows that calculating has arithmetic complexity that is polylogarithmic in Equally important is the fact that it seems to be much harder to calculate factorials: if there were a quick way of calculating , for example, then there would be a quick primality test. (Of course, one could say that there is a quick way: use AKS to test whether is prime and then the answer is if it is prime and otherwise. But being able to calculate factorials and similar polynomials quickly would have major consequences besides primality testing.)

But I don’t see how this applies to quadratics, because they are not closed under taking products.

I agree that that’s a problem. But here’s the type of thing one could do. You’ll agree that the map is a high-rank quadratic.

Yes. I think it has rank either or

And it’s also computable in linear time, has linear arithmetic complexity, etc.

Yes, but its associated matrix is very sparse, since it has only entries.

Sorry, but why rather than ?

Because the bilinear form you get is symmetric: corresponding to each term you get a term in the bilinear form.

Ah, I see.

The point I wanted to make is that you can produce less sparse matrices by producing some more interesting ensembles of linear forms than What you do is start with a straight-line computation that allows only addition mod 2. In other words, you write a sequence where each is a sum of two earlier terms in the sequence (either s or s). Then at the end of that process you create a quadratic form where each and each is one of the linear forms from the nice little supply that you have built up.

Ah, I like that. First of all, it really does seem a natural model for the computational complexity of quadratic forms over OK, it doesn’t allow for computations that produce quadratic forms by magic, but it does seem to allow all “natural” ways that one might want to produce them. The second thing I like about it is that it relates very closely to the Gaussian-elimination problem. In fact, the only difference is that in that problem when you work out the mod-2 sum of two of your functions, you have to choose one of them and vow never to use it again, whereas here we drop that restriction. It shows that what makes a quadratic computationally simple is the possibility of writing it as a sum of rank-one quadratics (by which I mean functions of the form where both and are linear) that are somehow related to each other in a “simple” way. We see that the rank is far too crude a parameter because it takes no account of this relationship (or lack of it).

Quick question: to what extent is the representation of a quadratic unique? In particular, might it have a “simple” representation and a “complex” one? Then in order to prove that a quadratic could not be computed in linear time (in this model) one would have to consider all possible representations rather than just one, which would be annoying.

I think uniqueness of any kind is too much to hope for. For example, if you work mod 3 instead and choose an arbitrary “orthonormal basis” (by which I mean a collection of binary sequences such that mod 3 for every ), then the matrix works out to be the identity (as can be seen if you multiply it by any of the vectors ). The resulting quadratic form is . There are annoying characteristic-2 problems if you want to do something similar over since then but I suspect it would be easy to come up with an example that avoids the diagonal and makes the same general point.

Sorry, I got lost there—what is the general point?

It’s that one can probably find “weird” ensembles of linear forms that give you quadratics that can also be produced in non-weird ways. But now it’s my turn to be lost. What was the point you were making?

I haven’t quite got it clear myself. But I liked this connection between the complexity of quadratic forms and the Gaussian-elimination problem and was trying to see how closely they were related. It seems that what you do here is a slightly generalized Gaussian-elimination procedure to produce two matrices (one with rows and one with rows ) and you then multiply the first by the transpose of the second to get the matrix of the quadratic form. So we are not in fact asking whether the matrix of the quadratic form can be obtained in linear time using Gaussian row operations, or anything like that at all. And one can express the matrix as a product of two other matrices in many different ways, so it looks as though it will be pretty difficult to identify some quasirandomness property that forces all decompositions of the form to be complex in the Gaussian-elimination sense.

I suppose what I’m saying is that even if we could solve the Gaussian-elimination problem, we’d still have a long way to go.

I think that you are right, which is unfortunate.

Thanks.

I don’t mean your correctness is unfortunate, but just that the correct fact you’ve identified (if it is correct) is a pity. But I still like your proposal that we should look at quadratics, because it still seems to be subject to the natural-proofs barrier, and we won’t be tempted to use crude parameters such as degree and rank because all our polynomials are quadratics and they can quite easily have maximal rank.

Yes, but now you haven’t the faintest idea how to prove anything.

That’s better than having lots of faint ideas that are guaranteed not to work. And in any case, now, despite what has pointed out, I want to think about the Gaussian-elimination problem. I have a simple question: how many Gaussian operations do you need in order to produce a quasirandom matrix?

What do you mean by that?

I mean a matrix where the number of 1s is approximately and for almost any two rows they agree in approximately half the places. (Actually, the second property implies the first.)

That sounds like a very natural property to me.

Thank you.

I didn’t mean it as a compliment. It sounds easily computable, and hence very unlikely to be a good indication of complexity. I’m sure it will be possible to produce such matrices in a linear number of steps.

Oh yes. Well, perhaps we should just confirm that.

A random sequence of row operations doesn’t work I think, because each row ends up depending on only very few of the original rows. To put it another way, the matrix you end up with is far too sparse. But one can deal with that by first forming a triangular matrix with 0s on one side of the diagonal and 1s on the other side—it’s easy to see how to do that. Perhaps if one then applies a random sequence of Gaussian operations one reaches a quasirandom matrix in linear time.

Hmm, I’m not sure. For a start there will be a few rows that you will never touch again.

Yes, but all I need is for almost all rows to look random at the end of the process.

I think you need to be fussier than that. After all, the Paley matrix is very quasirandom according to your definition, so if you can’t produce extremely good quasirandomness in linear time then the Walsh matrix needs a superlinear number of row operations.

That was indeed an example I had in mind actually.

What’s the Walsh matrix?

A quick definition is this: If is the matrix with just a 1 inside, then you’ll end up with a orthogonal matrix of s and s. If you change that to a 01-matrix in the obvious way then you get what one might call perfect quasirandomness, apart from the fact that one row is all 1s.

Right, how do you get that level of quasirandomness with linearly many row operations?

Hmm, we have an inductive construction, so presumably we can translate that into some inductively defined sequence of row operations. How do we produce from the identity matrix? That’s quite easy: add the second row to the first to get , then the first to the second to get

In general, if we’ve got the matrix , we can add the second “block row” to the first and then the first to the second. That takes operations. So if it takes operations to produce then we seem to have a recurrence like That recursion is satisfied by the function If we now set we get a bound of rather than a linear bound.

It would be very surprising if that was not the quickest way of producing It just seems so obvious and natural. Is it really true that the best known lower bounds for this problem are only linear?

Maybe we should check. There’s obviously lots to think about here, but I’m getting tired. Shall we stop for now?

OK.

In this next instalment, our characters discuss the norm you get by looking for the best possible correlation with a quadratic phase function. They end up discussing a heuristic argument that might, just conceivably, show that this norm is one of a wide class of norms that cannot possibly give rise to superlinear lower bounds. Along the way they have several thoughts, some of which are quite interesting, some not interesting at all, and some plain wrong. (The more interesting ones are mostly later on in the instalment.)

*******************************

Last time we met, I came to the understanding that if you build a norm by means of a formula of the form then there are two properties that might have that will give the norm an outside chance of being a useful quasirandomness norm for proving nontrivial lower bounds. The first is that the cardinality of should be superpolynomial in (or else there is a trivial polynomial-time algorithm for working out the norm). The second, which implies the first, but which I prefer to think of as a separate property, is there should not be some clever polynomial-time way of working out the norm—which, when has superexponential size would require one to exploit special properties of the particular set of functions.

As I see it, if you take to be the set of all quadratic phase functions, then you get a set that definitely has the first property and could well have the second. So I want to go back to thinking about this quadratic-correlation norm. Earlier I convinced myself that a random low-complexity function should not correlate with any quadratic phase function. But if for any fixed quadratic phase function I can get only an exponentially small probability of a huge correlation, and if there are superexponentially many quadratic phase functions, then perhaps we need to revisit this statement. Is it conceivable that every function of linear circuit complexity correlates quite heavily with a quadratic phase function?

Why should that happen?

I have absolutely no idea. In fact, I don’t think it does happen. But I’d like a convincing (even if nonrigorous) argument to that effect.

Why do you think it doesn’t happen?

Basically because there seems to be no reason for it to happen. But here’s what happens when I try to argue that it doesn’t. A first thought is to take a cubic polynomial. But I now see that that gives us two problems: how do you find a suitable cubic that you can calculate in linear time, and how do you prove a bound that’s better than ? The second question may have an answer but I’d be surprised if people can get down to .

Yes, but you said you just wanted a heuristic argument: is there perhaps a plausible guess for what the correct bounds should be for how little cubics can correlate with quadratics (which is just the same as asking how small the average of a cubic phase function can be)? Or a similar question: if you take a random set of triples and define a cubic polynomial to be then what is the expected size of ?

I’m afraid I don’t know the answers to these interesting questions. But let me point out that if you want to take a random cubic like that, then you’ll need to take the maximum over all quadratic correlations. That is, you’ll need to prove that the cubic phase function has some property that causes it not to correlate with quadratic phase functions. And I don’t know of a property that gives really good bounds for that.

So let me turn to the other suggestion, which is to take a random function that’s computable in linear time.

You mean with that clever pre-processing step?

Ah—as it happens, I’ve thought of a different model that seems better. It doesn’t use the pre-processing step and it doesn’t involve the worryingly low depth that was complaining about.

What you do is this. Let’s call the basic functions and their negatives level-0 functions. First create from the basic functions a collection of level-1 functions by picking random pairs of level-0 functions and randomly applying or to them. Here, is a constant less than 1: I like to think of it as 0.99. From these, create level-2 functions, again by picking random pairs and randomly applying either or Continue in this way, until you’re down to just one function. The total number of functions you create is linear in (since it is the sum of a GP that starts with and has common ratio ) but the depth of the circuit is all the coordinates make a big difference, and so on.

Surely with high probability at least some of the coordinates will be missed out when you go from level 0 to level 1.

Oh yes. Well, one could deal with that by just making sure that all the functions are used at least once when you go from one layer to the next. Incidentally, one could let the layers increase in size for a short while to begin with before then decreasing geometrically.

You’re right, I prefer this model to your previous one, but I have a semi-serious objection to it. By that I mean that it’s a serious objection but I think there will be a way of dealing with it.

It’s good to know that in advance. What is the objection though?

It’s that your final function is obviously not going to be pseudorandom.

Why not?

Because if your model creates pseudorandom functions, then presumably the two functions you get in the penultimate layer will be pseudorandom, and one would hope that they would also be “independent” in some sense (since if not then applying a Boolean operation to them could produce a highly non-random function). But if your final operation is say, then applying it to two independent pseudorandom functions gives you a function that takes the value 1 about 1/4 of the time instead of about 1/2 the time. This problem also applies to your earlier model of a random formula. I have to admit that I didn’t notice it myself, but was told about it by Luca Trevisan. He suggested using the basic operations from the Gowers model instead.

Well, that shouldn’t create too many problems, since we were using them in our previous model of random functions of linear circuit complexity. Let me have a go at defining something.

We start with a string of bits. We then apply a random sequence of basic operations (that is, permutations that act on just three bits at a time, as before). Then we randomly restrict to a set of bits of size And then we repeat.

In fact, I’d rather do basic operations before randomly restricting to a set of size Then we can do basic operations on just those bits before randomly restricting further, to a set of size bits. And so on until we get down to a constant number of bits, at which point we could just choose one.

Does that really avoid the problem that Luca Trevisan pointed out?

Yes. The big difference between this model and the previous one can be summed up as follows: the basic operations take random functions to random functions. If and are random functions, then will not be random, but if is a random even permutation of then the composition of with one of the basic 3-bit permutations will also be a random even permutation.

If you like that model, then you’ll agree that the following question is quite a good formulation of what I was asking just now: if you generate a function randomly in that way, will it necessarily correlate significantly with a quadratic phase function?

I can see that it might be hard to prove that the answer is no, but don’t you at least have a heuristic argument that the answer should be no?

Can I chip in with what is either rubbish or an amusing observation?

Of course.

Well, if you managed to prove that every random linear-complexity function correlated significantly with some quadratic, then, as you point out, you’d have a property that distinguishes between linear-complexity functions and purely random functions. Now Razborov and Rudich tell us that such a property cannot be computable in polynomial time. However, the property in question certainly belongs to NP (since once you know which quadratic phase function you’re supposed to correlate with, it is easy to check that you really do). So in order to get this approach to work, wouldn’t you have to prove that PNP?

If that argument is correct, then it’s weird. It shows that an attempt to obtain merely superlinear lower bounds would prove superpolynomial bounds for a different problem. I can’t get my head round this.

But I thought you said that Razborov and Rudich’s result was conditional on the existence of a strong hypothesis in complexity.

Ah yes. The argument I gave would rely on the existence of a certain sort of strong pseudorandom generator, which exists if factoring is hard. So one would have a proof that either factoring is easy or PNP. But that’s trivial: if P=NP then factoring is easy.

I’m wondering if there is a non-trivial statement one can make here. What your argument would show is that either factoring is easy or the quadratic correlation problem—let me call it QC—cannot be solved in polynomial time. But I don’t see any reason to suppose that QC is NP-complete, so although the intractability of QC would imply that PNP, the converse is far from obvious. So we’d have a proof that the hardness of factoring implies the hardness of QC. Or to put it another way, if we can solve QC then we can use that to factor quickly. That doesn’t seem trivial to me.

Of course, all this is so utterly conditional on statements we don’t know how to prove and don’t even necessarily believe that it is not all that exciting.

Which statements?

Well, it depends on a proof that linear-complexity functions all correlate with at least one quadratic phase function, which I don’t have any reason to believe, and it then depends on finding a clever algorithm for QC, which I am at best neutral about. (An argument in favour of such an algorithm existing is that it exists for linear correlations—take the Fourier transform—and there are at least some quadratic analogues of linear Fourier analysis. But those analogues are far far weaker than what would be needed, and for correlations as small as the ones that would interest us it is not clear that anything can be said at all.)

You say that you don’t have any reason to believe that every linear-complexity function correlates substantially with at least one quadratic phase function. But so far you haven’t given us any reason to believe the opposite either.

I know. So far all I have to say in that direction is that a surprising event like a quadratic correlation ought to happen, if it happens, for a reason. And I don’t see a reason. It’s a bit like most people’s belief that is normal because there is no earthly reason to suppose that the digits of would do anything special.

But for that analogy to work, you need to convince me that correlating a bit more than a random function would with just one out of the quadratic phase functions counts as “doing something special”. Perhaps it’s just what one would expect.

I think I see what you’re getting at, and it’s similar to the discussion we were having earlier. There are functions of linear circuit complexity, so no probabilistic statements we make can hope to be accurate to better than Furthermore, we actually expect them to be accurate to no better than If we believe strongly enough that detecting small quadratic correlations cannot be done in polynomial time, then we might argue that there is basically nothing we can say about the probability of such correlations occurring: the probability would be much smaller than exponential for a purely random function, but for pseudorandom functions we cannot say anything has a subexponential probability, so it could be that every linear-complexity function correlates with some quadratic phase function.

But if that were the case, how on earth might it be proved? One approach might be to show that each time you do a basic 3-bit operation, in some sense you multiply by a constant the number of quadratic phase functions that you no longer correlate with. Since there are quadratic phase functions, it might even take quadratically long to end up correlating with none of them.

That sounds worryingly optimistic.

I agree, so let me try to think about what I even meant in that heuristic argument, which as it stands is nonsense of course.

Let be a random sequence of functions, each produced from the previous one by using a random 3-bit operation (with occasional random restrictions to a fraction of the bits, as above). Let us also fix a Boolean function My guess is that the correlation of with will behave a bit like a random walk with drift: that is, it will multiply or divide by a constant amount at each step, but the expectation of the log of the factor is negative rather than zero.

Now if one could say that all these random walks (for different quadratic phase functions ) behaved reasonably independently of one another, then one might guess that after steps, an exponentially small fraction of them had done stupid things like never really getting any smaller. But that “reasonable independence” assumption looks decidedly shaky.

I think we ought to separate out two ideas here. The first is an attempt to prove superlinear circuit lower bounds by first showing that every function of linear complexity correlates heavily with some quadratic phase function. Of course, then one would have to find a function that did not correlate heavily with any quadratic phase function, which could be hard. Indeed, it might require one to improve on the result of Viola and Wigderson.

But for the discussion we have just been having about the hardness of QC, we didn’t need every function of linear complexity to correlate with a quadratic phase function. All we needed was for a substantial fraction of them to do so. After all, virtually no random functions do, so that would give us a way of distinguishing pseudorandom from random, which in turn would prove that either QC is hard or factoring is easy.

I’m slightly troubled by that last step. Wouldn’t one need to use Razborov and Rudich’s model of random low-complexity functions if one wanted to say anything about factoring?

Oh dear, you’re right. I think the conclusion one would reach is not interesting after all. So perhaps one would indeed need to prove that every function of linear complexity had a large quadratic correlation.

I think it is nevertheless helpful to separate out the two problems. I’d be very interested in a proof that almost every function of linear circuit complexity correlated quite well with at least one quadratic phase function. It would at least show that there are some differences between random low-complexity functions and genuinely random functions. And perhaps it would give a clue about how to change “almost every” to “every”.

Unless it was true for “almost every” but false for “every”.

Fine, but we’d still have the random/pseudorandom distinction, which would be pretty nice I think.

I’d like to give a possible heuristic argument that goes in the opposite direction to yours.

OK, that sounds interesting.

Well, I was convinced by your random-walk-with-drift idea when it came to individual correlations, but I think you are right to be suspicious of the independence.

Do you have some specific reason for that?

Yes. In fact, when I think about it a bit more I realize even the random-walk-with-drift idea could be wrong. I think what was in the back of your mind was that when you apply a basic 3-bit operation, it will usually chop things up, but it might put back together again something that you have already chopped up.

Something like that. In an extreme case it might even invert the previous basic 3-bit operation.

Right, but that happens with probability and it seems to me that in fact the tendency to get more complicated is very strong, and the “stupid” examples will occur with probability rather than And they will be very stupid examples that do things like inverting previous operations a lot.

If that is correct, then perhaps one could prove that every composition of basic 3-bit operations that obeys a few simple rules, such as not permuting the same set of bits too many times in a row (which would allow cancellation to take place), will end up failing to correlate with independently of which quadratic phase function one has chosen. Perhaps it just becomes “less quadratic”.

You’re beginning to convince me, or at the very least to weaken my already weak belief that this quadratic correlation idea could work.

But if your heuristic argument is correct, can we turn it into a much more general barrier? After all, it seems to be defeating a proof that is not obviously naturalizable.

I think you’re jumping ahead a little fast. To get my counterargument to work one would have to understand what it was about quadratic phase functions that caused “sensible” products of basic 3-bit operations to correlate less and less with them. Only then could one think about whether other classes of functions might have a similar property. But I agree that it would be interesting if we could identify a class of NP properties that were unlikely to be usable in any proof that PNP.

Before you dive into that, could you explain more clearly what you mean by a “sensible” product?

I can try. What I’m trying to formulate is a sort of non-triviality condition on products of basic 3-bit operations. For technical reasons I’m going to restrict my attention to just one 3-bit operation, which works like this. You choose an ordered triple of distinct elements of and then given an -bit sequence you look at the triple If this equals then you change it to and if it equals you change it to Otherwise, you leave it alone. For example, if and then the sequence maps to while the sequence maps to itself.

I haven’t checked, but I’m fairly sure that these operations are enough to generate all even permutations of and if they do then they will do so in a rapidly mixing way.

They clearly don’t generate all even permutations of

Oh, really? Why not?

Because the all-1s and all-0s sequences are fixed points of all those transformations.

Oh yes, of course. Well, even so I’m pretty sure they generate a very large group—possibly even all even permutations that fix those two sequences. Let’s not worry about that too much for now.

Instead, let us ask ourselves how cancellation can occur when we take a product of these generators. One way is trivial: if cancellation would have occurred even in the free group, then it occurs here. In other words, this is the cancellation you get just by removing inverse pairs. But there is also a less trivial form of cancellation that follows from the fact that the group is finite: we know that if we take all products of generators, then when is large enough we must have a lot of equalities. So we can take two products of generators that give you the same permutation and multiply one by the inverse of the other. In that way you get very random looking products of generators that miraculously cancel. However, the miracle depends on the products being very long indeed. To be more precise about it, the number of generators is and the number of even permuations of is so to start getting lots of coincidences you need to multiply at least generators together.

However, this observation leads to an intermediate way of obtaining cancellation, which is to take a smallish integer and look at products of generators that act on some fixed set of bits. Now if you want to get coincidences, you only need to multiply generators together.

Right, but what’s the point of these observations?

I’m coming to that. One might make the bold conjecture that the cancellations I have just talked about are in some sense the only ones that can occur. And in fact the first type (cancelling inverse pairs) is the special case of the second in which you take

Loosely, one could state the conjecture as follows: the only way of getting the identity as a product of these 3-bit operations is by means of cancellations that arise when you stick around in some fixed set of bits for a long time (relative to the size of that set of bits).

Turning that round, the statement would be that as long as you spread the operations around enough then you are guaranteed to produce a non-trivial element of the group And one could also propose a stronger heuristic principle that the function you get is maximally complex, in a sense that I haven’t worked out how to make precise. But a consequence would be that, for example, if is a product of 3-bit operations that obeys the non-triviality condition, then the correlation of the first bit of with any fixed quadratic phase function decays exponentially.

Don’t you have the problem that the first bit of might not depend on all that many of the basic operations you use?

Yes. I think what I mean is something like that the decay is exponential in the number of basic operations that actually make some contribution to the answer. I’ll have to think how to formulate that properly.

Before I do that, I want to make an important remark, which is that two basic 3-bit operations that act on disjoint sets of bits commute with each other. So the “trivial” cancellations I’m talking about don’t necessarily happen between consecutive terms in the product.

Here’s a way I like to visualize it. Let’s imagine we have an array of dots, one for each bit. We now start applying basic operations. Associated with each operation is a subset of of size 3. Let be the sequence of all these subsets, and form a graph inductively as follows. The vertices of the graph are ordered pairs of integers. You start with the graph which consists of isolated vertices (These are the dots I just mentioned.) Then if you add in vertices and joining all three of them to all three of and . That gives you the graph And you then continue this process. If you’ve got and (not the same ), then you add new vertices and choosing for and the smallest integers that have not yet been used (so will be the number of such that ) and joining all of and to all of and Let me also imagine that these edges are directed downwards. And let me say that the index of a vertex is

Given any set of vertices I can follow the paths down from that set. I call the graph non-trivial if I can’t find some small set of vertices such that when I follow the paths downwards I go a very long way without involving any new indices. And if the graph is non-trivial, then I say that the product of generators was non-trivial.

One can look at this as a conjecture that there is a sort of pseudo group homomorphism. Whenever I do have a bunch of generators that sticks around in the same place for too long (possibly after commuting them with other generators to get them together) then I’m allowed to cancel them. So I’m imagining a non-existent group where the generators are sets of size 3 and the relations are that disjoint sets of size 3 commute and all (rather than just some) large bunches of sets of size 3 that are subsets of a fixed set of size cancel.

Am I right in thinking that you’re introducing these thoughts in order to try to come up with some kind of heuristic principle that would suggest that norms like the quadratic correlation norm will not help us?

Yes, I was going to return to that, though I’m not sure I know how to do it.

Well while you’re thinking about it, let me float another idea, which is that if what you say is correct, then perhaps it could be used to prove lower bounds.

Eh? I was trying to use it to show that it was difficult to prove lower bounds.

Yes, but if what you say is true, then it seems to establish a very strong principle that says that you can’t compute the same function in two essentially different ways. So perhaps one could prove a lower bound by finding an NP function and a method of computing it that didn’t “collapse” when you “mapped it to your imaginary pseudogroup”.

Oh. I see what you mean. Of course, I was talking about reversible computations, but I don’t know whether I was relying on that. Perhaps I could have just added some extra “rubbish” bits and argued in more or less the same way.

I’m inclined to take what you say not as a promising avenue for proving complexity lower bounds, but rather as a strong indication that my “bold conjecture” is far too bold. After all, it would seem that it is often possible to compute the same function in radically different ways. For example, to work out mod I could start with cube it times and then square that times, or I could do the squaring first and then the cubing. That looks like two genuinely different ways of working out the function.

It now looks to me as though the word problem in with 3-bit operations as generators, is going to be fantastically difficult. Of course, nobody would expect it to be possible to determine in polynomial time whether a given word gave you the identity, but I was hoping for something much less: finding a tidy sufficient condition for a word not to be the identity. But it seems that even this is far too much to ask.

Are you sure? Wouldn’t it be the case that in order to do sensible computations like raising to the power mod you have to do quite a lot of “sticking around in the same small set of bits”? Perhaps your conjecture is true, and our impression that there are often genuinely different ways of computing the same function is an illusion, because we don’t know of any “non-collapsing” computation techniques. Maybe if you’re very strict about spreading the sets about, then you’ll only ever get very random-looking functions.

I see your point. I now don’t know what to believe.

Neither do I but I think it’s a nice question. Is there some sense in which computation takes place in an “almost free” group? Roughly speaking, this should mean that two circuits cannot give the same function unless they operate on similar sets of bits in a similar order (in so far as the order matters).

I think I have an argument against that.

Oh really? What is it?

It’s that if you take two unsatisfiable bunches of clauses you could create a function that’s identically zero in two different ways. And you could choose those sets of clauses randomly—if you have a reasonable number of them then they will give you unsatisfiable formulae.

Oh yes, I was a little careless there. But I think the question still makes sense if we look at basic 3-bit operations rather than AND, OR and NOT gates and consider whether two essentially different products can be equal.

But I thought you said you could simulate AND, OR and NOT gates using basic 3-bit operations.

You can, but in order to do so you have to introduce extra “rubbish” bits that start out as 0s and end up scrambled in some peculiar way that depends on the computation you carried out. So with your example of two different ways of calculating a function that’s identically zero, the difference would show up in the rubbish bits.

I see.

So I think the bold conjecture, if we can really call it a conjecture, is still alive, but one has to understand that it is very much a conjecture about reversible computations. Unfortunately, I don’t know enough about combinatorial/geometric group theory to know how to tackle such a problem: that is, a problem where one is trying to prove that a word is non-trivial, using certain features of how the word is put together. Does anyone know of any results of that kind?

I don’t.

Neither do I, I’m afraid.

In that case, can I float past you two more ideas that have occurred to me?

Of course.

Well, you wanted to prove that “sensible” products of basic 3-bit operations would give you functions that became “less and less quadratic”. I interpret that as follows. If is a “sensible” product of basic operations and we write then the correlation of with any quadratic decreases exponentially with until it becomes almost minimal.

I don’t know that I actually believe that, but it’s roughly the hypothesis that I was entertaining in order to cast doubt on your argument that functions of linear circuit complexity might magically correlate with quadratic phase functions.

The first thought is that it might follow from the “bold conjecture”. I’m going to oversimplify a few details here, but the bold conjecture has as a consequence that no “sensible” product of basic transformations can ever compute a function that can be computed by a “collapsing” product. Since the only computations we can think of tend to involve sticking around for a long time dealing with small sets of bits (though perhaps I am wrong about this — for example, does the PCP theorem give rise to “weird” computations that can be described by means of “sensible” products?), that would imply that no “sensible” product can ever give us any of the functions that we know and love.

At least if they can be computed reversibly, which is a big restriction.

Yes, you’re right. But it suggests that if we could find a reversible way of creating a quadratic polynomial, then maybe it would be reasonable to conjecture that a “sensible” product of basic transformations would automatically not correlate with it.

Aren’t you confusing permutations of the cube with Boolean functions defined on the cube?

Yes, but that was deliberate — it was the detail I was leaving out. I don’t know the best way of formulating a conjecture along these lines, but maybe if one could reversibly create a permutation that was quadratic in each coordinate, or something like that, then one could get the argument to work.

The big problem is that proving the general principle that the 3-bit operations create a group that is in some sense “free apart from the obvious kinds of relations” looks pretty hard. But my other thought was that we could at least have a look at what happens in the linear case, because I think it is reasonably straightforward to see what a basic 3-bit operation does to the Fourier transform of a function.

How do you mean?

Well, a basic 3-bit operation is a permutation of the cube but we can also think of it as a linear map that takes functions defined on the cube to functions defined on the cube. We can then change basis from the delta functions at each vertex to the Walsh functions. That is, we see what happens to the Fourier transform.

I think I’d like to see this in a bit more detail.

All right. Let stand for the function where I have written for the parity of in . (In other words, it’s 1 if has an odd number of 1s in and if it has an even number of 1s.) Now let’s take as an example the basic transformation that acts on the bits and and interchanges and , leaving fixed if it is not the case that The way that acts as a linear map is via the formula which equals since is its own inverse.

What is the expansion of in terms of Walsh functions? By definition it is which equals

where we are setting if and otherwise, and similarly for and

We can split the product up according to whether or not To simplify the expression, let us take (without loss of generality) to be If we do all this, then we obtain the expression

Now we can think of as independent random variables, and if we do then the expectation of the product above can be rewritten as a product of expectations. But unless , in which case it is 1. Therefore, the entire expression is zero unless for every This tells us that for every that does not equal outside the set (More generally, this will be true for for the same reason.) Another observation is that if is even, then replacing by does not change the parity of the restriction to so in that case for every which tells us that If, however, is odd, then a small calculation shows that is for four of the eight sets that agree with outside and for the other four.

Wow, that was more detail than I was expecting.

Well if you want to skip the calculations, just think about the conclusion. What we find is that the effect of on a Walsh function is either to leave it unchanged or to replace it by a linear combination of four Walsh functions , where the sets are close to

I simply have to make a remark here. What you have is an orthogonal map that takes the space of all linear combinations of Walsh functions, which we can identify with functions defined on , to itself. Moreover, it’s a particularly simple orthogonal map in that it takes each Walsh function to a linear combination of Walsh functions that correspond to sets that differ only in the places and If you identify with the sequence that is in and otherwise, then what this is saying is that each sequence is mapped to a linear combination of sequences that differ from it only in the coordinates and But this is precisely the sort of thing that one does in a quantum computation!

I thought quantum computations involved complex numbers.

They do, but it’s hardly surprising that we don’t get arbitrary unitary maps, since the permutations of the cube give rise to a very small subgroup of the unitary group. It’s just amusing that the Fourier transform of a basic transformation is one of the basic transformations of quantum computation.

Are you suggesting that quantum computation could be useful to us?

No I’m not. It was just meant as an amusing observation. Sorry if I’ve got you both excited.

Don’t apologize — I like it too. But let me get back to what I was saying. Once we see that Walsh functions have a tendency to map to linear combinations of closely related Walsh functions with coefficients equal to or , it becomes rather obvious that if you apply a typical sequence of basic operations to a Walsh function then it will spread out and out, preserving its norm but decreasing its norm. Eventually you’ll start getting superpositions and the rate of decrease will become slower, and harder to analyse, but at least to start with it will be exponentially fast.

I think I see what you mean.

Well, I wasn’t all that precise. But it seems to me that there might be some hope of proving that the decay happens not just for random products of basic 3-bit operations but even for 8)’ s “sensible” products.

You know, I think you’ve influenced my way of thinking.

Delighted to hear it, but in what way?

Well, very early on in our conversations, you pointed out that even a heuristic argument can have an influence on what proof methods one regards as potentially fruitful. And here, even if we haven’t the faintest idea how to prove the “bold conjecture”, it does seem to rule out certain methods for proving circuit lower bounds. Suppose, for example, that some ludicrously strong version of the principle held, and let be some class of nice functions (such as quadratic phase functions). We might attempt to prove lower bounds by showing that every low-complexity function correlates with some And this could be a highly non-natural property if has superexponential size, as it does with quadratics. Nevertheless, it could also be the case that any “sensible” product of basic 3-bit operations would have tiny correlation with every function in The key point here is that we go for “every” rather than “almost every”. To get that, we are arguing that when we pick a random product of 3-bit operations, the events “ correlates with ” for could be strongly dependent. This would be the case if they always held when the product had certain properties, such as being “sensible”.

What exactly are you claiming here?

I’m pointing out something very weak, in the sense that it’s supposition piled on supposition. But I think it’s at least possible that there is a very wide class of NP functions that cannot be used to prove circuit lower bounds. In fact, I’m tempted to say that there should be two sorts of NP functions: useless ones, such as “has low circuit complexity”, that lead to trivial reformulations of the problem, and potentially useful ones, such as “correlates with a quadratic”, that don’t work. I’d like to understand this distinction a little better, and then maybe I could formulate a very depressing conjecture that would kill off all ideas you have had so far and all that you are ever likely to have.

Oh. Well I have a confession to make.

What on earth could that be?

I did in fact have another idea for how to prove superlinear lower bounds. It’s still highly speculative.

You mean you don’t have a fully worked out argument? Wonders never cease.

Ha ha. Just let me try it out on you. I’ve just argued that if you take a linear phase function, a.k.a. a Walsh function, and apply to it a basic 3-bit transformation, then the result lies in twice the convex hull of some other linear phase functions. That means that the decay that we observed in the norm could not be too fast.

Now suppose we were able to prove that if you apply a basic transformation to a quadratic phase function then the result could be written more efficiently, in some sense, as a combination of other quadratic phase functions. Then perhaps this would prove that the maximum correlation with a quadratic phase function went down more slowly than it does in the linear case. The decrease would probably still be exponential, but perhaps the rate would be smaller.

If we could establish that, then perhaps we would be able to go on to show that if we considered polynomials of unbounded degree, then we could prove that the maximum correlation went down more slowly than exponential. But there still wouldn’t be too many of these polynomial phase functions, so a random function would have very small correlation with them. This might give us a distinction between functions of linear complexity and random functions.

Well, I’ve listened politely to what you say, but it doesn’t sound to me as though it would work. If you want to show that the maximum correlation goes down slowly, don’t you have to prove that if is a polynomial phase function and is a basic transformation, then has correlation with another polynomial phase function of the same degree? That doesn’t sound remotely plausible to me.

Hmm, maybe you’re right. But I think there may be something to the basic idea. Perhaps instead of showing that a single polynomial phase function transforms to a function that can be written efficiently as a convex combination, one would prove that if you do a sequence of basic transformations to a polynomial phase function, then you can write the result in some nice way as a linear combination of polynomial phase functions. I’m not quite sure what I’m proposing exactly, but it feels as though there is a method that could in principle prove non-trivial lower bound results.

Why don’t you at least see what a basic 3-bit operation does to a quadratic phase function?

I will, but I think I’d better do it in private and report back at our next meeting.

I don’t see how this would get past my “depressing conjecture”.

How about actually making the conjecture before you say that?

OK, I’ve got things to think about too. Let me have a quick go before we call it a day.

Let’s suppose we have some NP property we are trying to use. If we want to, we can phrase this in the following way: we are going to call a function “simple” if there is a Boolean function that depends on at most linearly many bits such that where is some property that can be computed in polynomial time. We want to contrast properties of two kinds. A good example of the first kind is this: if and only if is a quadratic and is large. And a good example of the second is this: encodes a circuit of size at most that computes .

Both these properties could be thought of as correlation with a function in some class It’s just that in the second case, the correlation has to be so perfect that you actually equal a function in A difference between the two is then that membership of is easy to establish in the quadratic-functions case and hard to establish in the low-complexity case.

What I think I might be heading towards here is the possibility that any NP property of the form “correlates with something in ” for some polynomially-computable property cannot work. Or it might be that, but with a stronger restriction on that said that it could be computed in a “collapsing” way. But that might apply to all properties in P that we could actually understand.

So we might still be able to prove lower bounds if we took a simple class but asked for a more complicated relation to our function than “correlates with”?

I think that would be worth thinking about, yes.

Just before we finish, can I suggest a way that might in principle lead to a more precise notion of “sensible product” or “collapsing product”?

That would be great.

Well, I haven’t actually done it, but perhaps one could come up with a presentation of the alternating group with the following properties. The generators are just the basic 3-bit operations. To each relation one associates a set such that the following properties hold. (They may not be a complete set of all the properties one would need.)

1. A relation associated with the set is a product of 3-bit operations that operate only on bits that belong to the set

2. For each let be the group generated by all the relations for which the associated set is a proper subset of Then is an infinite group, and no relation with index can be written as a short word in the group

3. Basic operations that operate on disjoint sets of bits commute.

The rough idea then would be to prove that if you have a product of basic operations that doesn’t stick around for a long time in the same set then there is no way of shortening it. So one would have the “no unexpected relations” property that you are talking about.

That sounds interesting, but it also sounds pretty hard to come up with a presentation of the kind you are talking about.

I wonder if we could find a combinatorial group theorist who would be able to comment on whether it is remotely feasible.

In an entry entitled “Negatives” in his Modern English Usage, Henry Fowler gave an amusing collection of examples of blunders that had been made with them. (If you follow this link, you have to scroll down a page to find the article I’m talking about.) Unaware of this, though not surprised to see it, I have been making a little collection myself. Since this is supposed to be a maths blog, let me feebly justify posting it by saying that it is a reflection on the fact that (and at one point on the corollary that ).

1. The first example is something I was once told about a landmark I had asked the way to. I was told that it was large, so that at a certain point on the route, “You can’t fail to miss it.” Fortunately, I did fail.

2. On February 25th 2009, David Thompson, a political correspondent from BBC News, wrote: “No-one would deny that David and Samantha Cameron come from anything other than extremely privileged backgrounds.” I find this a very hard example to understand directly. The easiest way to deal with it is to substitute “claim” for “deny” and see what you get. But I could just go ahead and be a counterexample to Thompson’s assertion: I hereby deny that David and Samantha Cameron came from anything other than extremely privileged backgrounds.

3. On May 2nd 2009, at the weigh-in before his fight with Manny Pacquiao, Ricky Hatton declared to his fans, “You will not go undisappointed.” This defeatist attitude is unusual in a boxer, but it was justified: he was knocked down twice in the first round and knocked out completely in the second.

4. In July 2009, Nathan Hauritz, part of the Australian cricket team, was talking about Australia’s prospects for the last two days of the Test match at Lord’s. He said: “None of the boys don’t think we can’t do the job.” As an Englishman, I am delighted to say that his team’s collective pessimism was again justified.

5. When it came to the one-day series a couple of months later, the tables were turned. An English supporter, reflecting on the situation, wrote in to the BBC website and included the sentence, “Doesn’t mean we shouldn’t overlook the weaknesses in this team though.” He gets a prize for a quadruple negative, but I cannot agree with him: moaning about the weaknesses of our national sports teams is one of life’s pleasures if you are English.

6. Finally, I come to a deeply sinister example. I recently learned that there is a website called Conservapedia, which describes itself as “An encyclopaedia with articles written from a conservative viewpoint.” I look forward to their mathematics section when they get round to it, which so far they haven’t: Grigory Perelman, a noted communist, claims to have solved the Poincaré conjecture, but he has REFUSED to publish his results in a conventional journal, in case they are scrutinized objectively and not just by a team of hand-picked liberal academics; the heat equation and the wave equation are examples of the now-discredited category of evolution equations; using a notion of creation and annihilation operators, quantum field theorists are starting to understand what the Bible could have told them all along; etc. etc. Unfortunately, it seems that Conservapedia is not a Wiki-style site — I can’t think why not — so we mathematicians cannot help them build up a decent mathematics section.

Just for fun I thought I’d have a look at what they have to say about Richard Dawkins. Some of their criticisms were only to be expected: he supports evolution, is a noted atheist, and so on. But I was taken aback by one of them: he is insufficiently solid in his support for Hitler. Here is the passage that shows this (at least as the article was on October 13th 2009 — I hope this blog’s portion of cyberspace is sufficiently disconnected from Conservapedia’s that it will remain that way, but if not, it will be interesting to see whether it can still be found in the history of the revisions of the article):

Dawkins’ Comment Regarding Adolf Hitler

When asked in an interview, “If we do not acknowledge some sort of external [standard], what is to prevent us from saying that the Muslim [extremists] aren’t right?”, Dawkins replied, “What’s to prevent us from saying Hitler wasn’t right? I mean, that is a genuinely difficult question. But whatever [defines morality], it’s not the Bible. If it was, we’d be stoning people for breaking the Sabbath.”
The interviewer wrote, regarding the Hitler comment, “I was stupefied. He had readily conceded that his own philosophical position did not offer a rational basis for moral judgments. His intellectual honesty was refreshing, if somewhat disturbing on this point.”

Well, in one sense it is a genuinely difficult question I suppose. HITLER WASN’T RIGHT. HITLER WASN’T RIGHT. HITLER WASN’T RIGHT!! Hmm, nothing seems to be preventing me from saying that, and I say it without apology. But why was Dawkins not offended by the suggestion that something should be preventing him from saying that Muslim extremists aren’t right**? Perhaps, being a somewhat literal-minded scientist, he was having thoughts such as this: “Well, it does at first seem as though there is nothing to prevent my saying that Muslim extremists aren’t right, but I can imagine circumstances under which I would be so prevented. For example, if I was kidnapped by al-Qaeda and held at knifepoint, then it would be reasonable to say that I was effectively prevented from drawing attention to the lack of rightness of their views. Or for a less artificial example, if I was in the middle of a dinner party with some of my super-bright liberal atheist friends, then I might be prevented by embarrassment from saying that Muslim extremists weren’t right: after all, the statement is normally held to be too obvious to be worth saying, so I might be thought to be protesting too much. Yes, on reflection, this is a genuinely difficult question. And similar difficulties apply if I substitute Hitler for Muslim extremists.”

So much for Dawkins. The motives of his interviewer, and of Conservapedia for gleefully reporting the interview, cannot be explained so innocently. The only reasonable explanation for the interviewer being “stupefied” is that he regarded it as manifestly and shockingly wrong to say that Hitler wasn’t right. In other words, the interviewer was not just a neo-Nazi, but so convinced of his/her neo-Nazism that a contrary view was stupefying. And Conservapedia wholeheartedly agrees with this. We live in worrying times.

As a postscript, let me deal with a small technical point concerning the last example. It might seem as though “What is to prevent us from saying that Hitler wasn’t right?” has just the two negatives “prevent” and “not”. So why do I call it a triple negative? The answer is that I am talking morality rather than semantics in this example. Hitler is the embodiment of evil, so we can build up as follows:

(i) Hitler — bad.

(ii) Saying that Hitler was right — bad.

(iii) Saying that Hitler wasn’t right — good.

(iv) Preventing someone from saying that Hitler wasn’t right — bad.

(v) Implying that one ought to be prevented from saying that Hitler wasn’t right — bad.

And therefore,

(v) Conservapedia — bad.

Needless to say, if anyone else has some good examples of triple negatives, I’d be delighted to hear them.

**Most embarrassingly, when I posted this, I wrote “are right” instead of “aren’t right” here. My only consolation is that I noticed before it was pointed out to me.

Another somewhat inconclusive instalment, but it introduces another candidate for a non-natural property that might be of some use. (In a later instalment, a heuristic argument is proposed that would show, if correct, that in fact it is not of any use …)

*****************************************

What’s on the agenda for today?

Well, there are two main things I’d like to discuss. One is our “proof” that a random low-complexity function was indistinguishable from a purely random function if you use anything -ish to do the distinguishing. I have a small problem with the argument, though I wouldn’t go so far as to say that it is wrong.

The second is that we haven’t got round to having a serious discussion of the lessons that can be drawn from the failure of the approach. If it’s true that it doesn’t work, then I think we can come up with a pretty general class of “simplicity properties” that fail for similar reasons. But I’d like to be as precise about that as possible.

Sounds like a good plan to me. Shall we start with your doubts about my argument against your -dual approach?

Yes. I’m talking about the modified one where we ignore the degenerate cases. And for now I’m forgetting about the duality. So the question I want to address is this. Suppose you take a random function of superlinear complexity and look at the probability, for a random subspace of of dimension (which is around ) that the restriction of to has an even number of 1s. (Here I am fixing and letting vary, so this probability is a parameter associated with the function .)

Now let’s just suppose that the subspace in question is the subspace generated by the first basis vectors (that is, the subspace consisting of all sequences such that ). Then if you do random Boolean operations, most of them will not involve any of the first coordinates. Indeed, if you do operations, then the probability that you miss the first coordinates completely is something like , or . And for that to be exponentially small we need to be .

Yes, but you’re now fixing and letting vary.

I know, and for that reason I think this argument may be going nowhere. But let me pursue it a tiny bit further. I’m now trying to think how many Boolean operations are needed if you want to scramble every single subspace that is generated by unit vectors. And it’s not obvious to me that linearly many operations will do the job. If we choose random Boolean operations, then we need around of them before we can be fairly sure that we’ve involved every single coordinate. And if we haven’t involved say, then flipping makes no difference to the value of the function, so is guaranteed to have even parity in any subspace that involves .

But why is it a problem for to have even parity in some subspaces?

That’s something we’d have to think about. Possibly one could go further and argue for some kind of bias. But let me continue with what I was saying.

OK.

We could try starting with a couple of non-random layers in order to ensure in a crude way that we involve every coordinate. But involving every coordinate isn’t enough. For example, suppose we chose random operations …

Can I stop you here?

I suppose so.

It’s just that if all you’re planning to do is look at subspaces of this special type then you’re looking at only subspaces. Even if you allow the fixed coordinates to take arbitrary (fixed) values, then you’re still looking at only subspaces. So it’s hard to see how you will end up saying anything that can’t be checked in polynomial time in .

OK you’re right. I suppose it means that there is some clever pre-scrambling that can be done with linearly many operations. For instance, perhaps there’s a nice function from to such that the image of every sequence has large support. If we applied that function and then did random Boolean operations, we’d probably get all our probabilities decaying exponentially.

You’re asking there for a nice code that can be computed in linear time. I don’t know enough about coding theory to know whether such a thing exists.

A quick Google search reveals that it does. [Apologies -- this file seems to be stored as an image and takes a long time to download.]

OK, I admit it. Any argument that tries to make use of the behaviour of the restriction of to subspaces generated by unit vectors is doomed to failure, and there are two ways of seeing why. The first is to observe, as did, that anything you can say about these restrictions will be checkable in polynomial time. The second is to note that in linear time you can map to in such a way that every vector has an image with lots of 0s and lots of 1s.

I’m sorry, I don’t understand the second of those arguments.

No apologies needed — I was far too vague. But explaining myself will take us to the second topic I wanted to discuss, which is trying to come up with informal, but reasonably precise, descriptions of the kinds of arguments that run up against the natural-proofs barrier. In the light of what we’ve just said, I want to refine the heuristic picture I had earlier that was based on the Gowers model of random computable functions.

I wish you wouldn’t call it the Gowers model. It’s just the obvious model of a random reversible computation.

Why is it obvious? In particular, why do the basic operations depend on three bits rather than say two or four?

That’s easy to answer. It’s because if you base them on two bits then the transformations you get are affine over and therefore are by no means quasirandom. By contrast, if you use three bits then you can generate all even permutations of the cube. So three is chosen because it’s the smallest number that works. A related reason is that if you allow some extra “rubbish” bits then you can do any computation you like. For example, to simulate an AND gate, you take the two bits you want to do AND to, and a third bit that you set to 0. Then on those three bits you do the transposition that swaps and At the end of that transposition, the only way that the third bit can end up as a 1, given that it started as a 0, is if the previous two bits are 11.

Gowers looked at random compositions of basic operations, but his probabilities were affected by the fact that if a pair of sequences differed in just one coordinate, then the probability of doing anything to that coordinate when you did a random basic operation was which meant that the probability of not touching it after steps was around which did not become smaller than until was around

This is actually a genuine phenomenon: for example, if you do a random walk in the discrete cube, then the mixing time is about because before then there will be directions that you have not moved in.

However, we should not be misled into thinking that we can somehow exploit this phenomenon in order to prove complexity lower bounds of , because if we do a bit of preprocessing, by applying a linear-time code first, so that any two sequences now differ in some constant fraction of their bits, then the expected time it takes to separate any two sequences is constant. This means that after only a linear number of operations we can expect to have chopped everything up.

Is it possible for you to say more precisely what you mean by “chopped everything up”?

I can try. Recall what a basic operation does. It takes the discrete cube and divides it into eight equal parts, according to the values taken by three selected coordinates. It then shuffles these parts by translating them. For example, if the basic transformation looks at the first three bits and exchanges 000 with 111, then to every sequence that begins 000 or 111 you add the vector So if and belong to the same part, then they haven’t really been shuffled relative to each other.

I suppose what I’m saying there is that remains the same if and are in the same part. But one can also look at more complicated but very important relationships. For example, for a composition of basic operations to look random, we need it to “break up linearity”. One way we could measure that is to look at the extent to which the equation implies that where is the composition of basic operations.

Now and differ in a constant fraction of places, as do and , and as do and . So with constant probability we choose three coordinates such that the restrictions of and are all different. And if then it follows that the restriction of is different again, and the four restrictions form a 2-dimensional subspace of With positive probability the basic transformation will turn that into a subset of size 4 that is not a subspace, and we will have “broken the linearity” of the quadruple.

At this point I want to jump straight to the vague thought that I hope to make precise. Let be a random Boolean function defined on as follows. You first map to as described, then do a random sequence of basic operations, and finally you look at the first coordinate of the result (or take raised to that power if you want an answer in ). I want to suggest that if is any set of Boolean functions on such that membership of can be computed in polynomial time (in or equivalently in exponential time in ), then the probability that is where tends to zero exponentially quickly.

I now want to think about (i) whether that is actually true and (ii) what the consequences are for complexity proofs if it is true. I feel as though it was something like the above fact that caused us problems earlier.

I don’t really have anything sensible to say about (i). We sort of know from Razborov and Rudich that something like it is probably true, so I think I’ll just concentrate on (ii), because that is what really interests me.

So let’s suppose that S is a set of Boolean functions. I’m looking for properties of S that ensure that it cannot distinguish between random low-complexity functions and purely random functions. And the starting point will be what you told me right at the beginning of our conversation: that if S is a polynomially computable property then it has no chance of working.

Now it follows trivially from this that if S (which we think of as our “simplicity” property) implies some property T that is polynomially computable and does not hold for random functions, then again S cannot work. Why? Because then T would hold for random low-complexity functions and not for purely random functions, while also being polynomially computable. And that takes us back to our starting point.

I should make clear that Razborov and Rudich included this observation in their paper, and indeed gave examples of properties S that had been used in the literature that were not themselves natural but that were “naturalizable” in the sense of implying natural properties T.

Yes, I think you mentioned that earlier. Perhaps I should make clear that I’m not claiming originality for any of this. One way of describing what I’m doing now is to say that I’m trying to describe general classes of properties that turn out to be naturalizable.

I agree that that’s interesting, even if it doesn’t count as a theoretical advance.

OK. Now one of the first “unnatural” (as far as I could tell) properties that I was interested in was the property of having a not too large dual norm. But as pointed out, that property is easily naturalizable, since it implies the property of having a not too small norm. I had been hoping that duality was a way of building nice properties that were not polynomially computable out of ones that were, and therefore of generating a large number of good candidates for properties that might distinguish between pseudorandom functions and random functions. However, it seems that no construction of this kind can work.

Can you say that formally?

It’s not at all hard to say it formally. The question is how generally one can say it. Here’s what I’ve got so far. Let be a polynomially computable norm. Then the property does not work as a simplicity property. The reason is that if is a Boolean function, then so the property implies the property So if is small for random functions (as it is for norms), then we have a polynomial-time way of distinguishing between random and pseudorandom, which is not allowed.

You don’t seem to have ruled out the possibility that there might be a norm that is not small for random functions, but such that the dual norm is large for random functions.

That’s true. So maybe I need to add that as an assumption. I’ll call a quasirandomness norm if it is small for random functions. So then the statement above is that duals of polynomially computable quasirandomness norms cannot work.

We should perhaps bear in mind what you’ve just said, though it seems to me to be a very long shot. Perhaps one could devise a norm that was large for random functions but small for functions that were “even better than random”. To give an example, if you define to be then the norm of a random function is rather than The reason is that each Fourier coefficient has a Gaussian concentration around 0 with standard deviation so if you’ve got Fourier coefficients to worry about, then you’ll expect one of them to have size equal to about standard deviations. However, if you construct a function using a high-rank quadratic form, then you can produce several functions with

Now the dual norm of is where this is the norm rather than the norm (meaning that you add up the moduli of the Fourier coefficients rather than taking their average). Since a typical Fourier coefficient of a random function has modulus about it seems that the dual norm of a random function is bigger by a factor of than the reciprocal of its (non-dual) norm.

Are you trying to say that you’ve got a new candidate for a property that could be used to prove nontrivial lower bounds?

I don’t know, since this idea has only just occurred to me and I haven’t had time to think about it. But let’s break off for a second and try to see whether we have some way of ruling out this property.

Of course we do. To calculate the dual norm you just work out the norm of the Fourier transform!

Oops. But that doesn’t quite kill off the underlying idea. For example, perhaps we could define a “quadratic” version of the above norm. We would let be the largest inner product of with any quadratic phase function There are of these, so we might expect one of the inner products to have size about times the typical size of . But perhaps we can also show that a random function correlates very well with a function that has only the expected correlation of

If that worked, then we might conceivably be able to get superlinear bounds. There would be two stages to the proof, both of which look as though they are either true but very hard to prove, or false.

What are they?

Well, unfortunately superlinear bounds for formula complexity are not interesting since they are already known (e.g. for the parity function). So we’d have to look at circuit complexity. We’d need to show that for every Boolean function of circuit complexity at most (for some function that tends to infinity) there is a quadratic phase function that has very slightly more than the expected correlation with . That is, one would be looking for a bound like or perhaps something smaller still. It seems highly unlikely that one could prove that by identifying a property of and using the property to find a with good correlation. But perhaps one could somehow prove that to get rid of all the correlations would take a superlinear number of steps. It might be a bit like the result that you need partitions of an -element set into two sets if you want every pair of elements to be in different cells of at least one of the partitions.

But at this stage I don’t even have a feel for what happens if you take a polynomial of the form for some collection of sets of size 3. That will have linear circuit complexity, and perhaps can be chosen so that it has essentially minimal norm and hence minimal correlation with any quadratic phase function. At the moment my guess is that that is indeed what happens.

Mine too I have to say.

But if we were very lucky and that didn’t happen, then the second stage of the argument would be to show that … oh wait. No, the second stage would in fact be easy, because I’m sure there is at least some cubic phase function with minimal norm. So that would be a function in P that had superlinear circuit complexity. The issue, therefore, is whether we can find one that’s computable in linear time.

There’s something that’s worrying me here. It’s that there’s a concept called algebrization that appears in a paper of Scott Aaronson and Avi Wigderson. It’s to do with low-degree polynomials, and says something like that if you have a proof that still works when you replace the field by some extension of it, then you’re in trouble.

Can you be more precise?

I’m afraid not. I don’t know enough about it to know whether it’s relevant here, but I think you should check. It’s another barrier to proving that PNP and it is known to kill off some proof techniques that get round the natural-proofs barrier. [Algebrization is discussed in more detail in a later instalment of this dialogue.]

Hmm, that is indeed slightly worrying.

Can I chip in here?

Of course.

Well, even if everything works, I don’t see how you are proposing to distinguish between random and pseudorandom functions. You’ve argued that it’s conceivable that a random function of complexity has a correlation with at least some quadratic phase function that is not quite minimal, but that’s true of random functions.

Yes, but then I was going to look at the dual of that, arguing that random functions do have maximal dual norm.

Right, but why shouldn’t random functions of complexity also have maximal dual norm?

Ah yes, I see your point. I’m not sure I have any reason to think that.

OK, let’s get back to the “negative” discussion and see whether we understand better the kinds of proofs that won’t work.

Oh, but hang on, I’ve just realized I’ve been making a calculation error.

Not for the first time …

I’m sorry about that — it’s a bad habit of mine, thinking about ideas without properly checking their flimsy foundations.

But it’s just occurred to me that the dreaded polynomial example isn’t as frightening as I thought. This is also relevant to the discussions we were having earlier about the norm for unbounded . Let me explain.

It can be shown that if is a polynomial of degree , is the polynomial phase function and is some other function, then Now the smallest that this can be is at least , which is much bigger than the correlation between and a random function.

But I thought you said that in the case where and is a suitably chosen quadratic you got rather than

So I did. What’s going on?

OK, I’ve done a few calculations, and it seems that using the inequality above is inefficient. You’ll just have to take my word for this, but the best bound one can hope to prove by easy Cauchy-Schwarz methods is that a well-chosen polynomial of degree has a correlation of at most with any polynomial of degree less than So when we get and when we get

When you say “by easy Cauchy-Schwarz methods” are you trying to suggest that more advanced techniques could give better bounds?

Possibly. This is very close to some difficult problems in analytic number theory concerning exponential sums of the form on the assumption that is not too close to a rational with small denominator. I’m not sure what the state of the art is when but I’m pretty sure people don’t know how to get to bounds like Also, I think the best bounds that are known use algebraic geometry.

This changes things, of course. It means that it is probably hopeless to try to kill off the approach using a polynomial. (And as for my earlier attempts to do that for polynomials of degree they now look laughable.) Instead, I think our best bet is to try to guess what happens if you take a random function of linear circuit complexity, where by that I mean a function taken from the model we were discussing earlier: first embed nicely into then apply random basic operations, and then look at the first coordinate.

I’m sorry to interrupt you yet again, but I think that model is still flawed.

Why?

Because most of the random basic operations you took will have no effect on the final answer. To see what I mean, let’s suppose that the sets of size 3 that are used for the basic operations are , in that order. Then we can certainly say that the th basic operation has no effect on the first coordinate unless So we’ll expect the last of the basic operations to have no effect. But that’s not all. Let’s suppose that is maximal such that We now know that no previous has any effect unless either And that happens with probability as well. Indeed, it seems to me that only a constant number of the basic operations will have any effect at all on the final answer.

You’re quite right of course. But I think we can deal with that problem in a simple way. Instead of looking at the first coordinate, let’s do something more global like taking the parity. In other words, the revised model is this. First you map into in a clever way, then you do a random sequence of basic operations, and finally you count how many 1s you’ve got and set if this number is even and otherwise.

It’s still not obvious to me that this model is a good one, because the randomized part (that is, the product of a random collection of basic transformations) is very simple and very easy to distinguish from a truly random function.

I find that quite interesting. It’s not clear to me either way. The random part does indeed look very simple, but I don’t see why it should remain simple when composed with a carefully chosen function. For instance, the random part will have constant depth and bounded fanin, which computer scientists would regard as very simple indeed, but the explicit parts that come before and after cannot be computed by bounded-depth circuits (because the parity function can’t), so perhaps one part of the procedure gives you depth and the other part gives you randomness.

But in a way, all this is irrelevant, because I still haven’t addressed ’s earlier question. Do I have any proposal at all for a property that would distinguish between pseudorandom functions and random ones? Unfortunately, I don’t think I do, or at least not one that I believe in. Let me explain and then we can get back to trying to rule out more general kinds of approach.

The property I’ve just been considering is the property of having a very large dual norm, where the norm itself is given by the maximum possible correlation with a quadratic phase function.

Let me say that more clearly. I’ll let be the set of all quadratic phase functions, which are functions of the form where is a quadratic in variables over I’ll then define to be Finally, the property I’m interested in is the property of satisfying the inequality

I don’t have a proof, but I’m guessing that random functions have this property (because they do if you replace “quadratic” by “linear”). Also, I’m guessing that this property is not in P.

Can you quickly justify that guess?

I can, but not all that quickly. A first step would be the claim that distinguishing between Boolean functions with (the random functions) and Boolean functions with (the better-than-random functions) is impossible in polynomial time. My reasoning there would be that there are functions in , and the sort of correlation one is looking for is so small that there are no clever methods to choose which of these functions will correlate with some random Boolean function . So one has to do a brute-force search, and is a superpolynomial function of

And then I’d claim that calculating the dual is even harder. One can in fact prove that the dual norm of is the smallest possible sum for which it is possible to write as a linear combination where each But there seems to be no earthly way of guessing which functions will be useful for the purposes of expressing as a linear combination of this kind.

Thanks — that sounds reasonably convincing.

I think so too, but unfortunately it doesn’t help. Prompted again by what goes on in the linear case, one might well guess that the dual norm of a random function is maximal to within a constant — in other words, it is around The rough reason for this would be that itself would correlate with some quadratic phase functions, but by modifying it a bit, one should be able to come up with a function that correlates well with but no longer correlates with those few quadratic phase functions that correlated with.

Now you have stopped convincing me.

Why?

Because it seems to me that there’s an important difference between the linear case and the quadratic case. The expected number of linear phase functions that a random correlates with is a small fraction of So it seems reasonable to suppose that with a small amount of tinkering one can get rid of these correlations without radically changing But in the quadratic case the number of troublesome correlations is a small fraction of which will be far bigger than the dimension of the space we are talking about. So it’s not at all obvious that we can get rid of all these correlations: the functions we are trying to avoid will probably span the whole space many times over.

That’s actually quite an interesting point. It affects what I was saying, but it doesn’t affect the main point I was leading up to, which is that I still don’t see any distinction between random and pseudorandom functions. To see why I’m saying this, consider some fixed quadratic phase function and let’s consider the behaviour of where is a pseudorandom function of complexity (from the model that doesn’t entirely trust). My heuristic principle is that the expected correlation decays exponentially, so when is something like then it is as small as it can be, on average at least.

Next, I get to a statement that I don’t really see how to justify, even heuristically, which is that the probability of deviating from the mean should be similar in the pseudorandom case to what it is in the random case. So if you now look at all quadratic phase functions, you will find that about of them will have correlations greater than

I think I might be able to justify that.

Oh really? How?

Well, one could say that the property of having an inner product of at least with some fixed quadratic phase function is a polynomially computable property that holds for an appreciable fraction of all Boolean functions (that fraction being ,) so the probability that it happens for a pseudorandom function should be roughly the same as the probability that it happens for a random function.

Oh yes. I think I buy that. I think from that argument we can extract a fairly general principle. Let be any collection of functions that take Boolean functions to the real numbers. (It may well be possible to genearalize this, but I’ll stick with real-valued functions for now.) Suppose that every function in is polynomial-time computable, in the sense that we can decide in polynomial time (in ) the approximate value of (up to an error of say). Then the distribution of over random functions should be almost exactly the same as the distribution over pseudorandom functions.

Oh … just a moment.

What’s happened this time?

Well, it’s occurred to me that there might after all be a difference between the norm of a random function and the norm of a pseudorandom function, where the norm is that quadratic-correlation norm.

It goes back to your argument that purported to show that the two norms were the same. It seems to me that that argument is fine up to but once you get bigger than that, the probability becomes subexponential, so exponentially approximating it is not good enough.

So at the moment it feels as though the norm of a random function should be around (because there are chances for it to be big, with a probability of each), whereas it seems very unlikely that a random model based on just random steps would give rise to events with that low a probability.

Are you saying that pseudorandom functions are more random than random ones?

Oh dear, something odd is going on here isn’t it? Maybe I am paying the price for ignoring the fact that the values of for the various quadratic phase functions are not independent. I was hoping that they would be sufficiently independent, however.

Yes, but once you know of those correlations, then morally you know all the rest, so perhaps it was just completely unrealistic to hope for events to have subexponential probabilities.

There’s also something else that I haven’t quite got to the bottom of, which is that if we define a norm by taking to be the set of all functions of circuit complexity at most and set to be then trivially all functions of polynomial circuit complexity have norm 1, whereas random functions have very small norm. But why can’t we say that for any fixed function the probability that given that has circuit complexity at most is roughly the same as the probability for a purely random ?

I think you can. It’s just that “roughly the same as” means that the two probabilities are exponentially close. So the fact that a random function is astronomically unlikely to have almost perfect correlation with a fixed function of circuit complexity at most allows us to deduce that a pseudorandom function has an exponentially small probability of this. The latter is compatible with every pseudorandom function correlating with at least one function of circuit complexity at most (as there are more than exponentially many of these), while the former is compatible with almost every random function not correlating with any such functions.

There’s a simple but quite useful general moral to draw from that, I think. It’s that if you want your simplicity property to imply correlation with a function from some collection then on the one hand you need to have superexponential size in or equivalently superpolynomial size in That’s because otherwise you can check in polynomial time (in ) just by brute force what the largest correlation is with a function in Conversely, if does have superexponential size, then there is at least a theoretical chance that it can be used as the basis for a simplicity property. All this makes me want to think some more about the quadratic correlation norm, where , the set of quadratic phase functions, has size

I’m getting a bit tired. Can we save that for another day?

Fine by me.

And by me.

Michael Nielsen and I have written an Opinion Piece for Nature about the Polymath project and related matters. Thanks almost entirely to Ryan O’Donnell, a preprint at last exists that contains Polymath’s proof of the density Hales-Jewett theorem with all the details. It will be posted on the arXiv very soon and I will update this post when it is.

Update: it can be found here. Owing to a misunderstanding, it was posted before I had any input into it, but in any case, the full proof is here, even if the version that is submitted for publication will have some changes.

The Notices of the AMS have published five back-to-back reviews of the Princeton Companion to Mathematics. They are by Bryan Birch, Simon Donaldson, Gil Kalai, Richard Kenyon and Angus Macintyre.

From Quomodocumque I learned of a new website, Math Overflow, where you can ask and answer mathematical questions. It seems to be very active, with a lot of users, rating systems for comments and commenters, and the like. So in principle it could be another mechanism for pooling the resources of mathematicians with the help of the internet. For example, if you need a certain statement to be true and do not know whether it is known, then my guess is that you could find out pretty quickly if you post a question there. For more discussion, see a post over at the Secret Blogging Seminar.

The story so far: our three characters, , and , have discussed in detail an approach that has to proving complexity lower bounds. At the beginning of the discussion, they established that the idea would be very unlikely to prove lower bounds for circuit complexity, which is what would be needed to prove that PNP. More recently, they have given arguments that strongly suggest that the idea cannot prove interesting new bounds even for weaker models. In this segment of the dialogue, they nevertheless consider what kinds of methods would be needed in order to prove lower bounds for circuit complexity (as opposed to formula complexity). They don’t really get anywhere, and this part of the discussion ends rather abruptly, but they mention one or two interesting facts along the way.

***************************************

After all that, I have to say that I’m still slightly disappointed that it seems that even if the ideas I’ve been having could be got to work, they are most unlikely to show that PNP.

I didn’t really understand why not. Can somebody remind me?

I can. Recall that a formal complexity measure is a function such that for every basic function , for every , and and are both at most for every and . Now if all you know about is that it is a formal complexity measure, then all you can deduce from the fact that is that has formula size at most . Therefore, if there exist functions with low circuit complexity but very high formula size, then one cannot use a formal complexity measure to prove that PNP unless it has some further properties.

Hmm … it seems that you need a pretty strong statement to back up what you’ve just said. If I can find a formal complexity measure and also a function in NP with , then that will prove that PNP unless there exist functions of polynomial circuit complexity and formula size at least . Is it believed that a function of polynomial circuit complexity could have exponentially large formula size, for instance?

I’d have to check. But if you take a fairly random circuit, then the obvious way of turning it into a formula certainly does require the size of that formula to be exponential, and it’s hard to see how to do any better.

How do you mean, “the obvious way of turning it into a formula”?

Well, if you’ve got a sequence of functions with each one either a basic function or a Boolean operation of two earlier functions, you can just build up a formula in the same way. For instance, if we write for the th basic function, then we could define a function by taking , , , , , , . Unpacking this, we get

If you don’t count the basic functions in the definition of circuit complexity, then the circuit complexity of is 3, while this formula has length 4. And that lack of economy goes up rapidly as more and more functions in the circuit depend on earlier ones. If you play around with it a bit, you’ll be convinced.

Great, thanks, I believe you. In fact, it’s sort of obvious really: as you increase the length of the sequence of functions, to get the formula complexity of you add the formula complexities of earlier functions, whereas to get the circuit complexity you just add 1. So the formula complexity really does look as though it goes up much more rapidly. Of course, that’s not a rigorous proof, since there might be some less obvious way of producing a formula that was far shorter. But I agree that that looks very unlikely.

Actually, couldn’t one use this thought to tackle the formula-size question? Perhaps one way to produce a function in NP with large formula size is actually to produce a function in P with large formula size by picking a random sequence of Boolean operations.

Two comments on that. One is that a random sequence of Boolean operations will give you a function of low circuit complexity, but that’s not quite the same as a function that can be computed in polynomial time. The second is that you’ll still need a proof that your random function has high formula complexity.

[See this comment of Ryan Williams for an explanation of why the first objection can be dealt with.]

I was wondering whether maybe it would have maximal formula complexity in the following sense: the obvious formula really is the best you can do, as long as you obey some simple rules that don’t allow you to produce a shorter formula in a trivial way. For example, you can shorten to , but that depended on the fact that and have some overlap. Perhaps in the random case you just wouldn’t get that kind of overlap.

If that’s the sort of thing you’d like to do, then let me tell you something depressing. It’s a well-known approach that morally ought to give a lower bound for formula complexity, but nobody has managed to get it to work.

Suppose that are formulae, and that they depend on disjoint sets of variables. Suppose also that is a formula that depends on variables. Then you might expect that the shortest formula for computing the function is the obvious one, where you compute for each and then apply . Therefore, if has formula size and all the have formula size , you might expect the formula size of to be .

Now choose a random function of variables. This will have formula size , which is somewhere close to , by a simple counting argument. We can then build a new function , where it is to be understood that we are applying to disjoint sets of variables. Then has formula size , if we believe our previous argument. Then we can build another function , which now depends on variables and has formula size , and so on. We keep going until we use all variables, which happens when we reach the function , where . This function has formula size , which equals , which is about , which is superpolynomial. But clearly the function can be computed in polynomial time, since can (as it depends on just variables).

That’s pretty cool. So what’s wrong with the argument?

Just the fact that I stated a major lemma and didn’t prove it. I didn’t justify the statement that the shortest formula for is the obvious one. And I think that’s an unsolved problem: certainly, it’s either an unsolved problem or someone’s found a counterexample, since I know that the formula-size problem is still open.

I suppose another point is that even if this argument worked, it wouldn’t actually give a superpolynomial bound for formula size, because the function that you used to build everything up was chosen randomly rather than given by some algorithm.

You’re right. In the absence of any further tricks, what this argument would show is that there was a function of polynomial circuit complexity and superpolynomial formula size. [See again Ryan Williams's comment.] But even without any new ideas, one would in principle be able to use this approach to obtain arbitrarily large polynomial lower bounds for formula size.

How would you do that?

Well, even if we can’t choose a random function to get things going, we could in theory take some fixed and find a suitable function by brute force.

Sorry to be slow, but how would you do even that?

You’d look at every single Boolean function of variables and every single formula of size at most, say, , and pick out a Boolean function that is not given by one of the formulae.

Wow. There are Boolean functions in variables, so that’s going to take a pretty long time, even for very small .

Yes. That’s why I used the phrase “in principle” earlier. But even distinguishing between polynomial circuit complexity and polynomial formula size would be very interesting.

Er, excuse me.

Yes?

We seem to have got sidetracked. I was just wondering whether it would be possible to return to the main topic.

How do you mean, “the main topic”?

I mean the question of whether any of the ideas we’ve been discussing could be used to prove that PNP rather than just the weaker statement that some explicit functions have large formula size.

Ah yes, your obsession with the million-dollar prize …

That’s not fair. I absolutely agree that the formula-size problem is a great problem, but that doesn’t mean we shouldn’t even think about circuit complexity.

Relax, I’m just teasing you. Everyone would prefer to solve the P versus NP problem if there was a chance of doing so, including me.

OK, well perhaps you’d like to answer a question that’s been bugging me. Here is a proof that the notion of a formal complexity measure is very natural if one is thinking about formula size. We’ve already discussed it, but let me quickly go through it again just so I can be clear about the question I want to go on to ask.

One can arrive at the notion of a formal complexity measure by asking the following question: if we define to be the formula size of , then what properties are satisfied by ? And one quickly observes that , and that and are both at most . Having done that, one can define a formal complexity measure to be any function with these properties, and one then observes that is the largest possible formal complexity measure.

Indeed.

The idea behind our previous discussion was this. In principle, one can prove that a function has large formula size by finding a formal complexity measure and proving that is large. But we have two problems. Obviously, to increase our chances of success, we’d like to be as large a function as possible. But if we take itself then we have done precisely nothing: we’ve shown that a function has large formula size if and only if it has large formula size. So there’s another requirement: we want to be a function that we can actually say something about, so that proving that is large is strictly easier than proving that has large formula size. However, here we run into the problem that if is too simple, and in particular if it can be calculated using a polynomial-time algorithm (in ), then it will almost certainly not work.

Let me quickly interject here that in their paper Razborov and Rudich showed that if a formal complexity measure is ever large, then it will be large for a positive fraction of all Boolean functions. So if you believe that natural proofs can’t solve the formula-size problem either, as you seem to do, then you definitely can’t use a polynomially computable formal complexity measure.

Right, that just confirms what I was saying.

Anyhow, on to my question. I’m just wondering whether we can go through the whole thought process again, but this time for circuit complexity. But I seem to have got stuck at the very first step.

How do you mean?

Well, the first step of the previous argument was to look for properties satisfied by the function So I thought I’d define to be the circuit complexity of and try to find some properties of But the properties I am coming up with don’t seem to be strong enough. For instance, if you know and what can you say about ? The best I can come up with is that it is at most which is no stronger than we have for formula size.

Ah ha, that is a very good question, particularly as there isn’t any reason to think that that statement can be strengthened.

Why do you say that?

Well, suppose that and depended on entirely different sets of variables. Then it is hard to see how to calculate more efficiently than calculating , calculating , and taking the max of the two. That gives an upper bound

What? You mean it’s even worse than it is for ?

I think so. The trivial bound requires you to add 1.

I ought to say that the argument I’ve just given is another “obvious” argument that is not in fact known to be correct. I claimed that calculating the max of two functions that depend on disjoint sets of variables cannot be done more efficiently than the obvious way of doing it. But if that is really so, then one can obtain arbitrarily large linear lower bounds for circuit complexity as follows. Let be a small constant, and by brute force find a function that has circuit complexity at least Then let be copies of , each depending on a disjoint set of variables. Here, is Then the obvious straight-line computation of has length around , which is around However, the best known lower bound for circuit complexity is something like so I’m pretty sure that the result about functions depending on disjoint sets of variables is not known.

What you’ve just said manages to be more irritating than one might have thought possible. I’m trying to say something about circuit complexity, and you have managed convince me not only that I can’t, but also that I won’t even be able to make rigorous the demonstration that I can’t.

Welcome to the wonderful world of unconditional results in theoretical computer science.

Hey, cheer up.

OK, let me try to say something to make you feel better. I want to point out that you’re not forced to define to be a number, and if you don’t then there’s a slightly better chance of proving something interesting.

I feel better already. Tell me more.

I’d better warn you that what I’m about to say is less promising than it might at first seem. It’s still worth mentioning, but promise me you won’t get too excited about it. It’s a brilliant idea of Razborov, but he has also proved that it cannot be used to obtain interesting lower bounds for circuit complexity.

What is it with this guy?

Well, I think like you he’d love to prove that PNP. But unlike you he has made (at least) two extraordinary contributions to the area: he has proved major results in the direction of the problem, and he has checked very very carefully that they cannot be pushed any further — indeed, he has proved this rigorously.

Anyhow, let’s not worry about that for now. I just want to show you that there is a different notion of formal complexity measure, where you allow it to take values that are sets rather than numbers.

To explain what I mean, let’s suppose that we have a function that associates with each Boolean function some set We are thinking of as being a measure of the complexity of , in some sense. What properties might have that could be useful? One obvious one is to insist that and are both contained in But that is not terribly helpful, because it implies that every is contained in the union So it looks as though can’t be bigger than times what it is for a basic function.

Hang on. It seems to me that you are implicitly taking your sets to have some notion of size attached to them.

Yes I am. What I want is for to be small for basic functions and very large for some function in NP. And I also want it to grow only slowly when you do Boolean operations.

Here is a set of axioms that would be very useful to us if we could obtain them. The idea is to allow a small error on the right-hand side. That is, instead of asking for and to be contained in , we ask for them to be contained in where is a set of small measure. And to get things started, we’ll assume that whenever is plus or minus a basic function.

If that is the case, then what can we say about if it has small circuit complexity? Well, we have a sequence of functions , and for each we can find such that for some small set From this and a trivial inductive argument it follows that So if we can now show for some function in NP that is a set with measure that is larger than that of any set by a superpolynomial factor, then we have proved that cannot have polynomial circuit complexity.

OK, I’ve got lots of questions about this, but here are just two to get started. How might one go about defining a useful set-theoretic complexity measure like this? And is there some trivial argument, as there is the case of formula complexity, that gives a kind of “maximal” set-theoretic complexity measure such that the size of is actually equal to the circuit complexity of ?

Let me deal with the first of these questions by telling you about Razborov’s method of approximations. The idea of this is to construct a lattice of Boolean functions, where all that this means is that you have the basic functions (and their negatives), together with some operations, and , under which is closed.

Aren’t you going to assume anything about those operations? For example, shouldn’t they be associative?

Oddly enough, we don’t need to assume that, but if is to be of any use, then the operations will have to be chosen very carefully, as you’ll see.

Indeed, what we want from is two properties. We would like the operations and to approximate the operations and and we would also like to be as small as possible.

Now suppose that we have a straight-line computation If we carry out exactly the same computation but replacing the operations and by and then we obtain a sequence Now for each and let’s suppose that and except on a small set Then define to be the set for which equals or An easy inductive argument shows that agrees with except possibly on So we can set to be

Something’s bothering me here. It’s that your function isn’t obviously well-defined. It seems to depend on the particular way that is computed.

You’re absolutely right of course. I must apologize. But it turns out that for some problems, one can prove highly non-trivial lower bounds using this kind of idea, though I have in fact oversimplified what Razborov did.

The bad news is that Razborov has proved that the method of approximations doesn’t yield even superlinear lower bounds for circuit complexity, unless you allow extra variables, in which case you can produce a suitable lattice in a trivial universal way.

Ah, that’s sounding close to an answer to my second question. But perhaps we can think about it for ourselves.

Wait, there is indeed a spectacularly trivial set-theoretic complexity measure if it’s allowed to depend on how the function is computed.

What’s that?

You just take the set of all functions that you used in the computation. Then if say, then you’d have So the size of would be at most and it’s not hard to show that the size of is in fact the length of a computation closely related to the one given. (For uninteresting reasons it could be different, since and might both be less than but in that case would not have been used in the computation of )

I’m still interested to know whether there might exist a set-theoretic complexity measure that applies to functions rather than to computations of functions. Has Razborov shown that they cannot exist?

I’m not sure. What I can say is that his method of approximations doesn’t give you one. If you wanted it to do so then you’d have to find some way of associating with each a function that belonged to the lattice in such a way that was equal to and was equal to But that would be asking for a partition of that somehow respected Boolean operations. I’m basically certain that such a thing doesn’t exist. In fact, I think I’ve got a proof. If is small, then by the pigeonhole principle would have to be the same for a large set of functions But then you’d have two functions and that were almost orthogonal, from which it would follow that was nothing like either or but was equal to and

I don’t think an additive combinatorialist would give up just yet.

What do you mean?

Well, I can see that there is a problem with random functions if you want to define to be a Boolean function. But couldn’t one generalize the notion of a lattice to allow functions to take values other than and ? In additive combinatorics, one tends to think of a random function as being “just a random perturbation of the zero function”.

If you want to try something like that, then you’ve still got some serious thinking to do.

Why?

Well, if you take two independent random Boolean functions and then is a random function as well, but it takes the value with probability instead of This suggests that if the zero function belongs to then should be the constant function But if and are not independent, then this is no longer appropriate. For instance, if , then is the constant function 1. So you can’t just send all random functions to the zero function and hope to preserve Boolean operations in a nice way.

I see what you mean. That looks pretty bad. But does it show that no set-theoretic complexity measure that depends just on can exist? Or does it merely show that it cannot be derived from some lattice ?

I’m not sure. I’d be quite surprised if one existed. But perhaps it’s possible to find one in a trivial way.

It’s difficult to see how one could define such a set without making any reference to circuits. So perhaps one would have to associate a set with each straight-line computation and then for each one would take the intersection of all the sets associated with computations of

I’m not sure I like the sound of that. For instance, if you apply it to the “trivial” function that associates with each computation the set of Boolean functions involved in that computation, then you don’t get anything interesting when you intersect over all computations.

Yes, you’re right. One would probably have to do something fairly clever.

It’s a very long shot, but I wonder whether the idea could somehow be converted into a set-theoretic complexity measure. For example — and I should quickly say that I do not for a moment expect this particular example to work — one could associate with each Boolean function the set of all Boolean functions such that is a witness to the fact that . That is, I’ve chosen that just because the sets get bigger when gets bigger. But I have no reason to suppose that anything much can be said about in terms of and

In fact, I can’t face thinking about it for now. I need a rest.

OK, see you again some other time.

But just to end on not too depressed a note, I’d like to point out that a property such as “The set of functions that witness that the dual norm of is at least 1 is not too large” is not completely obviously naturalizable. More generally, perhaps we can get somewhere by considering properties that are not in NP but in some class like #P (but probably not quite as extreme as that) where you have to recognise roughly how many ways that there are of checking that an NP property holds. We may manage to come up with some reason for this not working, but I think we’d have to think about it first.

Here is the next instalment of the dialogue. Whereas the previous one ended on an optimistic note, this one is quite the reverse: plausible arguments are given that suggest that the dual norm approach is unlikely to give superlinear lower bounds. A few other ideas are introduced in the process.

********************************************************

Can I make a slightly cheeky comment?

OK

Well, over the years I’ve heard about a number of approaches that people have had to famous problems, including the P versus NP problem, that have the following characteristic. Somebody who works in a different area of mathematics comes in and claims that the right way of solving the famous problem is to reformulate it as a problem in their area. They then make grand claims for the reformulation, and there’s quite a bit of excitement, though also scepticism from those who have thought hardest about the problem, and ten years later nothing has actually happened. I’m usually a little suspicious of these reformulations, unless they feel so natural as to be almost unavoidable. So the fact that you like additive combinatorics and are trying to reformulate the P versus NP problem (or even just the formula-complexity problem) as a problem in additive combinatorics makes me very slightly wary.

Yes, I was conscious of that myself, although the norms have already been considered by theoretical computer scientists. It’s another reason I don’t want to make grand claims for this approach. I certainly wouldn’t say that the additive-combinatorial perspective is forced, but I do think I can argue that at least some of the ideas — particularly the duality idea — are quite natural. I think the next step is for real computer-science experts to have a look at them. [This has to a large extent already happened in the comments on the previous instalments of this dialogue.] I hope, 8), that you don’t mind my implication that you are not a true expert — I’m very grateful for the comments you’ve made.

No no, absolutely not. I admit that there’s a lot I don’t know in this area, and I could easily have missed some serious objection to your proposals.

Also, I’ve been thinking a bit about your objection that the dual norm is too much of a guess to be taken seriously. I think I have a heuristic argument in its favour.

Ah, that’s much more interesting. What is it?

I can summarize it as follows. I have the following picture of what a random function of circuit complexity (or perhaps formula complexity — let me be vague about this, since we can always think about it properly later if we want to develop this heuristic argument further) looks like. It springs from a result that appeared in a little known paper by Gowers about 15 years ago. In it, he defined a model of random computable functions, though in fact they were rather special functions because they were reversible, and proved that if you choose a function of complexity from this model, then it will be approximately -wise independent, where depends in a power-type way on and

You’re losing me a bit here.

Let me not bother to say precisely what the model is. But to say that it is approximately -wise independent is to say that if you choose any Boolean strings of length and look at the value of a randomly chosen function of complexity , then the values that takes at those strings is distributed almost uniformly amongst all the possible assignments of values to those strings. In other words, you cannot tell that is not a purely random function by just looking at strings of its output.

So in a weak sense Gowers’s random model was a pseudorandom generator.

In a very weak sense, yes, but still quite an interesting sense.

Did you know that if Gowers had waited a decade or so then he could have had a stronger result for free?

Eh? He told me he had wondered about stronger results, but got nowhere.

That’s amusing, because it has subsequently been proved that almost -wise independence implies pseudorandomness for circuits of constant depth. This result is discussed by Scott Aaronson on his blog.

So you’re saying that in fact he defined a generator that was pseudorandom for circuits of constant depth, and provably so. I must ask him whether he knows about that.

Anyhow, all I need for this discussion is the original result. Let us take “random computable function” to mean a function that comes from his model. I should add that Gowers was not in fact the first to define reversible computations, though he rather sweetly appeared to think he was. In particular, they are important in quantum computation. But the result about almost -wise independence was his. And -wise independence is enough to imply that the norm is typically small, where . We’ve noted this already.

What follows is plausible guesswork. I’m going to guess that a random function of complexity is almost -wise independent for some that is comparable to . In fact, I’ll guess that the you get is almost as big as the trivial upper bound you’d get by counting how many different functions of complexity at most there are. And I’ll also guess that if you take larger than the bound for which almost independence holds, then in some sense the distribution is “maximally dependent”.

In other words, what I’m saying is that if you choose a random function of complexity , then in a certain sense it should be completely random on sets of size up to around (including on subspaces of dimension , which is what we care about when evaluating the norm), and as structured as it can be, given this “local” randomness, on larger sets.

If that is correct, then how might we detect the structure? Well, a fairly natural idea is to look at a norm for some such that is bigger than . The hope is that this would pick up the large-scale correlations that are invisible to lower uniformity norms.

This is supposed to give an answer to the following question. If we are presented with lots of random computable functions, then it makes sense to talk about almost -wise independence. But what does it mean to look at an individual function and say that “it looks like a -wise independent function but no more”? At first that seems nonsensical. But perhaps if , then it could say something like that the restrictions of the function to -dimensional subspaces are sufficiently related to each other for some correlation to be picked up by the norm. In other words, our attention turns from correlation between functions to correlation between restrictions of the same function.

Hmm, that does make me feel somewhat happier with your idea. But it also suggests to me that there might be other ways of exploiting these large-scale correlations. You haven’t given any particular reason to go for subspaces rather than arbitrary sets of size .

True, but you have to admit that subspaces are pretty natural objects.

Actually, wait a moment. Now that I’ve gone to the trouble of explaining my heuristic picture of what a random low-complexity function is like, I’m starting to wonder whether I might have stumbled on a two-line proof that PNP.

You haven’t, but go on, let’s hear it all the same.

Well, the idea is closely related to what I was saying just now. I was thinking of a random function of complexity as being utterly random if you look at any (or thereabouts) values, and much more structured if you look at more values than this. But I wanted to think about just a single function rather than a probability distribution based on such functions. However, I can convert a single function into a collection of functions in a very simple way: I just look at restrictions of it.

I hope there’s going to be more to your argument than looking at random restrictions, because those have been done to death.

I’m slightly worried about that, I have to admit, because I don’t seem to be using much more. But just hear me out.

OK.

Well, suppose you’ve got a function of circuit complexity . We can convert into a function of variables by fixing the values of variables and allowing the other to vary. It is easy to check that if you do that, then the resulting function will have circuit complexity at most . Now there are at most functions of circuit complexity , whereas the number of Boolean functions of variables is . Therefore, if is big compared with , the number of possible restrictions of to variables is tiny compared with the number of Boolean functions of variables.

We have been searching for a difference between random low-complexity functions and purely random functions. Well, here it is: a random low-complexity function has just a limited number of possible restrictions, whereas a purely random function is completely arbitrary.

Hmm, I don’t yet see what’s wrong with your argument, but I do have a proof that it’s wrong.

How can one prove that an argument is wrong without seeing what is wrong with it?

That’s not mysterious at all. For example, one can show that it implies something false. But that’s not what I’m going to do here. Rather, I’d just like to point out that the property that claims distinguishes between random low-complexity functions and purely random functions is a natural property.

You mean, not only have I proved that PNP, but I’ve also shown how to factorize large numbers?

Er, I think not.

Cool it, I was joking.

You had me there for a moment.

Can you explain why this property is natural? In fact, what property are we talking about?

I assume that what has in mind is to look at the restrictions of to variables and simply count how many of them are different. But the number of restrictions is which is polynomial in and checking how many of them are different can clearly be done in polynomial time as well.

Ah, thanks. But doesn’t that also show what’s wrong with the argument? After all, if there are only possible restrictions, then the number of distinct functions you can get is not but which is far less. Indeed, it’s far less than our upper bound for the number of functions of circuit complexity

Oops, you’re right. To see in an extreme way why the argument is wrong, one could take the case Then there is only one restriction of and the fact that it is one of at most functions has nothing to do with anything. It just points out the familiar fact that the number of low-complexity functions is much smaller than the number of all Boolean functions. How ridiculous that this argument could have fooled me for even a millisecond. Though I did in fact share your feeling that it was too simple to be correct.

But wait a moment.

Here we go again …

Sorry, but there’s something else to check. What if, instead of looking at restrictions to subspaces generated by of the basic variables, we were to look at arbitrary -dimensional subspaces? There are far more of these: indeed, if is much smaller than , as it is in our case, then almost any elements of generate a subspace, and those subspaces don’t coincide all that much. So, speaking crudely, there are something like subspaces. And that … damn, that’s still smaller than if is something like But perhaps we could still get a superlinear bound this way.

I don’t think so, because now it’s no longer the case that the restriction of a function of complexity at most has complexity at most

Oh yes, you’re right. But perhaps it must still be quite special. How many functions can you get by taking a function of variables of complexity at most and restricting it to a -dimensional subspace?

Hmm, one way of thinking about it is this. Each such function is obtained by taking a Boolean string of length , multiplying it by an matrix over , and applying a function of complexity at most