I also like the idea of presenting both and . I find that when trying to understand what is going on in a particular representation it is slightly inconvenient to have only half of the off-diagonal coefficients displayed.

BTW, there are some small values in the solutions with a double negative sign –. They should be positive. In fact here the solver returned a very small negative value for a variable that was supposed to be non-negative, so this is a tolerance issue.

2. There are some values which are quite small for m=50,100, and fairly small for several other values of n, e.g. d7 and d17. Could these values actually be set to exactly 0 and still give an optimal solution for all n?

Once some analysis of the solutions at hand has been done I’d be happy to run some larger cases as well.

The solution for 100 is finally done – the LP solver took almost 23 hours and more than 170,000 iterations to solve the problem. I’m not going to be able to solve any larger problems on my machine unless we restrict the problem somehow.
The data for 100 is here:
http://webspace.princeton.edu/users/moses/EDP/try-lp/

The solver is currently working on . It’s taking up about 3G of memory for this instance, so it is unlikely that I can solve bigger sizes, but we should be able to get solutions for larger by running an LP solver on a parallel cluster.

Another way of thinking about the multiplicativity condition is to say that the map is an isometric embedding for every .

\displaystyle \sum_{k,d}c_{k,d}|x_d+x_{2d}+\dots+x_{kd}|^2-\sum_ib_ix_i^2 look
at the following

\displaystyle \sum_{k,d}c_{k,d}|x_1+x_{2}+\dots+x_{k}|^2-\sum_ib_ix_i^2

the idea is to try to use the information that the function is multiplicative to get a simplification as it looks like the number of different types of sums that are squared depends only on the number of terms and that might be useful.

Let me describe the winning strategy if the idea works.
The player trying to force the discrepancy chooses
the lowest available prime and sets it positive.

Let me look at a specific case and show how the discrepancy can be forced. I believe that this idea can be extended to further cases.

The case is that the player who is trying to force the discrepancy moves
first and chooses 1/2 then for some reason the other player does not choose 1/3 so the first player chooses that and both primes are set to 1.

The first step of the proof is to group the prime
factors in pairs. Because of the way signs are assigned
they can be arranged in pairs. The first of these pairs will
be two and something. Now we can show that the
constributions made by all numbers having factors nontrivially
in a in any of the later pairs can be ignored. We note
that for any pair p,q and multiple rx, with x having factors only
in p and q we can order p pr qr p^2r etc in a way that the series
is alternating and starting with a positive number this will result
in a positive or zero discrepancy to the total discrepancy so if
the factors having factors in the first pair contributes k
to the discrepancy then we will be done if we can make k arbitrarily
large. So the idea is to ignore everything except the first pair. Now we can make the first pair consist of 2 and 7. And then if we look at these we can see the discrepancy will be high as we can pair powers of consecutive powers of 7 which will have different signs and which will increase the discrepancy. Anyway that is roughly the idea that I am looking at now.

Since the number of comments on this post is approaching 100, I am in the process of writing another post. I am using this opportunity to try to formulate this representation-of-diagonals approach as clearly as possible, and I will also mention there a few thoughts I have about how one might actually go about finding a good representation. But these thoughts could well need to be refined a great deal in the face of the experimental evidence. (Another possibility is that the optimal solution involves an apparently magical cancellation that leaves one none the wiser about how to approach the problem theoretically, but I hope it will give some useful clues.)

1. Proving that numbers are zero is easier than proving that matrices are positive semidefinite.

2. If we try to represent a diagonal matrix, we don’t have to decide in advance what that matrix should be, so we save ourselves from having to make some huge guess. All we care about is that on average the norms of the test vectors we use should be unbounded.

3. Trying to prove a stronger statement is often easier than trying to prove a weaker one, because one’s moves are somehow more forced. (There are counterexamples to this — consider van der Waerden’s theorem and Szemerédi’s theorem, for instance — so there is no guarantee that this principle applies in our case, but I have a hunch that it does.)

Never mind the fact that it is trivial — let’s see what it implies. As above, we know that the representation-of-diagonal approach fails if and only if there exists some matrix with 1s down the diagonal such that for every test vector . Therefore it succeeds if and only if for every matrix with 1s down the diagonal there is a test vector such that . As I noted above, that implies the existence of HAPs and such that . (Actually, I forgot the modulus sign earlier, but once it’s there the conclusion is valid.) Now if , then this tells us that .

Thus, if the method succeeds, then we get the following apparently stronger version of the vector-valued EDP: let and be any two sequences of vectors such that for every . Then there exist HAPs and such that for some that tends to infinity with . I haven’t thought about whether this statement is likely to be true. If we insist that for each then we recover the usual vector-valued EDP. Note also that the statement is potentially interesting even for real numbers: if we have two sequences and such that for every , does it follow that we can find HAPs and such that the product of and is unbounded in size?

Is this non-symmetric vector-valued version equivalent to the representation-of-diagonals statement? I was expecting to have something to check there, but now that I look back I think it’s trivial, since characteristic functions of HAPs are test vectors.

In order to prove a lower bound of on discrepancy via the SDP argument, we need the following property: for any positive semidefinite matrix with 1′s on the diagonal, there must be a HAP with corresponding characteristic vector such that . This is equivalent to saying that in the vector version of EDP (where elements of the sequence are unit vectors instead of ), the square discrepancy of some HAP is at least . This is by the argument in Tim’s comment above. Since is psd, we can find vectors such that . Then .

In order to prove a lower bound of on discrepancy via the representation of a diagonal matrix argument, we need the following property: for any matrix with 1′s on the diagonal, there must be a test vector
such that . I don’t think you can assume that is positive semidefinite, and if so, I don’t have a good interpretation for what this condition is saying.

On the other hand, consider the modification of the representation-of-diagonal argument to allow to be positive semidefinite (earlier we insisted that this was 0). For a proof of this kind to succeed, I think you need the following: for any psd matrix with 1′s on the diagonal, there must be a test vector such that . If this is the case, as Tim points out, there exist such that and the existence of the test vector implies that for some characteristic vector of a HAP. But then, . Does this mean that instead of test vectors , we could have used characteristic vectors of HAPs instead ? If this is true, it seems that this proof technique is equivalent in power to the SDP dual argument.

But what I can definitely say is that if is of that form then we can find and . That tells us that , which by Cauchy-Schwarz tells us that the sum of the in either or has norm at least . So the success of the method implies the vector-valued EDP, but it may prove something a bit more general still. I’ll have to think further.

with a standard error of .

On the other hand, if you insist that you get a diagonal matrix exactly, my guess is that the best such bound is incomparable with the SDP dual. Allowing the positive semidefinite generalization may allow you to handle situations where you have almost got the non-diagonal entries zero, but not quite.

Here is the simplest linear-algebraic question coming from EDP that I can think about: Let A_1, A_2,…,A_m be a partition of {1,2,…,n} into intervals of length at most t.

Consider the system of linear equations , where we have an equation for every r and j such that is contained in {1,2,…,n}.

Then we conjecture that this syetem of equations implies for a large n that one of the s equals to zero.

It is simple but sort of a strange question. (I suppose we also conjecture that in order to have any non zero solution the A_is should be very “structured” but we dont know what “structured” precisely means. It should be related to the lengths of the intervals being periodic)

An equivalent way to state it that takes us closer to SDP is that if we add to the equations the inequalities then the system is never feasible.

Maybe looking at the dual problem can add some insight?

The fact that we have equations for every way we try to partition [n] to short intervals makes it seemingly more difficult than the direct SDP approach.

The have the property that if then the discrepancy of on some HAP is at least . So if is positive semidefinite, then we can apply it to a sequence (that is, multiply on the left by and on the right by ) and we will find that is at least as big as the trace of (by positive semidefiniteness of the difference). It follows that the discrepancy of the sequence is at least the square root of this trace, as usual.

Put like this, it makes it look as though all the approach is doing is replacing characteristic functions of HAPs with convex combinations of plus or minus these characteristic functions. But this extra flexibility raises the possibility that one might be able to prove semidefiniteness of the difference by showing that the difference is actually zero. I’m still not sure whether that is attempting to prove something far too strong.