Conversely, if you have a low-discrepancy -function on then, as you say, its product with itself obviously has at most the square of that discrepancy with respect to products of HAPs.

Looking back at your question, it seems that I did indeed mean what you thought I meant, which is not what I actually wrote.

The one thing that I still haven’t quite seen the answer to is what happens if you look at products of HAPs of the same common difference and the same length, or perhaps even products of HAPs with themselves. I’m not sure how interesting that is, though.

In fact, you constructed this problem as a weakening of vector-valued EDP. In the vector valued EDP case the function would need to be PSD in addition to for all . I am not totally clear at what step the weakening came in. We took the dual of vector-value EDP, and then restricted it more by looking for a decomposition of a diagonal matrix as opposed to a decomposition of a PSD matrix. Taking the dual of a dual with additional constraints, resulted in this weaker discrepancy problem, I suppose?

One benefit is that this can be tried with linear programming and hopefully scale a little further than the semidefinite programming search.

On writing that, I have suddenly had an idea for a counterexample for the same-common-difference version. I’ll describe it very roughly and think about it properly tomorrow. The rough idea is to paint with diagonal stripes of constant width and alternating colours (the colours being 1 and -1), but at an angle whose tangent is a badly approximable rational. The hope would be that if and have the same common difference, then …

No, I don’t think that works. Given any two real numbers and we can find an integer such that and are both very close to an integer, and I think that fact can be used to kill off the idea I had.

Actually, I think a counterexample to this can be turned into a counterexample to the non-symmetric vector-valued EDP with the added condition that and have the same common difference. So maybe I believe that the answer is yes (but possibly without the same-common-difference condition). It also feels like a friendlier formulation of the problem, and one where it might be interesting to do some computer searches.

Just in case anyone’s reading this (it feels a bit monomathic, so maybe nobody is), here’s a search that would interest me a lot. How large a matrix, or even just a matrix with 1s on the diagonal, is it possible to find such that the sum over every product of HAPs is at most … well, whatever small number is most interesting? Are there very big matrices where it’s at most 2? What about 3?

What I think this (embarrassingly simple) observation shows is that if we want to prove EDP for primes and powers of 2, then we are forced to use in a strong way the fact that the sequences are -valued: we can’t use matrix decompositions to prove the result, since they prove more general results that are false.

And what that shows — at least if you believe that matrix decompositions are the most promising approach — is that we are not likely to make EDP easier if we try to prove a stronger result by restricting the common differences that are allowed. It seems to be important to use all common differences.

So I’m pretty sure that is a very simple example that shows that the non-symmetric vector-valued EDP relies strongly on taking HAPs of every common difference. Or at least, you have to have multiples of every positive integer amongst your common differences.

That doesn’t answer the question about whether EDP is true for common differences that are primes or powers of 2, but it says that a 6D generalization is false.

It is genuinely unclear to me whether the 6D counterexample should lead one to believe that there is a 1D counterexample. In one dimension there is a parity argument (in any sequence of odd length the number of 1s cannot equal the number of -1s) that has no counterpart in higher dimensions. Maybe that is just a superficial fact that stops a certain example from working. But maybe all examples have to be “essentially periodic” in some sense, in which case there is a genuine distinction between 1D and higher dimensions.

Perhaps we should try to find a complex example …

If that is correct, then we can set and prove that there is no efficient decomposition of the identity into products of HAPs of prime-power length. That ought to prove that some generalization of EDP is false for such HAPs. I’ll try to work out the details — I’m not quite sure that this is correct.

This gives us quite a strange EDP-like problem. We have a class of functions ( being the amount by which we dilate ) and we’d like to find a function , defined on , that has a bounded inner product with every function in . Of course, something has to force to be “large” (so we can’t just take the zero function). At first, the condition appears to be ludicrously weak — all we ask is that should equal 1 — but it isn’t weak because the functions are not bounded at zero.

As ever, we can dualize. It will be impossible to find such a function if one can write the function (which takes the value 1 at 0 and 0 everywhere else) as a linear combination of the functions in such a way that the absolute values of the coefficients can have arbitrarily small sum.

Another way of looking at the dual problem is that we try to find an efficient matrix decomposition into HAP products where the common differences in each product are the same, but instead of asking for the resulting linear combination to be a diagonal matrix with large trace, we ask merely that the matrix has large trace and that all diagonals apart from the main diagonal sum to zero.

(i) .

(ii) .

(iii) .

An example is the sequence where are . The discrepancy of this sequence along any HAP of common difference that is not a multiple of 9 is at most 1. Now let us define to be . Then we certainly have property (i) from the previous comment. How about property (ii)? I’m fairly sure we get it with at most something like 6, for all that are not multiples of 9, that is.

Here’s a quick proof. For every and the sums and are both zero. But the sum over can be decomposed into a sum of these zero sums plus the sum over a product of HAPs of length at most 8. So the discrepancy is bounded (by a constant I can’t be bothered to work out — it’s trivially at most 64 but that can be improved quite a lot; in fact, it’s trivially at most 16; in fact, it’s trivially at most 8, but even that isn’t best possible).

This shows that we can’t use products with the same common difference if consists only of non-multiples of 9. I have a strong suspicion that we can do the same for non-multiples of for an arbitrary . All we need is numbers such that and such that for every that divides apart from itself we have . It is trivially possible to satisfy those equations. Just choose the numbers arbitrarily except that each time you reach for some proper factor of make sure that it cancels out the sum .

This shows that if we want to decompose a diagonal matrix efficiently into products of HAPs of the same common difference, and if we insist that the differences all belong to some set , then must contain multiples of every positive integer (or every small positive integer if we are thinking more quantitatively).

But we’re still left with a 2D question related to EDP that might turn out to be interesting. Let be an arbitrary function such that for every . Is it true that for every there must exist HAPs and with the same common difference such that ?

If is forced to be of the form , then this question reduces to EDP. Since we are looking at a far more general class of functions, and since EDP appears to be “only just true”, it seems highly likely that the answer to the above question is no. I just haven’t seen it yet.

Now let’s think about what it says about if for every linear combination of products of HAP functions, where the two HAPs in each product have the same common difference that belongs to a set and the sum of the absolute values of the coefficients is at most 1. That will be the case if and only if it is the case for each individual product. So we are asking for whenever and are HAPs that both have common difference , where is required to belong to .

This is a kind of 2D version of EDP. If we scale things up, then it asks the following. Do there exist a constant and a function such that

(i) for every , ;

(ii) for every and every pair of HAPs of common difference , the sum has absolute value at most ?

In particular, do these exist if is the set of all primes or powers of 2? If so, then for that problem we are forced to consider products of HAPs of different common differences.

Maybe this is an interesting problem to add to our collection of EDP-related questions, but I think there may turn out to be a fairly simple example that has the given properties.

I think this means that for the primes and powers-of-two problem one basically has to take products of HAPs with different common differences.

So instead of a decomposition of a diagonal matrix, it is enough to look for a decomposition of a strongly diagonally dominant matrix, i.e. a decomposition of a symmetric matrix where each diagonal entry satisfies for . The goal is to maximise with respect to the coefficients in the decomposition.

Now let’s just look at primes and let’s suppose that the are actually constant matrices. Then the value of the sum of the at can be expressed by a formula of the form . But that expression just doesn’t look as though it can be anything like constant: for instance, if we restrict attention to small primes, then in any large rectangle we can find pairs and such that the highest common factor is a product of all small primes for which is large and positive, and other pairs that are coprime.

The weaknesses in that argument are that larger primes do exist, and they make things more complicated, and also that, as I noted earlier, one can impose quite a lot of oscillation on, say, the sequence , while still having a bounded total oscillation. So the large rectangle doesn’t obviously exist.

I’ll say that a matrix has total oscillation at most if . I’ll also call a matrix a dilate of a matrix if and is zero unless both and are multiples of .

I would like to find a matrix for each allowable common difference (that is, a prime or a power of 2) such that

(i) each is an matrix, where ;

(ii) the sum of the total oscillations of the is at most ;

(iii) if is the dilate of by a factor , then is a diagonal matrix with trace at least .

Now this looks really pretty hard if almost all the have to be prime, so I would like to try to argue that it cannot be done. If it can’t, then we might manage to find a very interesting sequence with bounded discrepancy (for one of the modified problems). If it can, then we’re in great shape for EDP.

We know how to define such sets, but how big do we want “small” to be in this context? Here is where it gets problematic. One might think that it would be a good idea if numerators and denominators were allowed to go up to some such that in some useful sense “most of ” is defined on . But what does “most of” mean here? It can’t refer to the sums of the absolute values of the coefficients , since we can make all but the first one arbitrarily small.

At this point one notices that even if the coefficients of the HAP products are small, the HAP products themselves are large in some pretty natural norms. For example, the Hilbert-Schmidt norm of , if , is 1, and we were in a situation where the length of the HAPs we needed to take was proportional to the reciprocal of the coefficient we attached to the product.

So, speaking extremely loosely, on the one hand we get a contribution of for every positive rational, since the diagonal contribution is , and on the other hand we get a contribution of at most for every positive rational, since the sum is .

It would be great if this were some kind of contradiction when , but clearly for that we need to go into a lot more detail about how things tend to infinity. I’ll turn to that in a new comment.

(i) can be written as , where and are characteristic functions of intervals of integers and .

(ii) .

(iii) If and are coprime and not both equal to 1, then .

Let me try to deduce EDP from the existence of . Barring a major surprise, I will fail, but I want to understand why.

Incidentally, there are two reasons that it would be extraordinary if the proof worked. The first is that I realized last time that it didn’t work. But that on its own is not a good enough reason — it’s easy to make mistakes when deciding that something can’t work. A better reason is that there is something too simple about this approach. A solution to EDP ought to involve more work, and I can’t escape the feeling that if an approach like this worked, then it would prove a number of other statements that we know to be false. (However, I don’t have a way of making that hunch precise.)

The first thing I’m going to do is create a matrix out of , this one indexed by the positive rationals. To do this, for each rational define by the formula , and then let . If is the largest rational that goes into both and , then for a pair and of coprime integers. Then , which is non-zero only if is an integer (since both and are integers and is an integer combination of them). The values of for which is an integer are , where takes the values . These sum to zero unless , by property (iii) of , while if , so , they sum to a constant greater than .

What we have here is a way of expressing the identity matrix indexed by as a linear combination of HAP products with coefficients that are in some sense “small”. The sense in which they are small is that each matrix contributes more to the diagonal than it does to the sum of the absolute values of the coefficients, by a factor of at least .

The hope was to use some kind of limiting argument to show that this clean and tidy infinite situation could be approximated by a less clean and tidy large finite situation. It was there that the trouble arose. I’ll start a new comment to think about it.

Once we have made this correction, we can go on and correct other pairs. Moreover, once we have ensured that the sum along multiples of is zero, we can be careful to avoid those multiples in all future products of intervals that we use — just by making those intervals start a very long way away from 0.

So in a rather trivial way we can find an infinite matrix that’s a linear combination of HAP products with long HAPs and coefficients whose absolute values sum to at most 1, and we can do so in such a way that the sum along the diagonal is at least and the sum along multiples of is 0 whenever and are coprime and not both 1.

I remember getting very excited about this, and then realizing that it doesn’t prove anything. I need to check why it doesn’t imply EDP, since that will make it necessary to impose an extra condition on the matrix .

And now, if we sum along multiples of we get .

I think what was in the back of my mind was that if you want to choose moduli such that choosing even quite random-looking signs results in bounded total oscillation, then you’d better make the moduli have a finite sum.