I have a comment/question concerning the history of the subject. In your blog you speak of the “Steinitz” exchange theorem. Terence Tao uses the same name in his blog, so it seems that this is the common name for this result. Do you know how this name got attached to this result? Who used this name first?

The result of Ernst Steinitz (1871-1928) was published 1913 in Crelles Journal. There he writes that (rough translation follows) “the basics of n-dimensional geometry could have been assumed as well known, but I prefer to rederive them here.” Apparently, the result was already known to Steinitz. Indeed, the result — in almost the exact form as in your blog — already appears in Hermann Grassmann’s Ausdehnungslehre of 1862, a book that was completed in the summer of 1861, i.e. 10 years before Steinitz was born. Steinitz does not cite Grassmann, which is fortunate for him, as now the result is known under his name.

I’m having a go at conveying these things in writing about a relatively infantile problem in number theory. It’s a challenge – I keep on slipping into mathematical-paper mode – but rewarding when it works.

2. Terry, my linear algebra is growing by leaps and bounds, and I am seeing why your comment “As regards the identification between matrices and linear transformations … it’s important to understand both perspectives, and how to swap back and forth.” makes much more sense than my first naive impression.

Week 227 of This Week’s Finds
Neighborhood of Infinity
Orange Juice Files

That’s very interesting. I had vaguely wondered about similar ideas, but I was thinking more about inverses than adjoints. I still haven’t got my head completely round what you’ve written, but will at some stage try to convert it into a very explicit demonstration of how the exchange lemma can yield an algorithm that is more or less the same as Gaussian elimination (but probably applied to columns rather than rows).

The last thing you mentioned reminds me of a fascinating fact that I learned as a result of editing an article on model theory for the Princeton Companion. It’s the most gorgeous proof (which, from what you write, you probably know of) of the following fact: if is an injective polynomial map from to then it must be surjective. The rough idea of the proof is the same as what you say above. By model-theoretic considerations one can show that the statement is true in if and only if it is true in a finite field of sufficiently high characteristic. But there the statement is trivial. It’s a great example of a non-obvious statement that doesn’t seem to have anything to do with logic, but which is proved by logical means. (The proof was discovered by Ax, apparently. For more details, see the PCM when it comes out.)

Another thing that’s interesting about your remark is that it is connected with a question that I like and that I intend to pursue more on this blog, namely what it means for two proofs to be “essentially the same.” It’s an intriguing phenomenon, because it so often happens that two rather different-looking proofs eventually turn out to be less different than they at first appear, as you make very clear in your article, “What is good mathematics?” Given that, it’s all the more interesting to find two proofs that really do seem to be genuinely different. And the model-theoretic argument that vectors in cannot be independent appears to fall into that category (which is not to say that it wouldn’t be worth trying to “expand it out,” just to check).

It seems to me that the key fact that underlies exchange lemma is in fact the adjoint of the key fact that underlies Gaussian elimination; the difficulty you are having in equating the two is thus the same difficulty one encounters when trying to identify a vector space with its dual.

To explain, let me use to denote the real vector space indexed by a finite set A. The exchange lemma algorithm is based on iterating the following:

Lemma 1. If is non-zero, then there exists such that is the direct sum of (which is isomorphic to ) and (i.e. these spaces are complementary).

(Proof: expand in coordinates. There must exist i such that the coordinate of T(1) is non-zero. The claim then easily follows for this choice of i.)

This lemma is what lets you exchange for one of the . (To exchange for one of the , one can first quotient out by the span of and then argue as before.)

In contrast, the ability to use Gaussian elimination to reduce any matrix to row-echelon form is based on iterating the following:

Lemma 2. If is non-zero, then there exists such that is the direct sum of (which is isomorphic to ) and .

(Proof: there must exist a basis element such that is non-zero. One can then express all the other as a linear combination of and something in the kernel of T.)

To see how row-echelon form follows from this, consider n vectors in a standard vector space , and let be defined by setting to be the first coordinate of . If T is non-zero, then by Lemma 2 we can use Gaussian elimination to extract one row with a non-vanishing first coordinate, and n-1 other vectors with vanishing first coordinate; if instead T is zero, then all rows already have vanishing first coordinate. Throwing away and the first column and iterating, one eventually gets row echelon form.

It is not hard to see that Lemma 1 and Lemma 2 are essentially adjoints of each other.

This strongly suggests that the exchange lemma should in fact be related to the use of column operations to place a matrix in column-echelon form. If we let denote the rows of an matrix, column operations correspond to “passive” transformations which change the basis for the vector space without affecting the vectors themselves (in contrast, the “active” row transformations modify those vectors but leave the coordinate basis unchanged). To formulate the exchange lemma in this language, I think what one needs to do is express the r independent vectors , together with the n spanning vectors , as linear combinations of the n spanning vectors , leading to a matrix which has an identity matrix at the bottom. If one applies adjoint Gaussian elimination to this matrix to place it in column-echelon form, I believe that one obtains all of the rows and n-r of the rows as pivot rows (because of the linear independence of the ), and that these pivot rows span the whole space, thus yielding the exchange lemma.

Incidentally, over a finite field F, it is trivial to show by a counting argument that a vector space V spanned by n vectors cannot contain n+1 independent vectors, since the former hypothesis easily implies and the latter hypothesis would imply . One can adapt this counting argument to vector spaces over the reals by various discretisation tricks (e.g. using metric entropy, or Minkowski dimension, or taking a numerical perspective and rounding off to some finite degree of accuracy). Alternatively, one can argue more algebraically (or model-theoretically) by invoking a “Lefschetz principle” or “compactness principle” to equate linear algebra assertions over the reals with linear algebra assertions over fields of sufficiently large characteristic.

It seems difficult (though perhaps not entirely impossible) to recast these arguments in algorithmic form.