Maximum entropy from Bayes’ theorem

The principle of maximum entropy asserts that when trying to determine an unknown probability distribution (for example, the distribution of possible results that occur when you toss a possibly unfair die), you should pick the distribution with maximum entropy consistent with your knowledge.

The goal of this post is to derive the principle of maximum entropy in the special case of probability distributions over finite sets from

• Bayes’ theorem and
• the principle of indifference: assign probability to each of possible outcomes if you have no additional knowledge. (The slogan in statistical mechanics is “all microstates are equally likely.”)

We’ll do this by deriving an arguably more fundamental principle of maximum relative entropy using only Bayes’ theorem.

Read Full Post »

Small factors in random polynomials over a finite field

Previously I mentioned very briefly Granville’s The Anatomy of Integers and Permutations, which explores an analogy between prime factorizations of integers and cycle decompositions of permutations. Today’s post is a record of the observation that this analogy factors through an analogy to prime factorizations of polynomials over finite fields in the following sense.

Theorem: Let be a prime power, let be a positive integer, and consider the distribution of irreducible factors of degree in a random monic polynomial of degree over . Then, as , this distribution is asymptotically the distribution of cycles of length in a random permutation of elements.

One can even name what this random permutation ought to be: namely, it is the Frobenius map acting on the roots of a random polynomial , whose cycles of length are precisely the factors of degree of .

Combined with our previous result, we conclude that as (with tending to infinity sufficiently quickly relative to ), the distribution of irreducible factors of degree is asymptotically independent Poisson with parameters .

Read Full Post »

Short cycles in random permutations

Previously we showed that the distribution of fixed points of a random permutation of elements behaves asymptotically (in the limit as ) like a Poisson random variable with parameter . As it turns out, this generalizes to the following.

Theorem: As , the number of cycles of length of a random permutation of elements are asymptotically independent Poisson with parameters .

This is a fairly strong statement which essentially settles the asymptotic description of short cycles in random permutations.

Read Full Post »

Fixed points of random permutations

The following two results are straightforward and reasonably well-known exercises in combinatorics:

1. The number of permutations on elements with no fixed points (derangements) is approximately .
2. The expected number of fixed points of a random permutation on elements is .

As it turns out, it is possible to say substantially more about the distribution of fixed points of a random permutation. In fact, the following is true.

Theorem: As , the distribution of the number of fixed points of a random permutation on elements is asymptotically Poisson with rate .

Read Full Post »

Noncommutative probability and group theory

There are, roughly speaking, two kinds of algebras that can be functorially constructed from a group . The kind which is covariantly functorial is some variation on the group algebra , which is the free -module on with multiplication inherited from the multiplication on . The kind which is contravariantly functorial is some variation on the algebra of functions with pointwise multiplication.

When and when is respectively either a discrete group or a compact (Hausdorff) group, both of these algebras can naturally be endowed with the structure of a random algebra. In the case of , the corresponding state is a noncommutative refinement of Plancherel measure on the irreducible representations of , while in the case of , the corresponding state is by definition integration with respect to normalized Haar measure on .

In general, some nontrivial analysis is necessary to show that the normalized Haar measure exists, but for compact groups equipped with a faithful finite-dimensional unitary representation it is possible to at least describe integration against Haar measure for a dense subalgebra of the algebra of class functions on using representation theory. This construction will in some sense explain why the category of (finite-dimensional continuous unitary) representations of behaves like an inner product space (with being analogous to the inner product); what it actually behaves like is a random algebra, namely the random algebra of class functions on .

Read Full Post »

Moments, Hankel determinants, orthogonal polynomials, Motzkin paths, and continued fractions

Previously we described all finite-dimensional random algebras with faithful states. In this post we will describe states on the infinite-dimensional -algebra . Along the way we will run into and connect some beautiful and classical mathematical objects.

A special case of part of the following discussion can be found in an old post on the Catalan numbers.

Read Full Post »

Finite noncommutative probability, the Born rule, and wave function collapse

The previous post on noncommutative probability was too long to leave much room for examples of random algebras. In this post we will describe all finite-dimensional random algebras with faithful states and all states on them. This will lead, in particular, to a derivation of the Born rule from statistical mechanics. We will then give a mathematical description of wave function collapse as taking a conditional expectation.

Read Full Post »

Noncommutative probability

The traditional mathematical axiomatization of probability, due to Kolmogorov, begins with a probability space and constructs random variables as certain functions . But start doing any probability and it becomes clear that the space is de-emphasized as much as possible; the real focus of probability theory is on the algebra of random variables. It would be nice to have an approach to probability theory that reflects this.

Moreover, in the traditional approach, random variables necessarily commute. However, in quantum mechanics, the random variables are self-adjoint operators on a Hilbert space , and these do not commute in general. For the purposes of doing quantum probability, it is therefore also natural to look for an approach to probability theory that begins with an algebra, not necessarily commutative, which encompasses both the classical and quantum cases.

Happily, noncommutative probability provides such an approach. Terence Tao’s notes on free probability develop a version of noncommutative probability approach geared towards applications to random matrices, but today I would like to take a more leisurely and somewhat scattered route geared towards getting a general feel for what this formalism is capable of talking about.

Read Full Post »

Lower bounds on off-diagonal Ramsey numbers

The goal of this post is to prove the following elementary lower bound for off-diagonal Ramsey numbers (where is fixed and we are interested in the asymptotic behavior as gets large):

The proof does not make use of the Lovász local lemma, which improves the bound by a factor of ; nevertheless, I think it’s a nice exercise in asymptotics and the probabilistic method. (Also, it’s never explicitly given in Alon and Spencer.)

Read Full Post »

Ultrafilters in topology

Remark: To forestall empty set difficulties, whenever I talk about arbitrary sets in this post they will be non-empty.

We continue our exploration of ultrafilters from the previous post. Recall that a (proper) filter on a poset is a non-empty subset such that

1. For every , there is some such that .
2. For every , if then .
3. is not the whole set .

If has finite infima (meets), then the first condition, given the second, can be replaced with the condition that if then . (This holds in particular if is the poset structure on a Boolean ring, in which case .) A filter is an ultrafilter if in addition it is maximal under inclusion among (proper) filters. For Boolean rings, an equivalent condition is that for every either or , but not both. Recall that this condition tells us that ultrafilters are precisely complements of maximal ideals, and can be identified with morphisms in . If for some set , then we will sometimes call an ultrafilter on an ultrafilter on (for example, this is what people usually mean by “an ultrafilter on “).

Today we will meander towards an ultrafilter point of view on topology. This point of view provides, among other things, a short, elegant proof of Tychonoff’s theorem.

Read Full Post »