CHAPTER 1

STARTERS

In this chapter we introduce some elementary concepts from measure

theory as a start in our examination of the dynamical origin of increasing
entropy. Section A draws a connection between thermodynamic systems
and measure spaces. Section B briefly considers dynamics and phase spaces,
while Section C introduces the notion of a density and postulates that the
state of a thermodynamic system is characterized by a density. In Section
D we introduce the Boltzmann-Gibbs entropy, and prove that it is the
unique (up to a multiplicative constant) entropy definition that satisfies the
physical requirement (additivity) that entropy be an extensive quantity.
A. THERMODYNAMIC SYSTEMS.
In defining a thermodynamic system we need some terms and concepts
that come from measure theory.
We first start with a set X. Measure theorists often like to keep X
pretty abstract, but for us X is going to be the phase space (more about
this in the next section) on which all of our dynamics operates. Sometimes
X will be a closed finite interval like [0, 1], sometimes it may be infinite in
extent like R+, R 6 , or even Rd, and sometimes X is a function space. In
any event, whatever X is we are going to assume that it does not have any
pathological properties.
Given a phase space X we turn to a definition of what measure theorists
call a a algebra. Namely, we let A be a collection of subsets (subspaces) of
X, and say that A is a a algebra if
{1) A E A implies that X\ A E A;
{2) given a sequence {infinite or not) {Ak} of subsets of X, Ak E A,
then UAk E A; and
{3) X EA.
[Note that properties {1) and {3) imply that the empty set 0 is a member
of A since 0 = X \ X.]
The final notion we need for the definition of a thermodynamic system
is that of measure. Any real valued function /-L defined on a a algebra A is
a measure if:
{1) 1-"(0) = 0;
(2) 1-L(A) ~ 0 for all A E A; and
{3) !-L(UkAk) = L:k /-L(Ak) whenever {Ak} is a sequence (infinite or
not) of pairwise disjoint subsets of A, i.e., Ai n Ai = 0 fori i= j.
With the three concepts of a set X, a a algebra A, and a measure /-L we
call the triple (X,A,/-L) a measure space. If, in addition, we can find a
sequence {Ak} of subsets of the a algebra A such that
00

and

M. C. Mackey, Time’s Arrow: The Origins of Thermodynamic Behavior

© Springer Science+Business Media New York 1992
2 TIME'S ARROW: THE ORIGINS OF THERMODYNAMIC BEHAVIOR

then we say that the measure space (X, A, J1.) is a-finite. All of the measure
spaces we consider will be a-finite.
Example 1.1. If we were considering a phase space like X = [0, 1] or
X = R, then a reasonable a algebra would be the· smallest collection of
closed intervals of the form [a, b]. These intervals have Lebesgue measure
Jl.L([a, b]) = b- a. •
Throughout, we will associate a thermodynamic system with a measure
space through the following postulate.
POSTULATE A. A thermodynamic system is equivalent to a
measure space.
Thus, every time we use the term thermodynamic system we are referring
to the triple consisting of a phase space X, a a algebra A, and a measure
Jl..
B. DYNAMICS.
We next consider a thermodynamic system operating in a phase space X.
On this phase space the temporal evolution of our system is described by a
dynamical law St that maps points in the phase space X into new points,
i.e., St: X- X, as timet changes. In general, X may bead-dimensional
phase space, either finite or not, and therefore xis ad-dimensional vector.
Timet may be either continuous (t E R) as, e.g., it would be for a system
whose dynamics were governed by a set of differential equations, or discrete
(integer valued, t E Z) if the dynamics are determined by discrete time
maps.
We only consider autonomous processes in which the dynamics St are not
an explicit function of the timet so it is always the case that St(St'(x)) =
St+t'(x). Thus, the dynamics governing the evolution of the system are
the same on the intervals [0, t'] and [t, t + t']. This is not a serious restric-
tion since any nonautonomous system can always be reformulated as an
autonomous one by the definition of new dependent variables.
Two types of dynamics will be important in our considerations, and
some preliminary discussion will be helpful. Consider a phase space X and a
dynamics St : X - X. For every initial point x 0 , the sequence of successive
points St(x0 ), considered as a function of timet, is called a trajectory. In
the phase space X, if the trajectory St(x 0 ) is nonintersecting with itself,
or intersecting but periodic, then at any given final time t f such that xf =
St 1 (x 0 ) we could change the sign of time by replacing t by -t, and run
the trajectory backward using xf as a new initial point in X. Then our
new trajectory S_t(xf) would arrive precisely back at x 0 after a time t1
had elapsed: x 0 = S_t 1 (xf). Thus in this case we have a dynamics that
may be reversed in time completely unambiguously. Dynamics with this
character are known variously as time reversal invariant {Sachs, 1987) or
reversible {Reichenbach, 1957) in the physics literature, and as invertible
in the mathematics literature.
 1. STARTERS 3

We formalize this by introducing the concept of a dynamical system

{StheR (or, alternately, t E Z for discrete time systems) on a phase space
X, which is simply any group of transformations St : X -+ X having the
two properties:
(1) So(x) = x and
(2) St(St•(x)) = St+t•(x) fort, t' E R or Z.
Since, from the definition, for any t E R, we have

it is clear that dynamical systems are invertible in the sense discussed

above since they may be run either forward or backward in time. Systems
of ordinary differential equations are examples of dynamical systems as are
invertible maps. All of the equations of classical and quantum physics are
invertible.
To illustrate the second type of dynamics, consider a trajectory that
intersects itself but is not periodic. Now starting from an initial point x 0
we find that the trajectory {St(x0 )} eventually has one or more transversal
crossings x.L of itself. If we let t.L be the time at which the first one of
these crossings occurs, and choose our final time t f > t .L, then picking
xf = St 1 (x 0 ) and reversing the sign of time to run the trajectory backward
from xf poses a dilemma once the reversed trajectory reaches x.L because
the dynamics give us no clue about which way to go! Situations like this
are called irreversible in the physics literature, while mathematicians call
them noninvertible.
Therefore, the second type of dynamics that is important to distinguish
are those of semidynamical systems {St}t>O• which is any semigroup of
transformations St: X-+ X, i.e.,
(1) So(x) = x and
(2) St(St•(x)) = St+t'(x) fort, t' E R+ (or N).
The difference between the definition of dynamical and semidynamical sys-
tems lies solely in the restriction of t and t' to values drawn from the posi-
tive real numbers, or the positive integers, for the semidynamical systems.
Thus, in sharp contrast to dynamical systems, semidynamical systems are
noninvertible and may not be run backward in time in an unambigious
fashion. Examples of semidynamical systems are given by noninvertible
maps, delay differential equations, and some partial differential equations.
Often there is a certain confusion in the literature when the terms re-
versible and irreversible are used, and to avoid this we will always use
the adjectives invertible and noninvertible. In spite of the enormous
significance of distinguishing between dynamical and semidynamical sys-
tems later, at this point no assumption is made concerning the invertibility
or noninvertibility of the system dynamics.
4 TIME'S ARROW: THE ORIGINS OF THERMODYNAMIC BEHAVIOR

C. THERMODYNAMIC STATES.
The usual way of examining the dynamics of systems is by studying the
properties of individual trajectories, but in keeping with the ergodic theory
approach adopted here we opt instead to study the way in which the system
dynamics operate on an infinite number of initial points.
More specifically, we will examine the way in which the dynamics alter
densities. What do we mean by a density? If f is an L 1 function in the
space X, i.e., if
[ lf(x)l dx < oo,
then f is a density if f(x) ~ 0 and II f II= 1. As usual, II f II denotes the
L 1 norm of the function/,

II ! II= [ lf(x)l dx.

The examination of the evolution of densities by system dynamics is equiv-

alent to examining the behavior of an infinite number of trajectories. This
apparently simple assumption concerning the way in which systems oper-
ate on densities is so fundamental and important to the understanding of
the foundations of thermodynamics that it is given a special status.
POSTULATE B. A thermodynamic system has, at any given
time, a state characterized by a density f{x), not necessarily in-
dependent of time.
Given a density f then the f measure ~-tr(A) of the set A in the phase

i
space X is defined by
1-Lt(A) = f(x) dx,

and f is called the density of the measure IL1. The usual Lebesgue measure
of a set A is denoted by ILL(A), and the density of the Lebesgue measure
is the uniform density, f(x) = 1/~-tL(X) for all points x in the phase space
X. We always write ~-tL{dx) = dx.
The Lebesgue measure of the entire phase space, denoted by ILL(X), may
either be finite or infinite. If it is finite, we often take it to be normalized
so ILL(X) = 1. It is important to realize that the measure of a set can be
quite different depending on the density f. Thus for example, the Lebesgue
measure of the positive real line R+ is infinite, whereas the measure of R+
with respect to the density f(x) = ke-kx is just

It is instructive to compare the approach used here with that of Boltz-

mann and Gibbs in their treatments of statistical mechanics. Both started
 1. STARTERS 5

from the assumption that they were dealing with systems of dimension
d = 28 whose dynamics were described by 8 position variables Xi and 8
momentum variables Pi· Boltzmann considered the phase space to be a
28-dimensional space which is usually called J.L space. He then consid-
ered the evolution of a large number N of identical particles, each with
the same dynamics, in J.L space. N is large and typically on the order of
Avagadro's number, 6 x 1023 • The limiting case of N-+ oo is the ther-
modynamic limit in which case the Boltzmann approach is equivalent
to studying the evolution of a density in J.L space. Gibbs also considered
N identical particles operating with these 28-dimensional dynamics in a
phase space (commonly called the r space) of dimension 28N. He then
considered an infinite number of copies of this original system, and gave
this construct the name ensemble. Thus Gibbs studies the evolution of
the ensemble density, and r space has proved to be the most useful in
statistical mechanics.
This book is devoted to the study of systems by the evolution of densities,
how system properties (dynamics) determine the character of the density
evolution, and how this is translated into entropy behavior. Later, it will
become clear what types of systems may be described by the evolution of
densities. However, if for now we accept Postulate B that such systems
exist, then it will be easy to examine the consequences of this postulate.
D. BOLTZMANN-GIBBS ENTROPY.
Having postulated that a thermodynamic system has a state character-
ized by a density f, we are now in a position to develop the physically
useful concept of entropy as both Boltzmann and Gibbs introduced the
term.
First we define an observable 0 to be a real number characterizing
some aspect of a thermodynamic system, for example the energy, pressure,
or temperature. As such, an observable corresponds to a map 0 : X -+ R
from the phase space X that yields this number. The expected, or average,
value of the observable 0 is just given by weighting O(x) with the system
state density f(x) and integrating over the entire phase space:

E(O) =< 0 >= [ O(x)f(x) dx.

In his celebrated work Gibbs, assuming the existence of a system state

density f on the phase Space X, introduced the concept of the index of
probability given by log f (x) where "log" denotes the natural logarithm.
Though Gibbs identified -log f with entropy, now it is customary to in-
troduce a quantity H(f) which is the negative of the phase space average
of the index of probability weighted by the density /, i.e.,

H(f) =- L f(x) logf(x) dx.

6 TIME'S ARROW: THE ORIGINS OF THERMODYNAMIC BEHAVIOR

This is now known as the Boltzmann-Gibbs entropy of a density f

since precisely the same expression appears in Boltzmann's work (with the
opposite sign) but the phase space is different for Boltzmann (J.L space)
and for Gibbs (r space). Clearly, the Boltzmann-Gibbs entropy is just
the expectation of the observable defined by the negative of the index of
probability.
As it stands, the definition of the Boltzmann-Gibbs entropy may seem
a bit obscure, and some motivation illustrates why it is the only reason-
able candidate for a mathematical analog of the empirical thermodynamic
entropy. It is easily shown that the only observable which is a function of
a thermodynamic state that gives the requisite additive property to make
the entropy an extensive quantity is the logarithmic function, and that it is
unique up to a multiplicative constant (Khinchin, 1949; Skagerstam, 1974).
To be more specific, consider two systems A and B operating in the
phase spaces XA and XB, respectively, and each having the densities of
states fA and fB· We now combine the two systems to form a new system
C operating in the product space Xc = XA x XB, so system C will have
a density of states fc(x, y) = !A(x)JB(Y) if A and B do not interact.
Experimentally we expect that when the two systems are combined into
a larger system C, then the entropy of system C should equal the sum of
the individual entropies of A and B, since entropy is generally held to be
an extensive system property. We wish to show that the Gibbs choice for
the index of probability is the only choice (up to a multiplicative constant)
that will ensure this.
To demonstrate this, assume that the index of probability is left as an
unspecified observable 0. If the observable 0 is such that it transforms
products to sums, O(fc) = O(!A!B) = O(!A) + 0(/B), then the relation
H(!A) + H(!B) = H(fc) holds. It is certainly clear that picking O(w) =
dlogw, where dis any arbitrary nonzero constant, will work but are there
any other functions 0 with the requisite property?
Assume there exists a second continuous observable 6 such that

Define two new functions vA (a) and v8 (b) through

!A(a) = evA(a) and !B(b) = evB(b).
Then we have
O(evA+vB) = O(evA) + O(evB),
or with h(w) = O(ew) this becomes
h(VA +VB)= h(vA) + h(vB)·
This, however, is just the famous Cauchy functional equation that has the
unique solution
h(w) = 8w with 8 an arbitrary constant
 1. STARTERS 7

0 X

Figure 1.1. The graph of 71(x) as a function of x. The solid line tangent
to the curve at x = v has slope 71'(x), while the dashed line connecting the
points v and w has slope [71(w)- 'l(v)]/(w- v). This illustrates the origin
of inequality (1.3).

(Kuczma, 1985). This in turn implies that O(ew) = 8w so

O(w) = 8logw.
Thus the only observable that gives the requisite additive property for the
Boltzmann-Gibbs entropy is the logarithmic function and it is unique up
to a multiplicative constant.
We formalize the entropy definition by saying that the Boltzmann-
Gibbs entropy of the density f is defined by

H(f) = L "1(/(x)) dx, (1.1)

where the function "1( w) is given by

"'(w) = { -wlogw, w > 0, (1.2)

0, w=O.
The function "1( w) has properties that are important for demonstrating
properties of the entropy H (see Figure 1.1). First, note that 71(w) is
continuous for w ~ 0 since 77(0) = 0 by definition. Further, 71"(w) = -w- 1
so 71( w) is strictly concave since 71" (w) < 0 for all w > 0. From this it is an
immediate consequence (see Figure 1.1) that

71(w) ~ (w- v)71'(v) + 71(v) (1.3)

for all w, v > 0. Combining Eq. (1.2) defining 71 with inequality (1.3) gives
the Gibbs inequality

w- wlogw ~ v- wlogv for w,v > 0. (1.4)

8 TIME'S ARROW: THE ORIGINS OF THERMODYNAMIC BEHAVIOR

Integrating Eq. {1.4) gives another useful inequality. If I and g are two den-
sities, and TJ{f(x)) and TJ(g(x)) are both integrable, then we have directly
from Eq. {1.4) that

- Ll(x)logl(x)dx ~- Ll(x)logg(x)dx, {1.5)

which we call the integrated Gibbs inequality. Only when I = g does

the equality hold.
E. SUMMARY.
This chapter has introduced some key concepts and postulates neces-
sary in our examination of the foundation of evolution of entropy. We have
identified thermodynamic systems with measure spaces and states of ther-
modynamic systems with densities. We have also defined the Boltzmann-
Gibbs entropy and proved it is the unique (up to a multiplicative constant)
definition of entropy that is consistent with the physically motivated re-
quirement of entropy additivity. In the next chapter we turn our attention
to maximal entropy principles and their consequences.