While Boltzmann had sidestepped the issue of determinism in the debate on the recurrence paradox, maintaining a somewhat ambiguous "statistical" viewpoint, he had to face the issue more squarely in another debate that came to a head at almost the same time he faced the debate about the Poincare Recurrence Theorem. E. P. Culverwell in Dublin had raised, in 1890, what might be called the "reversibility objection to the H theorem," not to be confused with the "reversibility paradox" discussed by William Thomson, Loschmidt, and Boltzmann in the 1870s. Culverwell asked how the H-theorem could possibly be valid as long as it was based on the assumption that molecular motions and collisions are themselves reversible, and suggested that irreversibility might enter at the molecular level, perhaps as a result of interactions with the ether.
The ether was always available as a hypothetical source and sink for properties of matter and energy that didn't quite fit into the framework of Newtonian physics, although some physicists were by this time quite suspicious of the tendency of their colleagues to resolve theoretical difficulties this way.
Culverwell's objection was discussed at meetings of the British Association and in the columns of Nature during the next few years. It was S. H. Burbury in London who pointed out, in 1894, that the proof of the H-theorem depends on the Maxwell-Boltzmann assumption that colliding molecules are uncorrelated. While this would seem a plausible assumption to make before the collision, one might suppose that the collision itself introduces a correlation between the molecules that have just collided, so that the assumption would not be valid for later collisions. Burbury suggested that the assumption might be justified by invoking some kind of "disturbance from without [the system], coming at haphazard" [Burbury 1894, p. 78].
Boltzmann, who participated in the British discussions of the H theorem, accepted Burbury's conclusion that an additional assumption was needed, and called it the hypothesis of "molecular disorder." He argued that it could be justified by assuming that the mean free path in a gas is large compared with the mean distance of two neighboring molecules, so that a given molecule would rarely encounter again a specific molecule with which it had collided, and thus become correlated (see Boltzmann 1896-1898, pp. 40-41).
"Molecular disorder" is not merely the hypothesis that states of individual molecules occur completely at random; rather it amounts to an exclusion of special ordered states of the gas that would lead to violations of the Second Law. In fact such ordered states would be generated by a random process, as Boltzmann noted in his discussion of the recurrence paradox.
In modern terminology, one makes a distinction between "random numbers" and "numbers generated by a random process" -- in preparing a table of random numbers for use in statistical studies, one rejects certain subsets, for example pages on which the frequencies of digits depart too greatly from 10%, because they are inconveniently-nonrandom products of a random process.
Boltzmann recognized that the hypothesis of molecular disorder was needed to derive irreversibility, yet at the same time he admitted that the hypothesis itself may not always be valid in real gases, especially at high densities, and that recurrence may actually occur.
In view of Boltzmann's partial abandonment of determinism on the molecular level, we must reconsider the view that 19th-century physicists always assumed determinism and used statistical methods only for convenience.
There is no doubt that some 19th-century thinkers did see determinism as the essence of science. Thus W. Stanley Jevons, a philosopher of science, wrote in 1877:
"We may safely accept as a satisfactory scientific hypothesis the doctrine so grandly put forth by Laplace, who asserted that a perfect knowledge of the universe, as it existed at any given moment, would give a perfect knowledge of what was to happen thenceforth and for ever after. Scientific inference is impossible, unless we may regard the present as the outcome of what is past, and the cause of what is to come. To the view of perfect intelligence nothing is uncertain." [Jevons 1877, pp. 738-39]
Hence, as Laplace himself had remarked in 1783 (see Gillispie 1972, p. 10), there is really no such thing as "chance" in nature, regarded as a cause of events; it is merely an expression of our own ignorance, and "probability belongs wholly to the mind" [Jevons 1877, p. 198].
But was this view really held by scientists themselves? By the time Jevons wrote the words quoted above, support for absolute determinism was already beginning to collapse. In arguing for some degree of continuity between the 19th and 20th centuries, I do not want to overstate the case; 20th-century events (including the discovery of radioactive decay, though it actually occurred just before 19OO) accounted for most of the impetus toward atomic randomness, while the 19th-century background accounted for a significantly smaller amount. Nevertheless the discussion of randomness and irreversibility in connection with kinetic theory and the Second Law of Thermodynamics was quite familiar to physicists in the early decades of the 20th century.
The claim that 19th-century kinetic theory was based on molecular determinism must rely heavily on the evidence of the writings of James Clerk Maxwell and Ludwig Boltzmann; though in the absence of any explicit statements one might legitimately infer that they tacitly accepted the view of their contemporaries. In fact as we have already seen in the case of Boltzmann, the situation is a little more complicated: the words were ambiguous but the equations pushed physical theory very definitely in the direction of indeterminism. As in other transformations of physical science -- the cases of Kepler, Fresnel, Planck, and Heisenberg might be adduced here -- mathematical calculation led to results that forced the acceptance of qualitatively different concepts.
Maxwell's earliest work in kinetic theory, in particular his introduction of the velocity-distribution law, seems to derive from the tradition of general probability theory and social statistics (as developed by Adolphe Quetelet) rather than from the mechanistic analysis of molecular motions. Maxwell's law asserts that each component of the velocity of each molecule is a random variable, which is statistically independent of every other component of the same and every other molecule. Only in his later papers did Maxwell attempt to justify the law by relating it to molecular collisions, and even then he needed to assume that the velocities of two colliding molecules are statistically independent. On the other hand, the computation of gas properties such as viscosity and thermal conductivity, whose comparison with experimental data provided the essential confirmation of the theory, did involve the precise dynamical analysis of collisions of particles with specified velocities, positions, and force laws. Without determinism in this part of the theory Maxwell could not have achieved his most striking successes in relating macroscopic properties to molecular parameters.
Maxwell did not consistently maintain the assumption of determinism at the molecular level, though he occasionally supported that position, for example, in his lecture on "Molecules" at the British Association meeting in 1873. Yet in the same year, in private discussions and correspondence, he began to repudiate determinism as a philosophical doctrine. A detailed exposition of his views may be found in a paper titled "Does the progress of physical science tend to give any advantage to the opinion of necessity (or Determinism) over that of the contingency of events and the Freedom of the Will?" presented to an informal group at Cambridge University. The answer was no -- based on arguments such as the existence of singular points in the trajectory of dynamical systems, where an infinitesimal force can produce a finite effect. (These arguments have led some contemporary scientists to list Maxwell as one of the precursors of "chaos theory.") The conclusion was that "the promotion of natural knowledge may tend to remove that prejudice in favor of determinism which seems to arise from assuming that the physical science of the future is a mere magnified image of that of the past" [Campbell & Garnett 1882, p. 434].
By 1875 Maxwell was asserting that molecular motion is "perfectly irregular; that is to say, that the direction and magnitude of the velocity of a molecule at a given time cannot be expressed as depending on the present position of the molecule and the time" [Maxwell 1875a, p. 235]. He also stated that this irregularity must be present in order for the system to behave irreversibly [Maxwell 1875b].
Two decades later, as noted above, Boltzmann seemed to have reached a similar conclusion. But he was not quite satisfied that his hypothesis of molecular disorder resolved the reversibility and recurrence paradoxes; in response to further criticisms by Zermelo he proposed a new hypothesis. Suppose we consider the curve of H as a function of time for the entire universe, or for a part of the universe isolated from the rest. A high value of H will correspond to a low-entropy highly-ordered state, where life can exist. If the recurrence theorem is correct then such a state can be regarded as one of an infinite number of maxima of an oscillating curve. If we follow H forward in time from one of these peaks, it will decrease in accordance with the H theorem; but it must eventually increase again to get to the next peak. Such an epoch of increasing H (decreasing entropy) would seem to violate the Second Law. But, Boltzmann suggested, if the irreversible processes in our environment and in our own bodies are "running backwards" then our own sense of the direction of time must also be reversed. Thus for any conscious beings who exist during this epoch, H must decrease when measured with respect to the time-changes of those beings, so for them the Second Law still holds.
Although Boltzmann did not regard this proposal as any more than a speculative hypothesis, he justified it as follows:
"One has the choice of two kinds of pictures. One can assume that
the entire universe finds itself at present in a vry improbable
state. However, one may suppose that the eons during which this
improbable state lasts, and the distance from here to Sirius, are
minute compared to the age and size of the universe. There must
then be in the universe, which is in thermal equilibrium as a whole
and therefore dead, here and there relatively small regions of the
size of our galaxy (which we call worlds), which during the
relatively short time of eons deviate significantly from thermal
equilibrium. Among these worlds the state probability increases as
often as it decreases. For the universe as a whole the two
directions of time are indistinguishable, just as in space there is
no up or down. However, just as at a certain place on the Earth's
surface we can call "down" the direction toward the centre of the
Earth, so a living being that finds itself in such a world at a
certain period of time can define the time direction as going from
less probable to more probable states (the former will be the
"past" and the latter the "future") and by virtue of this
definition he will find that this small region, isolated from the
rest of the universe, is "initially" always in an improbable state.
This viewpoint seems to me to be the only way in which one can
understand the validity of the Second Law and the Heat Death of
each individual world without invoking a unidirectional change of
the entire universe from a definite initial state to a final
state."
[Boltzmnn 1897, p. 242]
Boltzmann's hypothesis asserts that irreversibility -- the statement that "entropy increases with time" is not a law of nature but a tautology: the direction of time is determined by the direction of entropy increase. (Curiously this idea had recently been advanced by Ernst Mach, the most famous critic of Boltzmann's kinetic-atomic theories.) Alternatively it could be seen as foreshadowing Einstein's idea that time is not absolute but is somehow relative to the observer.