a-cosmology-group/acg#137

In https://arxiv.org/pdf/1908.08374.pdf, an otherwise eye-openning revisiting of Shrödinger’s What is Life, Rovelli explains in detail why, in his words:

The strict equivalence order = low entropy is a mistake.

What Rovelli is talking about is not entropy. We don’t know enough about the processes involved in “life” to derive fundamental laws of physics from it. He ‘short-circuits’ many levels of the hierarchy Feynman is talking about, a big no-no. Also, Rovelli makes the mistake of assuming that the Earth (where life is found), is a closed system. Earth receives massive amounts of radiation from the sun and re-radiates it as infrared light into space, it’s far from a closed system.

Order has increased on Earth, entropy has decreased, and this also happens with galaxies at Mpc and even Gpc scales. The problem is not with thermodynamics but with the simplifying assumption that some natural systems are closed when they are not.

The question is, what does entropy mean? I have never heard a clear, concise, and unambiguous definition among the innumerable ones on offer. It appears to have been originally conceived in the 19th century as a description of a preconfigured, closed thermodynamic system settling into equilibrium. In that context it makes some sense but out in the wilds of physical reality there are no closed systems - apart from the minds of Big Bang cosmologists.

“Entropy” appeared with the study of steam engines to make them more efficient. It turns out that entropy is also meaningful for gas engines, Brownian motion, the blackbody spectrum, photon in a laser, etc. What is common between a ‘steam engine’ and a ‘laser’? The concept appears everywhere because it is about the probability of configurations of a given system, where interactions outside the system are neglected (hence a “closed” system).

A simple definition is to consider how probable the state is. If it’s highly probable, it has a high entropy; if it’s not very probable, it has a low entropy.

Example: glasses eventually end up shattered into several pieces - this most probable state has a high entropy.

Example 2: a 52-card pack and its dealer is a closed system, and in poker

• a Royal Flush has very low entropy as this is not a very probable hand,
• 1 Pair has higher entropy because this is a more probable hand.

Entropy is not a fundamental physical quantity, it depends on the chosen system (which is an arbitrary choice made for convenience, e.g. study a steam engine). Thermodynamics works only if the system is closed. There are no perfectly closed physical systems, but some systems are isolated for a long enough time to behave almost as if they were closed. E.g. it takes 50 ms to expand the steam across the size of the piston of an engine, a time too short for a significant energy exchange between the steam and the walls of the engine - so the steam engine is a good approximation to a closed system.

However, the light, plasma and gases emitted by a galaxy can travel across distances that are much greater than the size of the galaxy. Stars and plasmas don’t shuffle like the cards in a deck.

I agree @budrap00, there is no closed system at cosmological scales.

Since “entropy” is a high level construct far above fundamental ideas, it cannot be used to explain something as fundamental as time. Perhaps that’s Rovelli’s motivation for redefining “entropy” as “relative entropy”, but the concept of “relative entropy” is still too far from (and above!) the fundamental nature of “time”. We don’t have sufficient knowledge to short-circuit so many levels in the “hierarchy of ideas”, and in any case it is silly to try to ‘explain’ a fundamental concept like time with a less fundamental concept like entropy.

I agree that entropy is way beyond the fundamental essence of time. Rovelli says a much (see The Order of Time). Yet in modern physics it is the only concept related to irreversibility, precisely because time is taken as granted from the get go.

Rovelli does not consider the Earth as a closed system, he explicitly says that the Sun contributes low entropy, which “makes everything go” down here (in the words of Feynman’s father).

Entropy has a very precise definition in statistical mechanics, as the (log) number of system microstates that are equivalent in terms of the specified macroscopic observables.

Rovelli’s significant contribution here is the concept of relative entropy: the count of equivalent microstates depends on the observing system, of the specified macroscopic variables. In other words, entropy is a measure of what the observing system does NOT know about the observed system; a direct link to Shannon’s entropy definition in information theory, which is equivalent to the one in physics.

In terms of the proverbial deck of cards, the measure of entropy is different, for example, whether the observer does not distinguish colour or distinguishes only colour: the number of equivalent microstates are grouped differently.

This becomes so obvious to me now, I am surprised and ashamed I did not think of it this way when I learned about entropy in my stat mech classes. I peg that shortcoming to the extremely narrow “gas in a box at rest” context of stat mech training.

Adding the missing “relative” part of the definition of entropy is significant when working outside the box, so to speak! Entropy is the (log) number of indistinguishable microstates indeed: but indistinguishable by which “observer”? This has to be specified.

There is a deep connection here with quantum mechanics, in terms of “what can be known”, in principle.

@RedshiftDrift , I don’t really disagree with what you say except to point out that you didn’t offer a clear concise and unambiguous definition of the term entropy, you just gave one example (statistics) among the myriad ways the term is deployed scientifically. It can be deployed under many different scenarios because it is not clearly defined. All you have to do is note the many “definitions” that appear in this Wiki article.

A side bar from the Wiki entry shows that John von Neumann understood the situation:

I thought of calling it "information", but the word was overly used, so I decided to call it "uncertainty". [...] Von Neumann told me, "You should call it entropy, for two reasons. In the first place your uncertainty function has been used in statistical mechanics under that name, so it already has a name. In the second place, and more important, nobody knows what entropy really is, so in a debate you will always have the advantage.

Conversation between Claude Shannon and John von Neumann regarding what name to give to the attenuation in phone-line signals[80]

Also, while there are certainly many examples of human constructed “approximately” closed systems there are no examples of approximately closed systems occurring in physical reality at any scale that I know of.

Ah yes, it is Von Neumann that suggested entropy in information theory!! Thanks for the reminder.

There is no ambiguity with the quantity called entropy today. It is defined as:

$\large S = -k_{\text{B}} \sum_{i}p_{i}\ln p_{i}$

(The Boltzman constant is a convention regarding units).

People using the concept loosely in various contexts does not mean that there is not rigorously defined. Just like “energy”, it is not because we use it colloquially that it is an imprecise concept.

@ftessier That’s just one of the myriad definitions. It may be your favorite but it is clearly not the only one on offer. Here’s some more (from the same Wiki article):

“_The following is a list of additional definitions of entropy from a collection of textbooks:

a measure of energy dispersal at a specific temperature. a measure of disorder in the universe or of the availability of the energy in a system to do work.[66] a measure of a system’s thermal energy per unit temperature that is unavailable for doing useful work.[67] In Boltzmann’s analysis in terms of constituent particles, entropy is a measure of the number of possible microscopic states (or microstates) of a system in thermodynamic equilibrium._“

Note that those definitions are from textbooks, not casual usage. von Neumann was right. The concept of entropy is vague and ill-defined in physics - nobody knows what it means.

All definitions of entropy rooted in thermodynamics are superseded with the statistical mechanics definition, since statistical mechanics supersedes thermodynamics.

As for the “disorder” definition, it is simply incomplete if order is not precisely defined. Here it is “order” that is ill-defined, not entropy.

As Rovelli notes, balls at rest at the bottom of a box are arguably more ordered than balls bouncing around in said box. Yet as the balls slow down, increasing order, entropy increases. Mixed oil and water separate into ordered layers, yet entropy is increased in the process. Entropy is NOT spatial disorder.

I don’t mean to be dismissive about the other “definitions” of the term. Of course one can call entropy whatever they want. I would argue it is even a useful catch-all for “disorder” in casual conversation: “My children increased entropy in the living room!” 😄

I wanted to explain that, in physics (and information theory), entropy is rigorously defined as: $\large S = -k_{\text{B}} \sum_{i}p_{i}\ln p_{i}$ (with $\large k_\text{B}$ an application-dependent constant, defining the units). I am not saying that the concept shall not be refined further, quite the contrary; case in point, its observer-dependent aspect. I am saying that this is, unless otherwise noted, what is meant by entropy by physicists today (at least the ones with whom I have lunch 😆). All thermodynamic definitions reduce to that.

As for von Neumann, one of the Martians and arguably one of the most intelligent human being who ever lived (subjective, I know, but still…), I can only presume that he meant that nobody ELSE knows what it means. He obviously stood by this rigorous definition, enough to encourage Shannon to use the term entropy for this very precise mathematical meaning.

The “other definitions” on the Wiki page are all equivalent. They are just formulated in terms of different observables.

If the discussion is limited to closed-thermodynamic systems I agree - mathematically. But the concept has no clear, coherent, unambiguous physical meaning which is how it winds up in biology, economics, cosmology, philosophy and of course, theoretical physics. Something with no meaning can mean anything.

It’s not because people do bad science that scientific concepts have no meaning. Extrapolating ‘entropy’ to philosophy and politics is bad science because it goes beyond the domain of applicability of entropy (a closed system).

Agreed that entropy increases in a closed system. However, entropy AND spatial order can increase in a closed system, at the same time. In the oil-water mixture mentioned above, the oil and water would spontaneously separate even in the absence of a heat bath and in the absence of gravity (although in the latter case the oil would not necessarily be above the water, of course). This occurs for entropic reasons, as inter-molecular potential energy is transferred to momenta degrees of freedom.

In other words, the mixture will equilibrate with a heat bath (and adopt a layered structure in a gravitational field), but spatial order does not arise because of it.

Entropy is not disorder in coordinates, it is disorder in phase space, so to speak. This is often overlooked in structure formation, even in (idealized) closed systems. For a long time I thought that organism could only achieve structure by releasing heat in their surroundings, but that is incorrect. Heat does not have to be exchanged for structure to form.

Living systems go against entropy

This is incorrect. If living systems went against entropy, they would be so unlikely as to never arise! Life’s processes and structures are keys to gateways that increase entropy.

Thanks @HanDeBruijn but I’ve already read the book in the original language. Besides, after learning English and German, Spanish is more likely to become my fourth language… I think the next “cosmological revolution” will first be embraced by the Spanish culture, but that’s just a hunch.

How so? Please provide an example.

The existence of complex life form development (reptiles,birds,mammals) require complex biological processes like respiration,digestion and circulation that can only function by proceeses modifying entropy.

Indeed, all these processes increase entropy. All these organism would not exist (or continue to exist) otherwise. Entropy is a measure of probability, and system on a large enough scale (above thermal fluctuations) go from less probable towards more probable configurations.

The statement is true, living systems increase entropy performing certain biological processes