Idle Analytics

Here I've been blabbering on and on about entropy, but all I've really talked about is changes in entropy. Irreversible things miss opportunities to do work, and that makes entropy go up. But what is the entropy of, for instance, a bottle of gas? It has pressure, temperature, volume, and internal energy. If the gas is being used in an engine, all of those quantities get reset by the end of the cycle. And entropy change for the engine is also zero, which indicates that there's an absolute entropy . For a certain jar of gas maybe it's 100 J/K---whatever, there should be a number!

It really ought to be simple enough. If we find the conditions under which is zero, then we can just add heat and calculate what is.

The thing is, it's fairly easy to say what it means that , , , or is zero. is the internal energy, so if all the atoms are standing still, . Since is in some way a measure of , it is also zero when the atoms aren't moving. If the atoms aren't pushing on anything, their pressure is zero. And if the atoms are in an extremely small space, the volume is zero.

But what should the entropy be when ? I suppose we could say it's 32 J/K, but let's be sensible and say that entropy should be zero when .

Seems simple enough. After all, where else would it be zero? You certainly can't take heat out of something where none of the atoms are moving. Ok, so if K, the entropy is zero.

Before we start talking about what the consequences of the 3rd law are, I need to clear something up about heat capacity. I said that the heat capacity is the amount of heat you have to add to raise the temperature by one degree. And then I stated some numbers for a few substances. I said that for liquid water the heat capacity was 1 cal per cm per K. But that's only partly true. It's 1 calorie at 289 K, but there's no reason to think it's the same for all temperatures. And in fact, it's not.

The graph above shows the heat capacity of measured by experiments (at constant volume). The units here are calories per mole per Kelvin instead of per cubic centimeter. There are several interesting things about this. First is that the liquid phase can hold a lot more heat per mole than either ice or steam can. This is what we thought might happen when we discussed this earlier. But also look what is happening as we lower the temperature of ice toward 0. The heat capacity is going to zero! And pretty rapidly too.

The heat capacity going to zero in ice is not a fluke; heat capacity goes to zero no matter what the substance is.

So, no matter what substance it is, near absolute zero the heat capacity is very small, and you can get a huge change in temperature from adding a small amount of heat.

Now, suppose we are trying to cool something down to absolute zero. We do this, of course, by way of a refrigerator, which takes in work and moves heat from the cold object (what we're trying to cool down) to another heat bath at higher temperature. The efficiency with which it does this depends on the actual refrigerator you use, but let's use the best possible one: a Carnot engine in reverse. The Carnot engine run in reverse between two temperatures takes in work and moves removes heat from the cold bath. The amount of heat can be shown to be the efficiency of the refrigerator times the work:

On the other hand, removing that much heat reduces the temperature of the cold bath by the heat capacity times the change in .

So the amount we reduce the cold bath temperature by is

This is a conundrum. What it says is that if I put work into my refrigerator every cycle, I get a small change in the temperature of the cold bath, but it's proportional to divided by , the heat capacity. But both of these are going to zero! So it's a competition: if approaches 0 more rapidly than does, then it is impossible to reach absolute zero, because will just keep getting closer and closer to 0. On the other hand, if goes to zero faster, then it's very easy to reach absolute zero. I do not know of a simple argument for why the heat capacity goes more slowly, as it is a legitimately highly technical question (A decent treatment of the issues involved can be found in Ernest M. Loebl, Jour. Chem. Ed. 37,7 (1960)). Suffice it to say that remains finite lower down that does, and so the amount we cool the object gets less and less every time. The consequence is that absolute zero can never be reached.

Not that people don't try. The lowest anyone has ever gotten was in 2008, when a lab in Helsinki cooled some rhodium metal down to 0.0000000001 K. People regularly get down to 0.001 K, and even 0.000001 K. But that's as good as you can do. Some people (including myself) think that the 3rd Law may as well just be this.

In any case you don't actually need to get to absolute zero to tell what the absolute entropy of something is. Just knowing that it should be zero at gives us a reference point so that we could actually state how much entropy a given system has. A gas of molecules confined in volume , with internal energy , has, for instance an entropy equal to

Ok, ok, I know this looks horrendous, but now you've seen it and you know that we can say with honesty that a given object has a certain amount of entropy, so that we don't just have to talk about changes.

I have to admit, I've been fibbing a bit. It is absolutely true that changing something's temperature by heating changes its entropy by . However, this is not the only way to change it. In fact, anything that's irreversible increases the entropy. And there are ways to decrease the entropy of a system without taking heat out of it (though there is no way to decrease the entropy of the entire universe).

Last time we talked about connecting a bottle full of gas to an empty bottle so that some of the gas would flow into the previously empty one, and that this was irreversible. As such there should be an increase in entropy from this; and indeed, there is. (Remember: something is irreversible if, watching a video of it going backwards, you see something that never happens.)

So for 2 identical volumes, the gas expands but gains an entropy . And the universe also gains that much, and you can never undo that, unless you want to wait a literally unimaginable time.

Now, as I said last time, you can force the gas to go back into the original bottle. To do that you have to do work, and to do work you have to use an engine (I use the term generally. A human can be considered an engine, anything that takes in heat and makes work is an engine). "Fine", you say, "I'll use a Carnot engine. The Carnot engine doesn't change the entropy of the outside at all, and by using the engine we press it back in and everything's just the same as it was.". Sorry, no. If you compress the gas, you do not put it back in the original state, because when you compress it you do work on it, which will heat it up. So it will be at a higher temperature, and to return it to where it was initially you have to reject some heat to the environment, which again means that the entropy of the universe is permanently higher than it was before you let the gas expand.

Entropy increase is permanent for the universe. It's a one-way street; once entropy is created, it simply cannot be undone.

Here's something to think about. If we pull out a piston with gas behind it, we have not done the same thing we just described. We did not let the gas freely expand, because we made sure that it did work as it expanded. The gas would cool, it's reversible. However, if we pull the piston out too fast, then it is like free expansion! So even in things that we think are reversible, we have to be careful to go slowly enough that the gas molecules can keep up; they have to sort of come to equilibrium before we keep moving on.

Another way to do something irreversible is to start a chemical reaction. Burning a hydrocarbon like methane (gas) in oxygen (gas) makes carbon dioxide (gas) and water (vapor), and releases a lot of heat. The reaction is irreversible. Now, modeling this system with a calculation would be hopeless, so I can't calculate from first principles how much the entropy will go up. But these can be looked up in tables, measured by people over the years. You can see a table of them, for instance, here. For each mole, methane () has 186.3 J/K, oxygen () has 205.1 J/K, (liquid) has 69.9 J/K, and carbon dioxide () has 213.7 J/K. (All of these values are for room temperature.) The reaction is

If you remember your chemistry, you know that this equation is not balanced, meaning that there are not as many oxygen atoms on the left side as the right side, not as many hydrogen atoms, etc. I.e., this is not the reaction that happens! Only once we balance it will we know how many moles of each actually go into a reaction. Ok, the right hand side has 3 oxygens, and the left hand has 2, so if we say that it makes two water molecules instead of 1, and 2 go in,

Now you can see that the equation has as many oxygens, hydrogens, nad carbons on the left as the right. The heat created is an amazing 800,000 J for one mole of . This heats the environment (at approximately 300K), creating an entropy of about 2500 J/K. So this is the actual reaction. Now if we look at the entropies (all numbers are in J/mol K)

or

That's very interesting! Notice that without the 2500 J/K of entropy created by heat, the entropy would actually go down. But as you can see, with the heat included the entropy of the universe increases a lot in this reaction. And since burning is obviously irreversible, this matches what we would think. Doing something irreversible increases the entropy of the universe.

Suppose you are recording a video of someone playing billiards. The balls crash into each other, and if you sit down and calculate the energy and momentum of the balls before and after a collision you will find that energy is conserved and momentum is conserved in each collision. Now, if you reverse the video, you will see the collision going backward, and if you do the same calculation, energy and momentum are still conserved. Just by seeing the video of a given collision, you have no way of knowing which way time is flowing. This is called time-reversal invariance, and Newton's laws of mechanics demand it.

Now let's go back to the example of putting an ice cube in a cup of hot coffee. Heat flows out of the coffee and into the ice until it all melts. This follows energy conservation, the first law of thermo, and the second law.

But what of the reverse time situation? Does the liquid in a coffee cup spontaneously heat up while forming an ice cube?  No! That cannot happen: the second law specifically says the heat flows from hot to cold. But energy and momentum would be conserved in that process, so why doesn't it ever happen?

This is a paradox, seemingly. Mechanics says that running a video backwards you see something that obeys the laws of physics. But if you run the film of the ice in the coffee backwards, now you can tell which direction time flows, since when the ice cube spontaneously forms, something is very wrong. The 2nd law contradicts Newton's Laws!

This leads me to something I must point out: the second law is not a rigorous law of nature. All rigorous laws obey time reversal symmetry. But heat only flowing from hot to cold is not time invariant. In principle, if you waited long enough, the occasion of the ice cube spontaneously forming would happen. You would, for reasons I discuss later, have to wait years or so! So, what we should say is that heat flows from hot to cold almost always. The occasions where this does not happen for a macroscopic (regular size) object are so rare that in the entire time the universe has existed, it has probably never happened.

The thing that determines how long you have to wait for an "irreversible" process to reverse itself depends on how many particles there are in the process. Suppose you have two bottles, one of which is empty (no molecules in it at all) and the other which has some molecules in it. If you put the bottles together, the gas flows to fill both of them, which is irreversible. If you watched a video of it backwards it would look very weird. But what if the bottle that had gas at the beginning had only one single atom? Then it wouldn't take very long at all for the situation to reverse itself. You just have to wait for that atom to bounce back into the original bottle it was in. If you had two molecules, it also wouldn't take long for the situation to reverse itself.

Let's do a little calculation, since this will segue into further topics. Once the gas molecules flow into the second bottle, how long would it take before the process was likely to reverse itself?

If we have just one atom, and we assume that molecules can rearrange themselves every 1 trillionth of a second (called a femtosecond or fs), it wouldn't take long at all. The molecule has a 50/50 chance of being in the container it came from. So within 2 fs it will likely have reversed itself. Now say there were two molecules. Call the original full container A and the other one B. Then the following arrangements can occur:

• Both in A container (the situation where the process reversed itself)
• Molecule 1 in A and molecule 2 in B
• Molecule 2 in A and molecule 1 in B
• Both in container B

So there are 4 possibilities, and 1 out of 4 is the one that reversed the irreversibility, meaning that if you waited 4 fs, you'd be likely to see it happen.

Obviously, we could keep going. Three molecules would have 6 possibilities, four would have 24 possibilities, etc. The numbers are getting big fast! Five molecules would have 120 possibilities, with only 1 of them being the situation where all the gas molecules spontaneously went back to their bottle. The total number of possibilities, as you may know is , where the exclamation point is "factorial", and means that you multiply times times ,etc., until you get down to 1. So 5! is 5*4*3*2*1. The probability of the situation reversing itself spontaneously is 1 divided by . Now, when , that's not too bad; you only have to wait 120 fs before the situation probably would happen. But on a real-world scale, equals 100,000,000,000,000,000,000,000. Can you imagine how large is in that case?! Your calculator will probably not even be able to compute it. (Mine insists that this number is infinity!) If you want an actual figure, I can only express it in an exponent: it's . And the probability of finding all the gas back where it was is 1 divided by that number. Can it reverse itself spontaneously? Well, you'd have to wait that many femtoseconds, and the universe has only been around for femtoseconds. So ... no.

Now, of course, you, an outside agent, can reverse it. You can set up a pump between the two bottles, and force the atoms back into the first one. But in that case you have to do some work. It will never spontaneously reverse itself. And this is also true of coffee spontaneously boiling and forming an ice cube. It could happen. But the odds against it are literally so low that they're not even calculable by me. In the most optimistic view of how long the universe will exist, it would never happen.

We'll come back to this shortly, but the bottom line is that the 2nd law implies that time goes in one direction. In addition to heat flowing from hot to cold, irreversible things like mixing or releasing a gas into another bottle don't undo themselves, at least when they involve a realistic number () particles, like they do in real life.

It's hard to say much of anything about particles if you don't know anything about the system they're in. When air is flowing through a tube, for instance, it's a constantly changing, turbulent mess. However, we can say a lot about systems that have reached equilibrium. Equilibrium is what happens when you leave the system sitting for awhile without doing anything to it.

Some kinds of equilibrium are obvious. If two systems are in direct thermal contact, so that they can exchange energy (by heating), then equilibrium is when the temperature is the same. If two gases are in two chambers separated by a movable wall, equilibrium is when the pressure is the same. And if the system is both in thermal contact and separated by a movable wall, both of these must hold.

So far we've talked about relatively straightforward quantities. Internal energy, volume, temperature, and pressure are simple to define. But because of the 2nd law, we have another quantity that defines equilibrium: entropy. This quantity will be as big as possible when the system has settled down, so it's unlike what we've seen before.

First, we should check with a simple example. If we have a cup of water at and another cup at , then we know that if we mix them they will exchange heat until the temperature has equalized. This is clearly irreversible. Let's think about how the entropy changes, which, as I've said, is heat added divided by temperature it was added at.

Of course, the final temperature of both will be halfway in between the two temperatures, the average, . As for the entropy, it will decrease for the hotter water and increase for the colder water. The heat that leaves one is the same as the heat that goes into the other one. But, the entropy is the heat divided by temperature, and since the temperature for the cold one is lower, the heat divided by temperature will mean the cold one will gain more entropy than the hot one will lose. Thus, for the universe, the change in entropy is positive. Calculating exactly is a little complicated, since the temperature is changing the whole time.

To get to equilibrium, the system makes the temperatures of the water the same, and it also raises its entropy by as much as possible. Once the temperatures are equal, there is no way for the system to raise its entropy anymore, so it stops. If things are left alone, the condition that they come to is naturally one that maximizes entropy.

So how often do you increase the entropy of the universe? All the time! Putting ice into water, heating water to brew coffee, driving your car, standing in the Sun, burning calories to keep your heart running: you and everything around you are entropy generating machines, because you all are constantly doing things that are irreversible.

Now, anybody who's seen a TV spy show knows that everything in the universe radiates "heat", which in the usual cases is dominated by infra-red light that IR cameras can see. Any object at non-zero temperature (that's everything) is radiating. That means that in the end, all objects in the universe will heat everything else. But that's irreversible. So eventually everything will become one temperature, energy will spread out as much as possible, and nothing else can happen. It's the end of the universe!

This is called the "heat death of the universe", and there really aren't too many convincing arguments against it. Entropy gets maximized, and nothing else happens ever again. There are a few reasons to think this may not happen, and we still have a lot of uncertainty about what exactly our weird universe is doing. But in the end, what happens to this universe may not be so different from what happens when you mix two cups of water at different temperatures.