In my last post I explored the Monty Hall problem, and specifically the differences between the standard version in which Monty has knowledge, and what I called the Opponent version in which a door is revealed randomly.
In this post I won't go into that detail, I'm going to assume that you're now familiar with both versions and you know that the Monty version wins 2/3 of the time if you switch, while the Opponent version is still 50/50, even if you don't understand why. In the comments of that original post I described an alternate way to view the problem that, for me at least, makes this whole thing really obvious. I haven't seen it described that way elsewhere, so I want to elaborate in the hopes that this will help some people intuitively see why Monty Hall works, as well as why that isn't initially obvious.
I want to show you the smoke and mirrors at work.
I'm going to describe a number of scenarios. It will not be obvious at first, but every one of these scenarios will be equivalent to the standard Monty Hall problem. I will begin with a version in which it is (probably) super-obvious to you that switching is the right thing to do, and then move gently towards the standard Monty Hall description where things are less obvious but still identical when you look behind the curtain. In all versions, the problem is the same as the standard Monty Hall except in the ways described.
It is my hope that you will see that these problems are all the same, and that they only differ in how well they hide the advantage you receive by switching.
Version 1. The super-obvious version:
You pick Door 1, but it is not opened.
Monty gives you the option to instead choose to open both Doors 2 and 3, and if the prize is behind either one of them, you win.
Should you switch?
This one is obvious, or should be. You get to win if the prize is behind either Door 2 or Door 3, giving you a 2/3 chance to win if you switch. Critical concepts that will be repeated in all other versions are:
- This is a 2-for-1 guess, that is why you get a 2/3 chance to win if you switch.
- Another way to look at that is that you've been given the opportunity to make 1 choice that actually reveals a pair of doors.
All future versions will contain these same features; that is why they work.
Version 2:
You pick Door 1, but it is not opened.
Monty tells you that, if the prize is behind Door 2, he is going to move it to Door 3. If the prize is behind Door 3, he is going to leave it there.
You are then given the chance to open Door 3 instead of Door 1.
Should you switch?
Yes, absolutely. Door 3 now wins if it had the prize originally, or if the prize was originally behind Door 2. Only if the prize was originally behind Door 1 does switching lose. You have a 2/3 chance, and again we have been given a single choice that essentially reveals a pair of doors.
Version 3:
You pick Door 1, but it is not opened.
Monty tells you that you can pick Doors 2 and 3 as a pair, and if the prize is behind either one of them, he will open that door, and you will win.
Should you switch?
Yes. If the prize is behind either door you win, giving you a 2/3 chance by switching. Again the doors are treated as a pair.
Version 4:
You pick Door 1, but it is not opened.
Monty tells you that you can pick Doors 2 and 3 as a pair. If you do so and the prize is behind one of them, he will open the other one (the loser) and then let you actually open the winner.
Should you switch?
Yes. If the prize is behind either door you win, giving you a 2/3 chance by switching. Again the doors are treated as a pair. Notice how close we are to the description of the original problem.
Version 5:
You pick Door 1, but it is not opened.
Monty tells you that he is going to open a losing door from the pair of doors made by Doors 2 and 3. He will then give you the option to switch to the unopened door.
Should you switch?
Yes. Same as before, these doors are treated as a pair. 2/3 chance to win by switching. Notice how this is identical to the previous version.
Version 6, the original problem re-framed to not specify which door Monty opened:
You pick Door 1, but it is not opened.
Monty tells you that he is going open a losing door, but he will never open the door you've already picked. He will then give you the option to switch to the remaining door.
Should you switch?
Yes. 2/3 chance to win by switching. Though he hasn't said so explicitly, Monty is treating Doors 2 and 3 as a pair because he isn't going to open the door you already picked. His choice therefore is out of Doors 2 & 3, and by using his knowledge to eliminate a sure loser from that pair, he makes it so that you have 1 choice that wins if either of these doors has the prize. It is more obvious that they are a pair here, than in the standard Monty version, because we have not specified which of the two is a loser, and which remains.
Version 7, the original problem:
You pick Door 1, but it is not opened.
Monty tells you that he is going to open a losing door for you.
Monty opens Door 3, revealing a goat.
You are given the option to switch to Door 2.
Should you switch?
Yes. You have a 2/3 chance to win by switching. What makes this trick your mind is that you are likely to frame the problem as "Should I pick Door 1 or Door 2?" when the correct framing is "Should I pick Door 1 or the door that remains from the Door 2/Door 3 pair?" (which in this particular case happens to be Door 2) As in all other versions, Doors 2 & 3 are treated as a pair. You get to win if the prize was behind either of them. You get a 2-for-1 guess, but if you're thinking of Door 2 as itself and not as representative of the pair, then your brain is likely to experience the disbelief that makes this a veridical paradox. Your choice is not between two doors, but between a single door and a pair of doors. When given the opportunity to switch to a pair of doors, your brain gets it. When given the opportunity to switch to a door that represents a pair of doors, your brain can easily miss the hidden value there.
The Monty version (knowledge based choice) isn't the same as the Opponent version (random choice) because in the Opponent version the opponent is opening a door that might be a winner. If the prize is revealed, you lose. Monty never reveals the prize, he reveals information that gives you the option to make a 2-for-1 guess. In this way, Monty's action is guaranteed to help you, whereas the opponent will cause you to immediately lose 1/3 of the time (the times when he guesses correctly.) Nothing that the opponent does creates a situation where you can win if either of 2 doors has the prize. The opponent does not create a 2-for-1 guess for you.
Does this help make the problem intuitive for anyone else?
Saturday, May 5, 2012
Thursday, May 3, 2012
Knowledge and Monty Hall
Sam Harris' book, The Moral Landscape, introduced me to the Monty Hall problem. People who love big words call this problem a veridical paradox, which basically means that the correct answer defies common sense. Most people, even after having the answer explained to them, still have this unsettled feeling about it. If you've never seen this before, you are almost guaranteed to think "but that can't be right!"
You can read about the problem on the Wikipedia page I linked above. You can also read a description by Barbara Drescher here. The purpose in bringing this up is to illustrate how our brains can fail us. How common sense can be dead wrong. How an assumption about the world can therefore be dead wrong, and how critical it is that we look deeply and think critically.
This applies to all sorts of things. Evolution, for example, is not an intuitively obvious process, and I suspect that this is the source of much resistance to the idea. Even after understanding evolution I sometimes think about the complexity of even a goldfish and think "wait, that isn't designed?"
So back to Monty Hall. Here's the most popular version of it:
Suppose you're on a game show, and you're given the choice of three doors: Behind one door is a car; behind the others, goats. You pick a door, say No. 1 [but the door is not opened], and the host, who knows what's behind the doors, opens another door, say No. 3, which has a goat. He then says to you, "Do you want to pick door No. 2?" Is it to your advantage to switch your choice?
The answer is yes, it is to your advantage to switch, because if the odds of Door No. 1 winning are 1 in 3, and the odds of Door No.3 winning are 2 in 3. Your brain might riot at this suggestion, but its true. I will briefly describe why this works a little later, but the links to Wikipedia and Drescher above include pretty pictures, so you may prefer that. The odds are not equal between these two doors. The key thing that your brain probably does not intuitively recognize is that you have the ability to take advantage of someone else's knowledge.
Drescher's description of the problem and its solution included this phrase:
"The issue of knowledge is a factor in our processing of the problem, but it’s not what Monty knows that matters. It’s what you (the subject of the problem) know."
It is not clear to me whether Drescher has misunderstood the deeper nature of the problem, or if she has just been unclear with her wording. In any case, the phrase "it's not what Monty knows that matters" is very misleading. What Monty knows is crucial. What Monty knows is what makes this whole thing work. What Monty knows about which door is the winner is what makes these two options unequal.
To illustrate this, we can change the problem slightly. This time, instead of Monty opening the other door, we will alter the problem so that you have an opponent:
Suppose you're on a game show, in the final round with your opponent. Because you are ahead, you are allowed to reserve a door (but not open it) before your opponent picks. Behind one door is a car; behind the others, goats. You pick Door No. 1 but the door is not opened. Your opponent picks Door No. 3 and this door is opened, revealing a goat. Your host now gives you the opportunity to switch to Door No. 2 if you wish. Is it to your advantage to switch?
In this case, the answer is no, it does not matter. Door No.1 and Door No.2 in this version of the problem have equal chances to be correct, and the difference is that your opponent did not have any knowledge of which door was the winner when she picked Door No. 3. It was not a revealing of information, it was a guess. You do get more information, in the sense that you now know that Door No. 3 is a loser, but in this version that doesn't help you pick between 1 and 2. Here's why:
I find this easier to explain in the multiple universes sense. In the original problem, there are 3 possible universes that you could be living in:
Universe 1:
C G G <- the prizes behind the doors
1 2 3 <- the door numbers
^ ? ? <- the various guesses and reveals
In this universe there is a car (C) behind door number 1, and you have picked this door as shown by the (^) symbol. In this case Monty will reveal either door 2 or 3 and if you switch you will lose. This is the 1 case out of 3 where switching is not to your advantage.
Universe 2:
G C G
1 2 3
^ ? X
In this universe there is a car (C) behind door number 2. You have picked door 1 as shown by the (^) symbol. In this case Monty will always reveal door 3 (X), because Monty knows which door has the car. This is a case where you win if you switch your guess.
Universe 3:
G G C
1 2 3
^ X ?
In this universe the car is behind door 3, you have picked door 1, and Monty will always reveal door 2 (X) as a loser, because Monty knows. This is a case where you win if you switch your guess.
So that's the original problem, and in 2 of the 3 possible scenarios you win if you switch your guess.
Here's the altered version. In it, your opponent's guess is random, as was yours. For clarity we will pre-assume that you have guessed door no 1, and your opponent has guessed door no 2 in all cases. Since neither of you know anything about which doors is the winner, this configuration of guesses is essentially the same as any other.
Universe 1:
C G G
1 2 3
^ L ?
In this universe the car is behind 1, and your opponent has lost by guessing door 2 (L). If you switch you will lose.
Universe 2:
G C G
1 2 3
^ W ?
In this universe the car is behind 2, which your opponent has guessed (W). You lose. If you have a choice to make at all, you are not in this universe. Critically, this is the condition that never happens in the version where Monty reveals a door that he knows to be a loser for you, and this is what makes the two versions different. In the previous version this Universe is a winner for you, but in this one it is a loser.
Universe 3:
G G C
1 2 3
^ L ?
In this universe the car is behind 3, and you will win if you switch.
Out of these 3 universes, you can only be in 2 of them if you have a choice to make at all, and of those 2 possibilities, in one of them you win if you switch and in the other you lose if you switch. It's 50/50 this time.
So there's an extra layer of paradox. You picked 1, and someone opened another door, revealing a goat. Should you switch? It depends on what the person who opened the door knows. If Monty opened the door with the deliberate intent to reveal a loser to you, then you should switch because that makes your odds 2/3. If an opponent selected a door randomly and just happened to reveal that same losing door, then your chances are not affected by switching. One door is as good as the other.
This is so counter-intuitive that I built a computer simulation of the problem just to verify that I'm not crazy. The car placement is random. The contestant's guess is random. In the Monty version, Monty's reveal is not random, it is based on Monty's knowledge. In the Opponent version, the opponent's guess is random from the remaining choices.
Here's 1 million runs of the Monty version (Monty reveals, information added):
Opportunities for you to switch: 1,000,000
Wins: 667,103
As we claimed, you win 2/3 of the time in this version by switching, and you always have the chance to switch.
Here's 1 million runs of Opponent version (Opponent guesses, no information added):
Opponent guesses correctly: 333,246 times. (no chance for you to guess)
Opportunities for you to switch: 666,754 times.
Wins by switching: 333,417 times.
As you can see, in this version you win about 1/2 the time that you are given the opportunity to switch, or 1/3 of the time overall. In the Monty version all of those times when the Opponent guessed correctly would have instead been converted to a condition where you win if you switch.
We might as well call your opponent Scotty, because Scotty doesn't know. (NSFW, sexual humor, language. Also, Matt Damon!) Monty does know, and knowing is half the battle... or 2/3rds... or something.
Failures of human reasoning fascinate me.
You can read about the problem on the Wikipedia page I linked above. You can also read a description by Barbara Drescher here. The purpose in bringing this up is to illustrate how our brains can fail us. How common sense can be dead wrong. How an assumption about the world can therefore be dead wrong, and how critical it is that we look deeply and think critically.
This applies to all sorts of things. Evolution, for example, is not an intuitively obvious process, and I suspect that this is the source of much resistance to the idea. Even after understanding evolution I sometimes think about the complexity of even a goldfish and think "wait, that isn't designed?"
So back to Monty Hall. Here's the most popular version of it:
Suppose you're on a game show, and you're given the choice of three doors: Behind one door is a car; behind the others, goats. You pick a door, say No. 1 [but the door is not opened], and the host, who knows what's behind the doors, opens another door, say No. 3, which has a goat. He then says to you, "Do you want to pick door No. 2?" Is it to your advantage to switch your choice?
The answer is yes, it is to your advantage to switch, because if the odds of Door No. 1 winning are 1 in 3, and the odds of Door No.3 winning are 2 in 3. Your brain might riot at this suggestion, but its true. I will briefly describe why this works a little later, but the links to Wikipedia and Drescher above include pretty pictures, so you may prefer that. The odds are not equal between these two doors. The key thing that your brain probably does not intuitively recognize is that you have the ability to take advantage of someone else's knowledge.
Drescher's description of the problem and its solution included this phrase:
"The issue of knowledge is a factor in our processing of the problem, but it’s not what Monty knows that matters. It’s what you (the subject of the problem) know."
It is not clear to me whether Drescher has misunderstood the deeper nature of the problem, or if she has just been unclear with her wording. In any case, the phrase "it's not what Monty knows that matters" is very misleading. What Monty knows is crucial. What Monty knows is what makes this whole thing work. What Monty knows about which door is the winner is what makes these two options unequal.
To illustrate this, we can change the problem slightly. This time, instead of Monty opening the other door, we will alter the problem so that you have an opponent:
Suppose you're on a game show, in the final round with your opponent. Because you are ahead, you are allowed to reserve a door (but not open it) before your opponent picks. Behind one door is a car; behind the others, goats. You pick Door No. 1 but the door is not opened. Your opponent picks Door No. 3 and this door is opened, revealing a goat. Your host now gives you the opportunity to switch to Door No. 2 if you wish. Is it to your advantage to switch?
In this case, the answer is no, it does not matter. Door No.1 and Door No.2 in this version of the problem have equal chances to be correct, and the difference is that your opponent did not have any knowledge of which door was the winner when she picked Door No. 3. It was not a revealing of information, it was a guess. You do get more information, in the sense that you now know that Door No. 3 is a loser, but in this version that doesn't help you pick between 1 and 2. Here's why:
I find this easier to explain in the multiple universes sense. In the original problem, there are 3 possible universes that you could be living in:
Universe 1:
C G G <- the prizes behind the doors
1 2 3 <- the door numbers
^ ? ? <- the various guesses and reveals
In this universe there is a car (C) behind door number 1, and you have picked this door as shown by the (^) symbol. In this case Monty will reveal either door 2 or 3 and if you switch you will lose. This is the 1 case out of 3 where switching is not to your advantage.
Universe 2:
G C G
1 2 3
^ ? X
In this universe there is a car (C) behind door number 2. You have picked door 1 as shown by the (^) symbol. In this case Monty will always reveal door 3 (X), because Monty knows which door has the car. This is a case where you win if you switch your guess.
Universe 3:
G G C
1 2 3
^ X ?
In this universe the car is behind door 3, you have picked door 1, and Monty will always reveal door 2 (X) as a loser, because Monty knows. This is a case where you win if you switch your guess.
So that's the original problem, and in 2 of the 3 possible scenarios you win if you switch your guess.
Here's the altered version. In it, your opponent's guess is random, as was yours. For clarity we will pre-assume that you have guessed door no 1, and your opponent has guessed door no 2 in all cases. Since neither of you know anything about which doors is the winner, this configuration of guesses is essentially the same as any other.
Universe 1:
C G G
1 2 3
^ L ?
In this universe the car is behind 1, and your opponent has lost by guessing door 2 (L). If you switch you will lose.
Universe 2:
G C G
1 2 3
^ W ?
In this universe the car is behind 2, which your opponent has guessed (W). You lose. If you have a choice to make at all, you are not in this universe. Critically, this is the condition that never happens in the version where Monty reveals a door that he knows to be a loser for you, and this is what makes the two versions different. In the previous version this Universe is a winner for you, but in this one it is a loser.
Universe 3:
G G C
1 2 3
^ L ?
In this universe the car is behind 3, and you will win if you switch.
Out of these 3 universes, you can only be in 2 of them if you have a choice to make at all, and of those 2 possibilities, in one of them you win if you switch and in the other you lose if you switch. It's 50/50 this time.
So there's an extra layer of paradox. You picked 1, and someone opened another door, revealing a goat. Should you switch? It depends on what the person who opened the door knows. If Monty opened the door with the deliberate intent to reveal a loser to you, then you should switch because that makes your odds 2/3. If an opponent selected a door randomly and just happened to reveal that same losing door, then your chances are not affected by switching. One door is as good as the other.
This is so counter-intuitive that I built a computer simulation of the problem just to verify that I'm not crazy. The car placement is random. The contestant's guess is random. In the Monty version, Monty's reveal is not random, it is based on Monty's knowledge. In the Opponent version, the opponent's guess is random from the remaining choices.
Here's 1 million runs of the Monty version (Monty reveals, information added):
Opportunities for you to switch: 1,000,000
Wins: 667,103
As we claimed, you win 2/3 of the time in this version by switching, and you always have the chance to switch.
Here's 1 million runs of Opponent version (Opponent guesses, no information added):
Opponent guesses correctly: 333,246 times. (no chance for you to guess)
Opportunities for you to switch: 666,754 times.
Wins by switching: 333,417 times.
As you can see, in this version you win about 1/2 the time that you are given the opportunity to switch, or 1/3 of the time overall. In the Monty version all of those times when the Opponent guessed correctly would have instead been converted to a condition where you win if you switch.
We might as well call your opponent Scotty, because Scotty doesn't know. (NSFW, sexual humor, language. Also, Matt Damon!) Monty does know, and knowing is half the battle... or 2/3rds... or something.
Failures of human reasoning fascinate me.
Subscribe to:
Posts (Atom)