@Berny pls would you explain (as in ELI5) Bayes Theorem?

Blimey, it's been nearly 40 years but (first stab) here goes... 

It's basically a conditional probability, where the overall probability of an event occurring is conditional upon a different event also occurring.  The example I remember learning all that time ago concerns being late for work. 

Let's say the probability of being late for work on a Monday is x, given no preconditions.  But if you set your alarm clock when turning in on Sunday night, x will quite likely change to a different value y.  Your probability of being late for work is now dependent on the probability you remembered to set your alarm clock!  

Bayes Theorem stitches these probabilities together, to give the true probability that you'll be late for work.   This "late for work" example stuck with me because it's quite a good way to envisage what Bayes is all about.   

I'll have a root around Google to see if I can add any flesh to this, with a worked example.     

Investopedia gives a reasonable – though not great - example.  I felt Investopedia was quite appropriate to a financial forum.

They offer the example of picking a single card from a deck.  What is the probability you’ve picked a King?  How does that probability alter if you’re then told the card you’ve picked is a face card?  This should be an easy, real world example to follow. Forgive me, but I’m doing as you asked, explaining like you’re 5…!


A good starting point is to remember all probabilities always lie between 0 and 1.  Terminology below: P(A|B) is read “the probability of event A, given event B has occurred first”. That’s how I’ve always remembered it.


Lay out the probabilities you know about:-


P(A) = probability you picked a King = 4/52.  With four Kings in the deck, this is the starting probability, given no other prior knowledge of anything else in the system.


P(B) = prob picked a face card = 12/52. With 12 face cards in the deck, this was the initial probability of that event.  Similar idea as above.  You don’t know you’ve picked a face card up front, you find it out later and now modify your probabilities according...

P(A|B) = prob you’ve picked a King, having now been told you’ve picked a face card. This is the new probability of event A (you’ve picked a King), armed with the new knowledge that the card you’re holding is a face card.  This is the new probability you want to establish.
 
P(B|A) = prob you’ve picked a face card, if you’re now told you’ve picked a King.  This (hypothetical) event has to have probability 1, since a King is by definition a face card.  This event doesn’t actually take place, it’s purely hypothetical…. If you had randomly picked out a face card - and were then told you have a King -  clearly you’re guaranteed to have a face card in your hand.  So P(B|A) = 1.


Now apply Bayes Theorem to evaluate the new probability that you’ve picked a King, having now been told you have picked a face card.  P(A) was previously just 4/52, but the new probability that you hold a King will now be something much bigger, because you've learned significant new information.


So P(A|B), the probability you’ve picked a King, given the new information that you’ve picked out a face card, is given by P(A|B) = P(A) x P(B|A) / P(B). [This is the statement of Bayes Theorem].


So, plugging in the values already laid out above, P(A|B) = (4/52) x 1 / (12/52)

= 4/52 x 52/12

=4/12, or 1/3rd.


The same answer as given by Investopedia (they say 33.3%), but strangely they’ve not derived it using the formula directly, as I have above, but by a commonsense argument. I find that slightly unsatisfactory, as indeed I find this particular example unsatisfactory, given the P(B|A) = 1 starting condition.  But it’s helpful as a guide to the theorem, because you can equate it to a real world event.


A better example is given on Wiki, using a beetle having certain markings, but is perhaps less intuitive to follow and understand what’s happening. I follow the beetle thing because I’ve studied all this before, but if it’s the first time you’ve come across Bayes, the beetle example may serve to confuse. 

The beetle worked example, for those (a) feeling a bit braver, and (b) entirely sober... 

About halfway down the page... en.wikipedia.org/wiki/Bayes%27_theorem 

Bayes Theorem is about the laws of conditional probability, the probability of something happening given something else has happened. Say you have two dice A and B what is the chance of getting a 6 and then a 6. Well that is just 1/6 * 1/6 = 1/36. Now say they are magic dice and that if you roll a 6 on the first dice then the chance of rolling a 6 on the second becomes 1/3, the chance of getting a 6 and a 6 becomes 1/6 * 1/3 = 1/18. That is an application of Bayes Theorem because a 6 on the second dice is condition on the first. The formula becomes: P(A) * P(B|A) now when the two events are independent like when they are not magic P(B|A) becomes P(B), when they are magic P(B|A) does not equal P(B).

Now that is kind of interesting but there is a mathematical law of conditional probability here which has to apply to all conditional probabilities and can be used to derive things that are not so obvious:

P(A) * P(B|A) = P(B) * P(A|B) or more famously P(A) = P(B) * P(A|B) / P(B|A)

P(B|A) means probability of B occurring given A has already occurred


Basically like alot of math it starts with something very obvious and then ends up with something not obvious and is some hidden law of reality like Pythagoras' theorem. This law would even apply to our magic dice or Assetz Capital*.

* - although it would be very hard to work out what getting your money back is actually conditional on.

This stuff is tricky and lawd knows it's confused me enough times over the years, but I'm afraid you've mixed up some of your As and Bs in the equation highlighted.

The simplest starting point is this: the probability of event A given event B is the probability of both events happening divided by the probability of event B happening.

This can be written P(A|B) = P(A AND B) / P(B). The AND here is usually written with the (think Venn diagram) intersect sign, ꓵ , so P(A|B) = P(AꓵB) / P(B).

In a similar way, the probability of event B given event A is the probability of both events happening divided by the probability of event A happening, or P(B|A) = P(BꓵA) / P(A).

Now since the probability of both A AND B happening is exactly the same as both B AND A happening, P(AꓵB) = P(BꓵA) and we can rewrite the two equations above to give

P(B) * P(A|B) = P(AꓵB) = P(BꓵA) = P(A) * P(B|A),
or P(B) * P(A|B) = P(A) * P(B|A)

[Note the subtle difference from your equation highlighted].

Now from here it's easy to derive Bayes Theorem. Simply divide both sides by P(B). Always assuming P(B)≠0 of course.

That gives   P(A|B) = P(B|A) * P(A) / P(B)

This equation, in blue, is Bayes Theorem, in the most famous form you'd usually find it expressed.  It causes so much confusion... and I include myself in that bracket!

Ah, join the club!  It's one of the things I initially found strange about the whole of statistics when I studied it.  It's a non-exact science based very much on trying to apply the 'correct' distribution to the data.  Testing the null hypothesis, for example, is a highly mature science, yet once again it's all based on probabilities, it's never perfectly exact like the maths I was used to before touching upon statistics. 

Often times, applying your best initial guess is the only starting point you have available, and having done so will still lead to a better estimate of the final probability you are seeking.  

I remember one example a very good mathematician once used to try to explain all this to me.  Something to do with a US game show where you have to choose a prize behind a door, and how to maximise your chances of winning eg the car.  And how the correct second go was... non-intuitive.

Not sure I understand it to this day .