1. Bayesian philosophy#

1.1. Having a disease given a positive test#

Let’s start with, perhaps, the most common example to introduce the Bayes’ rule.

There is a laboratory test that detects a disease A. Denote by \(T=1\) the event of a positive test and \(T=0\) the event of a negative test. Meanwhile, \(E=1\) denotes that the patient has the disease and \(E=0\) denotes that the patient does not have the disease.

The test has the following characteristics:

\[\mathbb{P}(T=1|E=1)=0.92,\quad \mathbb{P}(T=0|E=0)=0.99,\]

while the prevalence of the disease (the proportion of people with the disease) is 0.12, that is

\[\mathbb{P}(E=1)=0.12.\]

I take the test (assuming that I belong to the referred population) and it is positive, what is the probability that I have the disease A?

Note that \(\mathbb{P}(T=1|E=1)\) or \(\mathbb{P}(T=1|E=0)\) are not the quantities we need. Instead, we want to know what happens given that \(T=1\).

Bayes’ rule gives us the answer:

\[\mathbb{P}(E=1|T=1) = \frac{\mathbb{P}(T=1|E=1)\mathbb{P}(E=1)}{\mathbb{P}(T=1|E=1)\mathbb{P}(E=1)+\mathbb{P}(T=1|E=0)\mathbb{P}(E=0)}.\]

The next cell makes the calculation for us:

PT1GivenE1, PT0GivenE0 = 0.92, 0.99
PT1GivenE0 = 1 - PT0GivenE0

PE1 = 0.12
PE0 = 1 - PE1

PE1GivenT1 = PT1GivenE1 * PE1 / (PT1GivenE1 * PE1 + PT1GivenE0 * PE0)

round(PE1GivenT1, 4)

0.9262

But, in reality, I have or I do not have the disease; then…

What’s the meaning of “the probability of having the disease is 0.9262”!?

There are not frequencies of repeated events: I am or I am not sick!

The Bayesian statistician Bruno de Finetti began his 1973 book on probability theory with the declaration: “PROBABILITY DOES NOT EXIST.” The capitals appeared in the original. What did he mean?

1.2. The probability is conditional and subjective#

Note

These examples were taken from the notes of Bayesian Statistics of J. Andrés Christen.

One of the fundamental points of Bayesian statistics is the concept of probability (and its definition). Let’s start with some examples in which we use the concept of probability and let’s try to find a definition for it.

  1. What is the probability of getting “head” when flipping a coin?

  2. What is the probability that there will be an eclipse tomorrow?

  3. What is the probability of raining tomorrow?

  4. What is the probability that it is raining in Mexico?

  5. What is the probability that there are more than \(10^9\) stars in our galaxy?

Click the button to reveal the answers!

One might think that the answer to the first quention is 0.5 but, what coin are we talking about? does the coin has a “head”? it could be a foreign coin without “head”. For example, Mexican coins have “eagle” and “sun”, there are no “heads” in them. The coin that we are using, was given to us at the store? or is a coin that an illusionist just took out of their pocket?

For the fourth question, I could search if it is raining in Mexico (the city? the state? the whole country or just some part?) and then the probability would be 0 or 1; but right now, how much do we know about the event “it’s raining in Mexico”?

Let be \(A\) the event “it’s raining in Mexico”. A person that lives some km away from Ittoqqortoormiit, without internet, wouldn’t have any reason to assign a larger propability to \(A\) than \(A^c\). If we denote by \(\mathcal{H}_1\) the information of this person, then

\[\mathbb{P}(A|\mathcal{H}_1)=\mathbb{P}(A^c|\mathcal{H}_1).\]

GreenlandMap

Meanwhile, a person that lives in El Bajío has a different information \(\mathcal{H}_2\), and could assign the following probabilities

\[\begin{split}\mathbb{P}(A|\mathcal{H}_2)=\begin{cases} 3/4 & \text{if it's raining in El Bajío}, \\ 1/4 & \text{otherwise}. \end{cases} \end{split}\]

ElBajio

On the other hand, someone that lives in Mexico has an information \(\mathcal{H}_3\), such that

\[\begin{split}\mathbb{P}(A|\mathcal{H}_3)=\begin{cases} 1 & \text{if it's raining in Mexico}, \\ 0 & \text{otherwise}. \end{cases} \end{split}\]

MexicoCity

Thus, the probability of an event \(A\) is the measurement of the uncertainty that some agent has on that event. This means that the probability is always contextual, given some assumptions and considerations, even for the simplest cases.

1.3. Marbles in a bag#

Note

This example was taken from [McE18].

Assume that a bag has 4 marbles, which could be white or black, but we don’t know how many are of each color. Thus, there are five posibilities: \(\{W,W,W,W\}\), \(\{B,W,W,W\}\), \(\{B,B,W,W\}\), \(\{B,B,B,W\}\), \(\{B,B,B,B\}\). Let’s called these posibilities conjectures. We want to know which conjecture is more likely given some evidence about the content in the bag.

Because at the beginning we have no information about the plausibility of each conjecture, we assign a probability of 1/5 to each one. Then, we extract 3 marbles at random, one at a time and with replacement, and observe \((B,W,B)\).

We can calculate the probability of the event \((B,W,B)\) for each one of the conjectures. To do so, we just multiply our current probability in each conjecture (1/5) times the probability that each one could have generated such sample.

Conjecture

Prior Probability

Posterior Probability

\(\{W,W,W,W\}\)

1/5

\(\propto 1/5\times 0\)

\(\{B,W,W,W\}\)

1/5

\(\propto 1/5\times 3\)

\(\{B,B,W,W\}\)

1/5

\(\propto 1/5\times 8\)

\(\{B,B,B,W\}\)

1/5

\(\propto 1/5\times 9\)

\(\{B,B,B,B\}\)

1/5

\(\propto 1/5\times 0\)

Thus, the conjecture \(\{B,B,B,W\}\) is the most plausible.

1.3.1. Today’s posterior is tomorrow’s prior, the future is not as it was#

Assume that we take another marble, and it is black. With this new information, we can update the probability of each conjecture.

Conjecture

Prior Probability

Posterior Probability

\(\{W,W,W,W\}\)

\(\propto 0\)

\(\propto 0 \times 0 = 0\)

\(\{B,W,W,W\}\)

\(\propto 3\)

\(\propto 3 \times 1 = 3\)

\(\{B,B,W,W\}\)

\(\propto 8\)

\(\propto 8 \times 2 = 16\)

\(\{B,B,B,W\}\)

\(\propto 9\)

\(\propto 9 \times 3 = 27\)

\(\{B,B,B,B\}\)

\(\propto 0\)

\(\propto 0 \times 4 = 0\)

Note that what was our posterior has became our prior now.

Up to now, all the information that we received has been of the same nature. However, previous information and new information might be different. Assume, for example, that someone at the marble’s fabric tells you that black marbles are rare. For every bag of type \(\{B,B,B,W\}\) there are two of type \(\{B,B,W,W\}\) and three of type \(\{B,W,W,W\}\). Moreover, all the bags have at least one black marble and one white marble. With this information, we can update once again the probability of each conjecture

Conjecture

Prior Probability

Posterior Probability

\(\{W,W,W,W\}\)

\(\propto 0\)

\(\propto 0 \times 0 = 0\)

\(\{B,W,W,W\}\)

\(\propto 3\)

\(\propto 3 \times 3 = 9\)

\(\{B,B,W,W\}\)

\(\propto 16\)

\(\propto 16 \times 2 = 32\)

\(\{B,B,B,W\}\)

\(\propto 27\)

\(\propto 27 \times 1 = 27\)

\(\{B,B,B,B\}\)

\(\propto 0\)

\(\propto 0 \times 0 = 0\)

With this information, the most plausible conjecture is now \(\{B,B,W,W\}\).

Bayesian learning or sequential analysis

It’s not difficult to show that if we would have waited until we had all the information together, the posterior would have been the same. Instead, updating our posterior each step at a time is known as Bayesian learning or sequential analysis.

1.4. Subjectivity is always present#

One way or another, the subjectivity has always been present in the scientific activity, starting from the assumptions to analyze a phenomenon, tipically it is said that they are reasonable assumptions, but they are “reasonable” just from the experience and (subjective) information of the agent studying the phenomenon at that moment.

Accordingly to [Wol94]

… scientific objectivity has only limited value, for the way in which scientific ideas are generated can be highly subjective …. It is, however, an illusion to think that scientists are unemotional in their attachment to their scientific views … scientific theories involve a continual interplay with other scientists and previously acquired knowledge… and an explanation which other scientists would accept.

Accordingly to [Pre03]

… the subjectivity is an inherent and required part of statistical inference and the scientific method…. However, excessive, informal, untested subjectivity in science is also responsible for some basic errors, misrepresentations, overrepresentations, or scientific beliefs that were later shown to be false, that have occurred in science.

In Bayesian statistics, subjective information means all prior information about a phenomenon of interest, before collecting or making new measurements, it includes: historical data, theories, opinions and conjectures of experts, conclusions based on previous studies, etc.

1.5. Chapter summary#

Probability as a measurement of uncertainty

The probability of an event \(A\) measures the uncertainty that some agent has about the ocurrence of \(A\) based on some informathion \(\mathcal{H}\). Thus, all probabilities are conditional.

The parameters have their own distribution

Let be \(\mathcal{P}=\{p(Y|\theta),\theta\in\Theta\}\) a parametric family of distributions. Since all uncertainty must be measured through a probability, and we have uncertainty on the parameters, they have to be modeled through a measurement of probability, that is, they have their own distribution of probability.

1.6. Exercises#

Warning

The next exercises present a sensitive topic, discretion is advised.

  1. According to the National Survey on Sexual and Gender Diversity (ENDISEG) 2021, in Mexico, the LGBTI+ population surpasses 5 million people (5.1% of the Mexican population of 15 or more years old). The survey analyzed different aspects that affect the LGBTI+ community, such as the marital situation, the environment and social rejection, emotional health, among others. The next tables summarize some of the results found in the survey.


    1. Marital situation

    LGBTI+

    No LGBTI+

    Single

    64.9%

    26.9%

    Married

    30.6%

    58.5%

    Divorced, widowed or separated

    4.51%

    14.7%

    1. Between the age of 12 and 17 years old, people made you feel different to most of the girls/boys for …

    LGBTI+

    No LGBTI+

    your way to dress?

    24.7%

    11.9%

    your interests?

    27%

    9.4%

    how you speak or express?

    22.9%

    8.9%

    your behavior?

    24.8%

    9.4%

    1. During the last 12 months, at the job… did you received a different treatment with respect to job benefits or promotions? did you received offensive comments or ridicule?

    LGBTI+

    No LGBTI+

    At least one yes

    28.1%

    18.4%

    No

    71.9%

    81.6%

    1. Have you ever thought to commit suicide?

    LGBTI+

    No LGBTI+

    Yes

    26.1%

    7.9%

    No

    73.9%

    92.1%

    1. Have you ever tried to do it?

    LGBTI+

    No LGBTI+

    Yes

    14.2%

    4.2%

    No

    85.8%

    95.8%

    1. The main reason was …

    LGBTI+

    No LGBTI+

    problems with family and partner

    57.2%

    63.6%

    problems at school

    20.3%

    8.0%

    health problems

    19.1%

    18.6%

    economic problems

    16.6%

    19.5%

    problems for your sexual orientation or genre identity

    14.0%

    0.0%

    problems at the job

    12.6%

    4.9%

    other

    7.0%

    5.8%

    What is the probability that a person belongs to the LGBTI+ population if …

    1. is single?

    2. has thought in commiting suicide?

    3. has tried to commit suicide and the main reason was problems at school?

    4. has tried to commit suicide and the main reason was their sexual orientation?

    5. between the age of 12 and 17 years old, people made they feel different to most of the girls/boys for their interests?

    6. Select another variable in the previous tables, and calculate the probability that a person belongs to the LGBTI+ population given such variable.

  2. According to the data of 2023 of The Joint United Nations Programme on HIV/AIDS (UNAIDS), the prevalence of HIV in the Mexican population is 0.4%, however the prevalence of HIV of the Mexican population of men who have sex with other men (MSM) is 11.9%. The same resource estimates the population of MSM in 1,226,000 people, while according to the Census of Population and Housing 2020 from the National Institute of Statistics and Geography, the Mexican population is estimated in 126,014,024 people. If we know that a Mexican man lives with HIV, what is the probability that he belongs to the MSM population?

Comparing probabilities

For a better understanding on the implications of such probabilities, you can compare them with the probability of a person belonging to the LGBTI+ population in the Mexican population. If these probabilities are simmilar, that means that there is no correlation between belonging to the LGBTI+ population and the variable which you are considering. However, if such probabilities differ too much, it means that there is correlation.

Note

Sensitive topics might be uncomfortable, but it is important to point out the social problems of our societies intead of considering them taboo.

Paraphrasing a sentence of Sara Sefchovich:

Data make visible evident truths.

The original sentence appears in the book Living life, and says: words make visible evident truths.