\n",
" **Jupyter Notebooks for Teaching and Learning**

\n", " C. Hardebolle, P. Jermann, R. Tormey, CC BY-NC-SA 4.0 Int.

\n", "

\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n", " C. Hardebolle, P. Jermann, R. Tormey, CC BY-NC-SA 4.0 Int.

\n", "

Introduction to hypothesis testing

\n", "\n", "An important part of the scientific process is to make hypotheses about the world or about the results of experiments. These hypotheses need then to be checked by collecting evidence and making comparisons. Hypothesis testing is a step in this process where statistical tools are used to test hypotheses using data.\n", "\n", "**This notebook is designed for you to learn**:\n", "* How to distinguish between \"population\" datasets and \"sample\" datasets when dealing with experimental data\n", "* How to compare a sample to a population, test a hypothesis using a statistical test called the \"t-test\" and interpret its results\n", "* How to use Python scripts to make statistical analyses on a dataset\n", "\n", "In the following, we will use an example dataset representing series of measurements on a type of flower called Iris." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Introduction\n", "\n", "\n",
" \n",
"\n",
"###### Iris Virginica (Credit: Frank Mayfield CC BY-SA 2.0)\n",
"\n",
"

\n",
"\n",
"In 1935, an american botanist called Edgar Anderson worked on quantifying the morphologic variation of Iris flowers of three related species, Iris Setosa, Iris Virginica and Iris Versicolor [[1]](#Bibliography). He realized a series of measures of the petal length, petal width, sepal length, sepal width and species.\n",
"Based on the combination of these four features, a British statistician and biologist named Ronald Fisher developed a model to distinguish the species from each other [[2]](#Bibliography).\n",
"\n",
"## Question\n",
"\n",
"A recent series of measurements has been carried out at the [Iris Garden of the Vullierens Castle](https://chateauvullierens.ch/en/) near Lausanne, on a sample of 50 flowers of the Iris Virginica species. \n",
"**How similar (or different) is the Iris sample from the Vullierens Castle compared to the Iris Virginica population documented by Edgar Anderson?**\n",
"\n",
"## Instructions\n",
"\n",
"This notebook will guide you in the use of Python tools for analyzing this experimental dataset and perform statistical tests which are widely used in hypothesis testing. \n",
"It includes:\n",
"* **explanations to read** about how to analyze experimental data to answer a research question,\n",
"* **code to execute** to illustrate how to perform data analysis using Python.\n",
"* **questions** to help you think about what you learn along the way.\n",
"\n",
"\n",
- "**Solutions** of all the questions are available [in this file](./solution/StatisticsNotebook-solution.ipynb), we recommend you to **check your answer** after each question, before moving to the next piece of content."
+ "**Solutions**. We recommend you to **check your answer** after each question, before moving to the next piece of content."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
" How to use this notebook?

\n", "

\n",
"\n", "

- \n",
"
- To
**execute**the code in this notebook, simply click on the cell containing the code and then click on the \"play\" button (►) in the tool bar just above the notebook, or type`shift + enter`

.

It is important to execute the code cells in their order of appearance in the notebook. \n",
" - You can
**change the content**of all the code cells of this notebook, and also**add new cells**to the notebook by clicking on the \"plus\" button (+) in the tool bar just above the notebook.

\n", " By default, cells you add to the notebook are made to contain code.

\n", " If you want a new cell to contain text, select \"Markdown\" in the drop down menu in the same tool bar. \n",
"

\n", "\n", "While using the notebook, you can also **take notes on a piece of paper** if you feel this is helpful.\n", "\n", " \n", "\n", "\n", "--- " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Getting started" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Python tools for stats\n", "Python comes with a number of libraries for processing data and computing statistics.\n", "To use these tool you first have to load them using the `import` keyword. \n", "The role of the code cell just below is to load the tools that we use in the rest of the notebook. It is important to execute this cell *prior to executing any other cell in the notebook*." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# plotting and display tools\n", "%matplotlib inline\n", "import matplotlib.pyplot as plt\n", "plt.style.use('seaborn-whitegrid') # global style for plotting\n", "\n", "from IPython.display import display, set_matplotlib_formats\n", "set_matplotlib_formats('svg') # vector format for graphs\n", "\n", "# data computation tools\n", "import numpy as np \n", "import pandas as pan\n", "import math\n", "\n", "# statistics tools\n", "import scipy.stats as stats\n", "from lib.dataanalysis import * " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Data available on the Anderson population\n", "\n", "Anderson has published summary statistics of his dataset. \n", "You have the **mean petal length of the Iris Virginica species** documented by Anderson: $\\mu = 5.552$ cm, which we define in the code below." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Define mu as mean petal length of Iris Virginica species from Anderson\n", "mu = 5.552\n", "\n", "# Display mu\n", "mu" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

\n",
" Question

\n", " What does the first line of code above do? And what is the role of the second line of code?

\n", " How would you do to define another value in the code, for instance the mean petal length of Iris Versicolor $\\mu_{versicolor}= 4.26$ cm?

\n", " Type your code using the cell below and execute it to test the result. \n", "

"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Define mu_versicolor here\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"jupyter": {
"source_hidden": true
}
},
"source": [
"\n",
"\n", " What does the first line of code above do? And what is the role of the second line of code?

\n", " How would you do to define another value in the code, for instance the mean petal length of Iris Versicolor $\\mu_{versicolor}= 4.26$ cm?

\n", " Type your code using the cell below and execute it to test the result. \n", "

You can check your answer by clicking on the \"...\" below.

\n", "\n",
" Solution

\n", " The first line of code defines a variable called

\n", " The role of the second line of code is to display the value of

\n", " Based on the same model, below is the code to define

\n",
"\n",
"\n", " The first line of code defines a variable called

`mu`

and sets its value to `5.552`

.\n", " The role of the second line of code is to display the value of

`mu`

\n", " Based on the same model, below is the code to define

`mu_versicolor`

with a value of `4.26`

and display it. \n",
"\n", "# Define mu_versicolor here\n", "mu_versicolor = 4.26\n", "\n", "# Display beta\n", "mu_versicolor\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Data available on the Vullierens sample\n", "\n", "You have the raw data collected on the petal length and petal width of the Vullierens sample, which is stored in the file `iris-sample-vullierens.csv` that you can see in the file explorer in the left pane. \n", "If you double click on the file it will open in a new tab and you can look at what is inside.\n", "\n", "Now to analyze the data using Python you have to read the file:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Read the Vullierens sample data from the CSV file\n", "sample_data = pan.read_csv('iris-sample-vullierens.csv')\n", "\n", "# Display the first few lines of the dataset\n", "sample_data.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "After reading the file, its content is stored in the variable `sample_data`, which is a kind of table. The output above shows us an extract of the table, limited to the first 5 lines. We see above that each line of the table is given an index number to identify it. We also see that, appart from the index, the table contains two columns, called `\"petal_length\"` and `\"petal_width\"`, which contains all the measurements made on the Vullierens Irises.\n", "\n", "To get the complete list of all the values stored in one specific column such as `\"petal_length\"`, you can use the following syntax: `sample_data[\"petal_length\"]`." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# All values stored in the \"petal_length\" column of the \"sample_data\" table\n", "sample_data[\"petal_length\"]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

\n",
" Question

\n", " How would you access the data stored in the other column of this table, named

\n", " Type and test your code using the cell below.\n", "

"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Access the values stored in the \"petal_width\" column of the \"sample_data\" table\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"jupyter": {
"source_hidden": true
}
},
"source": [
"\n",
"\n", " How would you access the data stored in the other column of this table, named

`\"petal_width\"`

?\n", " Type and test your code using the cell below.\n", "

You can check your answer by clicking on the \"...\" below.

\n", "\n",
" Solution

\n", " Below is the code to access the data stored in the

\n",
"\n",
"\n", " Below is the code to access the data stored in the

`\"petal_width\"`

column of the table: we simply change the name of the column we want to access. \n",
"\n", "# Access the values stored in the \"petal_width\" column of the \"sample_data\" table\n", "sample_data[\"petal_width\"]\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ " \n", "\n", "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# First look at the data" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Descriptive statistics\n", "\n", "A first important step in analyzing data is to get an idea of its basic characteristics using **descriptive statistics** such as the **mean** (i.e. the average value or \"moyenne\" in French) and the **standard deviation** (\"écart-type\" in French, generally abreviated

\n",
" Question

\n", " From the table above, what is the mean value of the petal length in the Vullierens sample?

\n", " And the standard deviation (*std*) of the petal length in the Vullierens sample?\n",
"

"
]
},
{
"cell_type": "markdown",
"metadata": {
"jupyter": {
"source_hidden": true
}
},
"source": [
"\n",
"\n", " From the table above, what is the mean value of the petal length in the Vullierens sample?

\n", " And the standard deviation (

You can check your answer by clicking on the \"...\" below.

\n", "\n",
" Solution

\n", " From the table above, we can read in the first column, second line that the mean value of the petal length of the Vullierens sample is

\n", " We can read in the first column, third line that the standard deviation of the petal length is

"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can access individual elements of the `sample_stats` table using the corresponding names for the line and column of the value. \n",
"The following cell illustrates how to get the **sample size** (named `count` in the table above):"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Extract the sample mean from the descriptive stats\n",
"sample_size = sample_stats.loc[\"count\",\"petal_length\"]\n",
"\n",
"# Display the result\n",
"sample_size"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Another interesting information to extract from these descriptive statistics is the **mean value of the petal length** in the sample:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Extract the sample mean of the petal length from the descriptive stats\n",
"sample_mean = sample_stats.loc[\"mean\",\"petal_length\"]\n",
"\n",
"# Display the result\n",
"sample_mean"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n", " From the table above, we can read in the first column, second line that the mean value of the petal length of the Vullierens sample is

`5.713045 cm`

.\n", " We can read in the first column, third line that the standard deviation of the petal length is

`0.518940 cm`

.\n",
"\n",
" Question

\n", " How could you access the value of the standard deviation of the petal length in the

\n", " Type and test your code using the cell below.\n", "

"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Extract the sample standard deviation of the petal length from the descriptive stats\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"jupyter": {
"source_hidden": true
}
},
"source": [
"\n",
"\n", " How could you access the value of the standard deviation of the petal length in the

`sample_stats`

table?\n", " Type and test your code using the cell below.\n", "

You can check your answer by clicking on the \"...\" below.

\n", "\n",
" Solution

\n", " Below is the code to access the value of the standard deviation of the petal length in the

\n",
"\n",
"\n", " Below is the code to access the value of the standard deviation of the petal length in the

`sample_stats`

table: we use the name of the line containing the value, `std`

, and we store the result in a variable called `sample_std`

.\n",
"\n", "# Extract the sample standard deviation of the petal length from the descriptive stats\n", "sample_std = sample_stats.loc[\"std\",\"petal_length\"]\n", "\n", "# Display the result\n", "sample_std\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Visualization\n", "\n", "After having looked at simple descriptive statistics, another important step is to **visualize the data**, to better identify its characteristics. \n", "Histograms are useful to visualize the [frequency distribution](https://en.wikipedia.org/wiki/Frequency_distribution) of the sample values: the horizontal axis displays intervals of the variable we are looking at, in our case the petal length, and the vertical axis indicates the number of samples in each interval." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Plot the histogram representing the distribution of the samples\n", "plt.hist(sample_data[\"petal_length\"], color=\"green\")\n", "plt.xticks(np.arange(4.6, 7.2, 0.2))\n", "\n", "# Add a vertical line for the sample mean\n", "plt.axvline(x=sample_mean, color='black', linestyle='-.', linewidth=1, label=\"sample mean $m$\")\n", "\n", "# Add a vertical line for the population mean\n", "plt.axvline(x=mu, color='black', linestyle=':', linewidth=1, label=\"population mean $\\mu$\")\n", "\n", "# Add a legend\n", "plt.legend();" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

\n",
" Question

\n", " From the graph above, how many irises from the Vullierens sample have a petal length between 4.7 and 4.95 cm?

\n", " How is the mean petal length of the Vullierens sample represented? And the mean of the Anderson population?

\n", " How close are they to each other?\n", "

"
]
},
{
"cell_type": "markdown",
"metadata": {
"jupyter": {
"source_hidden": true
}
},
"source": [
"\n",
"\n", " From the graph above, how many irises from the Vullierens sample have a petal length between 4.7 and 4.95 cm?

\n", " How is the mean petal length of the Vullierens sample represented? And the mean of the Anderson population?

\n", " How close are they to each other?\n", "

You can check your answer by clicking on the \"...\" below.

\n", "\n",
" Solution

\n", " The irises with a petal length between 4.7 and 4.95 cm are represented by the first bar of the histogram (counting from the left) and we can read on the vertical axis that there are 5 irises represented in this bar.

\n", " According to the legend, the mean petal length of the Vullierens sample is represented by a vertical dash-dotted line (-·-·-) and the mean of the Anderson population by a vertical dotted line (·····).

\n", " These two means seem to be quite close to each other, with a difference of around 0.15 cm.\n", "

"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Interpretation and hypothesis\n",
"\n",
"The simple analyses we have made so far allow us to have a preliminary idea about how the Irises from Vullierens compare to those observed by Anderson. One feature to look at for the comparison is their respective mean petal length. We see above that the mean petal length $m$ of the Vullierens sample is quite close to the mean $\\mu$ reported by Anderson. However, we also see that there is some variability in our sample, meaning that some irises in our sample actually have a petal length quite far from that of the Anderson population. So are the two means really that close to each other?\n",
"\n",
"Let's formulate this as an **hypothesis** which we state as: the sample mean $m$ is similar to the mean of the reference population $\\mu$, which we will note $m = \\mu$ (in this notation, the equal symbol should not be interpreted literally). This hypothesis is noted $H_0$ and called the \"null\" hypothesis because it states that there is no difference between the sample and the population. \n",
"The \"alternate\" hypothesis $H_a$ is that the sample mean is not similar to the mean of the reference population, $m \\neq \\mu$.\n",
"\n",
"How can we test our hypothesis? In the following, we use a **statistical test** to answer this question.\n",
"\n",
" \n",
"\n",
"---\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Testing our hypothesis"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In our hypothesis we compare the mean of one sample to a reference value. To test this hypothesis we can use a statistical test called a **one-sample t-test**. \n",
"\n",
"But what does it mean when we test the hypothesis that a sample mean is potentially equal to a given value? "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Sample versus population\n",
"\n",
"\n", " The irises with a petal length between 4.7 and 4.95 cm are represented by the first bar of the histogram (counting from the left) and we can read on the vertical axis that there are 5 irises represented in this bar.

\n", " According to the legend, the mean petal length of the Vullierens sample is represented by a vertical dash-dotted line (-·-·-) and the mean of the Anderson population by a vertical dotted line (·····).

\n", " These two means seem to be quite close to each other, with a difference of around 0.15 cm.\n", "

\n",
"\n",
"Figure 1. Population and samples.\n",
"\n",
"Figure 2. Distribution of the means of all possible samples coming from a given population (Anderson's population in this case).\n",
"

\n",
"\n",
"To understand this, it is useful to start by thinking about a population, in this case our population of Irises which has a mean petal length of $\\mu = 5.552$ cm, illustrated by the big black circle on Figure 1 on the right.\n",
"\n",
"Now imagine you take a sample of (i.e. a subset of), say, 50 flowers from this population, represented by the green circle on Figure 1. The mean petal length of this sample is $m_1 = 6.234$ cm. You then take a second sample of 50 flowers (another subset, in blue on Figure 1), which ends up having a mean petal length of $m_2 = 5.874$ cm. You then take a third sample of 50 which gives you a mean petal length of $m_3 = 5.349$ cm, in yellow on Figure 1.\n",
"\n",
"If you keep taking samples from this population, you will start to notice a pattern: while some of the samples will give a mean petal length which is not at all close to the population mean, most of the mean petal lengths are reasonably close to the population mean of 5.552 cm. Furthermore, the mean of the mean petal length of the samples will be the same as that of the population as a whole i.e. 5.552 cm. \n",
"\n",
"In fact, if we keep taking samples from this population, it turns out that the distribution of the mean of these samples will take a very particular pattern that looks like a normal curve, as illustrated by Figure 2 on the right. Actually, if you take bigger sample sizes (say 130 instead of 50) the distribution will get closer and closer to being a normal curve for which the mean is equal to the mean of the population. For these smaller samples, the distribution is called the **[Student's t-distribution](https://en.wikipedia.org/wiki/Student%27s_t-distribution)** (actually it is a family of distributions, which depend on the sample size).\n",
"\n",
"\n",
"This is useful because it allows us to rephrase our question as to how similar or different our sample from Vullierens Castle is to the population of Irises as described by Edgar Anderson. \n",
"**What we have from the Vullierens Castle is a sample**. We want to know if it is a sample that might have come from a population like that described by Edgar Anderson. We now know the shape (more or less a normal distribution) and the mean (5.552 cm) of all of the samples that could be taken from the population described by Edgar Anderson. **So our question becomes \"where does our sample fall on the distribution of all such sample means?\"**. \n",
"If our mean is in position A on the figure on the right, then it is plausible that our sample came from a population like that of Edgar Anderson. If our mean is in position B, then it is less plausible to believe that our sample came from a population like Anderson’s.\n",
"\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Significance level and cutoff point\n",
"\n",
"\n",
"\n",
"Figure 3. Distribution of the means of of all possible samples coming from Anderson's population with zones defined by the significance level $\\alpha=0.05$.

\n", "

\n",
"\n",
"\n",
"You might be wondering, how far away is far enough away for us to think it is implausible that our sample comes from a population like Anderson’s. The answer is, it depends on how sure you want to be. \n",
"\n",
"One common answer to this question is to be 95% sure - meaning that a sample mean would need to be in the most extreme 5% of cases before we would think it is implausible that our sample comes from a population like Anderson’s. This value of 5% is called **significance level** and it is noted $\\alpha$, with $\\alpha=0.05$. These most extreme 5% cases are represented by the zones in light blue on Figure 3. If the sample mean falls into these most extreme zones, we say that *the difference is \"statistically significant\"*.\n",
"\n",
"A second, common answer is 99% sure meaning that a sample mean would need to be in the most extreme 1% of cases before we would think it is implausible that our sample comes from a population like Anderson’s ($\\alpha=0.01$). \n",
"\n",
"In the following, **we will work on the basis of being 95% sure**.\n", "

\n", "Let's define our significance level $\\alpha=0.05$:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Define alpha at 0.05\n", "alpha05 = 0.05\n", "\n", "# Display alpha\n", "alpha05" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

\n",
" Question

\n", " In the code cell below, create another variable called

"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Define alpha at 0.01\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"jupyter": {
"source_hidden": true
}
},
"source": [
"\n",
"\n", " In the code cell below, create another variable called

`alpha01`

to define a significance level of $\\alpha = 0.01$.\n",
" You can check your answer by clicking on the \"...\" below.

\n", "\n",
" Solution

\n", " The cell below defines the variable

\n",
"\n",
"\n", " The cell below defines the variable

`alpha01`

and displays it.\n",
"\n", "# Define alpha at 0.01\n", "alpha01 = 0.01\n", "\n", "# Display alpha\n", "alpha01\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If our distribution of sample means is a normal curve then we know that the most extreme 5% of sample means are found above or below ±1.96 standard deviations above and below the mean. In our case, because our sample size is less than 130 (it is 50), our distribution is close to normal but not quite normal. \n", "In this case, it is possible to find out the relevant cut off point from [looking it up in statistical tables](https://en.wikipedia.org/wiki/Student%27s_t-distribution#Table_of_selected_values): for a sample size of 50, the most extreme 5% of cases are found above or below approximately 2.01 standard deviations from the mean. \n", "\n", "The good news is that **Python gives us automatically the value of the cutoff point** based on the value of the significance level $\\alpha$ chosen and the sample size, thanks to the `stats` library which offers useful functions related to many statistical distributions such as Student's t:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Get the cutoff point for alpha at 0.05\n", "cutoff05 = stats.t.isf(alpha05 / 2, sample_size)\n", "\n", "# Display cutoff\n", "cutoff05" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

\n",
" Question

\n", " How would you get the value of the cutoff point for the significance level $\\alpha = 0.01$?

\n", " Type and test your code using the cell below.\n", "

"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Get the cutoff point for alpha at 0.01\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"jupyter": {
"source_hidden": true
}
},
"source": [
"\n",
"\n", " How would you get the value of the cutoff point for the significance level $\\alpha = 0.01$?

\n", " Type and test your code using the cell below.\n", "

You can check your answer by clicking on the \"...\" below.

\n", "\n",
" Solution

\n", " To get the value of the cutoff point for the significance level of $\\alpha = 0.01$, we can copy the first line of code

\n", " To save the value of this new cutoff point for later, it is good to store it in a new variable

\n", " See the solution code below.\n", "

\n",
"\n",
"\n", " To get the value of the cutoff point for the significance level of $\\alpha = 0.01$, we can copy the first line of code

`stats.t.isf(alpha05 / 2, sample_size)`

and replace `alpha05`

with the variable `alpha01`

that we have previously defined.\n", " To save the value of this new cutoff point for later, it is good to store it in a new variable

`cutoff01`

.\n", " See the solution code below.\n", "

\n", "# Get the cutoff point for alpha at 0.01\n", "cutoff01 = stats.t.isf(alpha01 / 2, sample_size)\n", "\n", "# Display cutoff\n", "cutoff01\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Error in the distribution of means\n", "\n", "So far we know a lot that will help us to test the hypothesis that our sample mean is similar to Anderson’s population mean. We know:\n", "* Our sample mean $m$\n", "* The population mean $\\mu$\n", "* The shape of the distribution of the mean of all samples that would come from this population (a normal curve, centred on the population mean)\n", "* Our cut off point defined by $\\alpha$ (the most extreme 5% of cases, above or below 2.01 standard deviations from the mean)\n", "\n", "The last piece of information missing that would enable us to test this hypothesis is the size of the standard deviation of the distribution of sample means from Anderson’s population. \n", "It turns out that a good guess for the size of this standard deviation can be obtained from knowing the standard deviation of our sample.\n", "If $s$ is the sample standard deviation of our sample and $n$ is the sample size, then the standard deviation of the distribution of sample means is:\n", "\n", "$\n", "\\begin{align}\n", "\\sigma_{\\overline{X}} = \\frac{s}{\\sqrt{n}}\n", "\\end{align}\n", "$ \n", "\n", "This standard deviation of the distribution of sample means is called the **\"standard error of the mean\" (also noted SEM)**. \n", "We can compute it by using the sample size and the standard deviation from the descriptive stats we have computed earlier: " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Extract the sample standard deviation from the descriptive stats\n", "sample_std = sample_stats.loc[\"std\",\"petal_length\"]\n", "\n", "# Compute the estimation of the standard deviation of sample means from Anderson's population (standard error)\n", "sem = sample_std / math.sqrt(sample_size)\n", "\n", "# Display the standard error\n", "sem" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

\n",
" Question

\n", " In the code above, what function is used to compute the square root of the sample size, $\\sqrt{n}$?

\n", " How would you compute the square root of 2?

\n", " Type and test your code using the cell below.\n", "

"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Compute the square root of 2 and display the result\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"jupyter": {
"source_hidden": true
}
},
"source": [
"\n",
"\n", " In the code above, what function is used to compute the square root of the sample size, $\\sqrt{n}$?

\n", " How would you compute the square root of 2?

\n", " Type and test your code using the cell below.\n", "

You can check your answer by clicking on the \"...\" below.

\n", "\n",
" Solution

\n", " We see in the code making the calculation of the standard error of the mean (

\n", " Therefore we can replace

\n", " See the solution code below.\n", "

\n",
"\n",
"\n", " We see in the code making the calculation of the standard error of the mean (

`sem`

) above that the way to get $\\sqrt{n}$ in Python is `math.sqrt(sample_size)`

.\n", " Therefore we can replace

`sample_size`

by `2`

to get $\\sqrt{2}$.\n", " See the solution code below.\n", "

\n", "# Compute the square root of 2\n", "sqrt2 = math.sqrt(2)\n", "\n", "# Display the result\n", "sqrt2\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Comparison, and definition of the *t* statistics\n", "\n", "We can now restate our question in more precise terms: **\"is our sample mean in the most extreme 5% of samples that would be drawn from a population with the same mean as Anderson’s population?\"**. \n", "Or to be even more precise, **\"is the gap between our sample mean and Anderson’s population mean greater than 2.01 times the standard error of the mean?\"**. \n", "\n", "This would be equivalent to compare\n", "$\n", "\\begin{align}\n", "\\frac{m - \\mu}{\\sigma_{\\overline{X}}}\n", "\\end{align}\n", "$\n", "to our cutoff point of 2.01. \n", "\n", "That is the **definition of the *t* statistics**: the value $t = $\n", "$\n", "\\begin{align}\n", "\\frac{m - \\mu}{\\sigma_{\\overline{X}}}\n", "\\end{align}\n", "$ \n", " has to be compared to the cutoff point we have chosen to determine if the sample mean falls into the most extreme zones and to be able to say whether the difference is statistically significant or not.

\n", "Let's compute $t$:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Compute the t statistics:\n", "t = (sample_mean - mu) / sem\n", "\n", "# Display t\n", "t" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we can compare $t$ to our cutoff point. \n", "\n", "One issue here is that **when $m$ is smaller than $\\mu$, the value of $t$ can be negative**. This is because, just like for the Normal distribution, Student's t-distribution is symmetrical and centred on zero, zero meaning there is no difference between the mean of the sample and the mean of the population. So when comparing $t$ to the cutoff point, either we take its absolute value, which is what we do below, or if $t$ is negative we compare it to the negative value of the cutoff point (i.e. -2.01 for a significance level of 0.05)." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Compare t with our cutoff point\n", "if abs(t) > cutoff05: \n", " print(\"The difference IS statistically significant.\")\n", "else: \n", " print(\"The difference is NOT statistically significant.\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We see in the results above that for our Vullierens sample $|t| > 2.01$, therefore the difference between the two means is greater than 2.01 times the standard error. In other words, **our sample mean IS in the most extremes 5%** of samples that would be drawn from a population with the same mean as Anderson's population. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

\n",
" Question

\n", " How would you compare $|t|$ to the cutoff point corresponding to a significance level of $\\alpha = 0.01$?

\n", " Type and test your code using the cell below.\n", "

"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Compare t to the cutoff point for alpha=0.01\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"jupyter": {
"source_hidden": true
}
},
"source": [
"\n",
"\n", " How would you compare $|t|$ to the cutoff point corresponding to a significance level of $\\alpha = 0.01$?

\n", " Type and test your code using the cell below.\n", "

You can check your answer by clicking on the \"...\" below.

\n", "\n",
" Solution

\n", " To compare the absolute value of $t$ to the cutoff point corresponding to $\\alpha = 0.01$, we can simply replace

\n", " In this case, the comparison would tell us if our sample mean is in the most extremes 1% of samples that would be drawn from a population with the same mean as Anderson's population. We already know that our sample mean IS in the most extremes 5%, but the result here shows that it is NOT in the most extremes 1%. \n", "

\n",
"\n",
"\n", " To compare the absolute value of $t$ to the cutoff point corresponding to $\\alpha = 0.01$, we can simply replace

`cutoff05`

in the code above by the variable `cutoff01`

we have defined earlier with the appropriate value for the cutoff point. See the solution code below.\n", " In this case, the comparison would tell us if our sample mean is in the most extremes 1% of samples that would be drawn from a population with the same mean as Anderson's population. We already know that our sample mean IS in the most extremes 5%, but the result here shows that it is NOT in the most extremes 1%. \n", "

\n", "# Compare t to the cutoff point for alpha=0.01\n", "if abs(t) > cutoff01: \n", " print(\"The difference IS statistically significant.\")\n", "else: \n", " print(\"The difference is NOT statistically significant.\")\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The statistical test we have just performed here, where we compare our sample mean to the mean of a population, is called a **one-sample t-test**: *one-sample* because we compare a sample to the mean of a population, and *t-test* because the distribution of all the possible sample means of the population follows a distribution called *Student's t-distribution*. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Visualization of *t*\n", "\n", "Using Python we can visualize what the t-test means graphically by plotting the t-distribution of all the possible sample means that would be drawn from a population with the same mean as Anderson's population and showing where `t` is in the distribution compared to the zone defined by our $\\alpha$ of 5%.\n", "\n", "It the *t* statistics falls outside of the rejection zone defined by $\\alpha$, then that means that the difference between our sample mean and the population mean is not statistically significant. If it falls into the rejection zone, then the difference is statistically significant and the sample should not be considered as coming from the Anderson population under the significance level we have chosen.\n", "\n", "The cell below uses an external library to generate a graphical visualization of the result of the t-test." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Visualize graphically the result of the t-test with alpha at 0.05\n", "visualize_ttest(sample_size, alpha05, t)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

\n",
" Question

\n", " What happens to the rejection zone in red on the figure when we choose an $\\alpha$ of 1%?

\n", " Type and test your code using the cell below.\n", "

"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Visualize graphically the result of the t-test with alpha at 0.01\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"jupyter": {
"source_hidden": true
}
},
"source": [
"\n",
"\n", " What happens to the rejection zone in red on the figure when we choose an $\\alpha$ of 1%?

\n", " Type and test your code using the cell below.\n", "

You can check your answer by clicking on the \"...\" below.

\n", "\n",
" Solution

\n", " To visualize the rejection zone for an $\\alpha$ of 1%, we can simply replace

\n", " By comparing the two visualizations, we see that the rejection zone for $\\alpha=0.01$ is much smaller than for $\\alpha=0.05$, which means we want to reject only samples that have a mean extremely different from the mean of the Anderson population.\n", "

\n",
"\n",
"\n", " To visualize the rejection zone for an $\\alpha$ of 1%, we can simply replace

`alpha05`

in the code above by the variable `alpha01`

we have defined earlier. See the solution code below.\n", " By comparing the two visualizations, we see that the rejection zone for $\\alpha=0.01$ is much smaller than for $\\alpha=0.05$, which means we want to reject only samples that have a mean extremely different from the mean of the Anderson population.\n", "

\n", "# Visualize graphically the result of the t-test with alpha at 0.01\n", "visualize_ttest(sample_size, alpha01, t)\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Conclusion\n", "\n", "What can we conclude from there? What the one sample t-test tells us is that we have evidence which would lead us to think that the sample doesn't come from an Anderson like population. Therefore we **can reject our hypothesis $H_0$**. \n", "\n", "Now there are some limitations to keep in mind when using the one sample t-test, that we will explore in the section below.\n", "\n", " \n", "\n", "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Influence of the sample size\n", "\n", "Above, we have seen that $t = $\n", "$\n", "\\begin{align}\n", "\\frac{m - \\mu}{\\sigma_{\\overline{X}}}\n", "\\end{align}\n", "$ and that $\\sigma_{\\overline{X}} = $\n", "$\n", "\\begin{align}\n", "\\frac{s}{\\sqrt{n}}\n", "\\end{align}\n", "$.\n", "\n", "Therefore we can rewrite the *t* statistics as:\n", "\n", "$\n", "\\begin{align}\n", "t = \\frac{m - \\mu}{\\frac{s}{\\sqrt{n}}}\n", "\\end{align}\n", "$\n", "\n", "This means that *t* is actually:\n", "\n", "$\n", "\\begin{align}\n", "t = \\frac{m - \\mu}{s}\\sqrt{n}\n", "\\end{align}\n", "$\n", "\n", "From there, we see that the **sample size $n$ influences the value of $t$**: all else being equal (i.e. sample mean, sample standard deviation and population mean), **a larger sample would result in a higher value of $t$** and therefore more chances to find a significant result for the t-test.\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

\n",
" Question

\n", " For our irises from the Vullierens Castle, which sample size would make the value of $t$ reach our cutoff point of 2.01, all else being equal (i.e. with identical sample mean, sample standard deviation and population mean)?

\n", " Use the code cell below to write your answer in Python and test it.\n", "

"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Make your calculation in Python here\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"jupyter": {
"source_hidden": true
}
},
"source": [
"\n",
"\n", " For our irises from the Vullierens Castle, which sample size would make the value of $t$ reach our cutoff point of 2.01, all else being equal (i.e. with identical sample mean, sample standard deviation and population mean)?

\n", " Use the code cell below to write your answer in Python and test it.\n", "

You can check your answer by clicking on the \"...\" below.

\n", "\n",
" Solution

\n", "We want to find the value of $n$ that would make $t$ at least equal to 2.01.

\n", "We know that $t = $\n", "$\n", "\\begin{align}\n", "\\frac{m - \\mu}{s}\\sqrt{n}\n", "\\end{align}\n", "$.\n", "

In other words, we are looking for the value of $n$ such as:\n", "$\n", "\\begin{align}\n", "\\frac{m - \\mu}{s}\\sqrt{n} = 2.01\n", "\\end{align}\n", "$.

\n", "We can rewrite this expression to find $n$, which gives: \n", "$\n", "\\begin{align}\n", "n = \\left(\\frac{2.01 s}{m - \\mu}\\right)^2\n", "\\end{align}\n", "$ with $s$ the sample standard deviation, $m$ the sample mean and $\\mu$ the population mean.

\n", "

\n", "Then we have to write this in Python using the variables we have defined earlier and display the result. For squaring, either we just replace by a multiplication, which is what we have done below, or we use the Python operator

\n", "We obtain

\n",
"\n",
"\n", "We want to find the value of $n$ that would make $t$ at least equal to 2.01.

\n", "We know that $t = $\n", "$\n", "\\begin{align}\n", "\\frac{m - \\mu}{s}\\sqrt{n}\n", "\\end{align}\n", "$.\n", "

In other words, we are looking for the value of $n$ such as:\n", "$\n", "\\begin{align}\n", "\\frac{m - \\mu}{s}\\sqrt{n} = 2.01\n", "\\end{align}\n", "$.

\n", "We can rewrite this expression to find $n$, which gives: \n", "$\n", "\\begin{align}\n", "n = \\left(\\frac{2.01 s}{m - \\mu}\\right)^2\n", "\\end{align}\n", "$ with $s$ the sample standard deviation, $m$ the sample mean and $\\mu$ the population mean.

\n", "

\n", "Then we have to write this in Python using the variables we have defined earlier and display the result. For squaring, either we just replace by a multiplication, which is what we have done below, or we use the Python operator

`**`

for power raising ($x^2$ is then written `x ** 2`

). See the solution code below, in which we have used the variable `cutoff05`

instead of the raw number `2.01`

but that would work too.\n", "We obtain

`n = 41.88`

, which means that a sample size of 42 flowers or more with the same mean and standard deviation for the petal length would make the t-statistic above our cutoff point.\n",
"\n", "# Make your calculation in Python here\n", "n = ((cutoff05 * sample_std) / (sample_mean - mu)) * ((cutoff05 * sample_std) / (sample_mean - mu))\n", "\n", "# Display the result\n", "n\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "So for instance, for our irises from the Vullierens Castle, **a sample of 144 flowers instead of 50** with exactly the same mean and standard deviation for the petal length would be considered as statistically different from the Anderson population. \n", "\n", "This is why when doing experiments, researchers generally try to get samples as large as possible - but of course this has a cost and is not always possible!\n", "\n", " \n", "\n", "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Using the *p-value*\n", "\n", "In scientific studies, researchers use frequently the t-test but they generally report not only the t-statistic but also **another result of the t-test which is called the p-value**. In the following, we explore what is the p-value, how it relates to the t-statistic and how it can be used." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Testing our hypothesis using a predefined Python function\n", "\n", "So far we have made the computations by hand but Python comes with a number of libraries with interesting statistical tools. \n", "In particular, the `stats` library includes a function for doing a **one-sample t-test** as we have done above. \n", "\n", "Let's now use it and then look at what information it gives us." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# Compute the t-test\n", "t, p = stats.ttest_1samp(sample_data[\"petal_length\"], mu)\n", "\n", "# Display the result\n", "print(\"t = {:.3f}\".format(t))\n", "print(\"p = {:.3f}\".format(p))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We see that the predefined Python function for doing the one-sample t-test gives us the same value for the $t$ statistic as the calculations we have made by hand: $t = 2.194$. \n", "In addition, we see that it also returns another value, $p = 0.033$. \n", "\n", "Actually, the two values `t` and `p` returned by the function say the same thing but in two different ways:\n", "* `t` tells us where our sample mean falls on the distribution of all the possible sample means for the Anderson population ;

\n", " `t` has to be compared to the cutoff value (2.01) to know if our sample mean is in the most extremes 5%.\n", "* `p` is **called the \"p-value\"** and is the **probability to get a more extreme sample mean** than the one we observe ;

\n", " `p` has to be compared to $\\alpha$ (0.05) to know if our sample mean is in the most extremes 5%.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

\n",
" Question

\n", " How does

\n", " And how does

\n", "

"
]
},
{
"cell_type": "markdown",
"metadata": {
"jupyter": {
"source_hidden": true
}
},
"source": [
"\n",
"\n", " How does

`t`

compare to the cutoff value (2.01)?\n", " And how does

`p`

compare to $\\alpha$ (0.05)?\n", "

You can check your answer by clicking on the \"...\" below.

\n", "\n",
" Solution

\n", " \n", "We see above that:\n", "* $t = 2.294$ therefore $|t| > 2.01$, which means that the difference between the two means is larger than 2.01 times the standard error \n", "* and $p = 0.033$ therefore $p < 0.05$, which means that the probability of getting more extreme sample mean than the one we observe is smaller than 5% so our sample mean can be considered as one of the 5% most extreme possible values. \n", "\n", "

"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"As expected from the calculations we have made by hand above, the test using the predefined Python function confirms that the difference between the mean petal length of the Vullierens sample and the mean petal length of Anderson's population is **not statistically significant**.\n",
"\n",
"As we have just seen, **you can use either `t` or `p` to interpret the result of the t-test.** In practice, most people use the p-value because it can be directly compared to $\\alpha$ without having to look for the cutoff value in tables. However, as we will see more in details below, **`t` and `p` do not provide exactly the same information about the result of the test**, and it is important to understand how they differ."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Visualization of the p-value\n",
"\n",
"Using Python we can visualize what the t-test graphically by plotting the t-distribution of all the possible sample means that would be drawn from a population with the same mean as Anderson's population and showing where `t` is in the distribution compared to the zone defined by our $\\alpha$ of 5%.\n",
"\n",
"In addition to displaying the value of *t*, the visualization below also **shows the *p-value*** (represented by the hatched zone), which is the **area under the curve of the t-distribution** representing the probability of getting a more extreme sample mean than the one we observe. When this area is larger than the rejection zone defined by the $\\alpha$ we have chosen, then that means the difference between the sample mean and the population mean is not statistically significant."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Visualize graphically the result of the t-test and the p-value with alpha at 0.05\n",
"visualize_ttest_pvalue(sample_size, alpha05, t, p)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n", " \n", "We see above that:\n", "* $t = 2.294$ therefore $|t| > 2.01$, which means that the difference between the two means is larger than 2.01 times the standard error \n", "* and $p = 0.033$ therefore $p < 0.05$, which means that the probability of getting more extreme sample mean than the one we observe is smaller than 5% so our sample mean can be considered as one of the 5% most extreme possible values. \n", "\n", "

\n",
" Question

\n", " What values does the function

\n", " In the code cell below, use this function to generate the visualization of a value of $t=-1.702$ and $p=0.095$ with the same sample size and same value for $\\alpha$ as in the example above.

\n", " What can you observe with this negative value for $t$?

\n", " How would you say that the p-value (the hatched zone) evolves when $|t|$ gets bigger?

\n", "

"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Visualize graphically the result of the t-test and the p-value with alpha at 0.01\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"jupyter": {
"source_hidden": true
}
},
"source": [
"\n",
"\n", " What values does the function

`visualize_ttest_pvalue`

need to generate a visualization of the result of a t-test?\n", " In the code cell below, use this function to generate the visualization of a value of $t=-1.702$ and $p=0.095$ with the same sample size and same value for $\\alpha$ as in the example above.

\n", " What can you observe with this negative value for $t$?

\n", " How would you say that the p-value (the hatched zone) evolves when $|t|$ gets bigger?

\n", "

You can check your answer by clicking on the \"...\" below.

\n", "\n",
" Solution

\n", " The

\n", " To generate a new visualization, we can simply copy-paste the

\n", " When comparing with the previous graph, we see that the hatched zone representing $p$ is larger than the red zone that represents alpha.\n", "\n", "

\n",
"\n",
"\n", " The

`visualize_ttest_pvalue`

function needs 4 different values to generate the visualization: the sample size, the significance level $\\alpha$, the value of $t$ and the value of $p$.\n", " To generate a new visualization, we can simply copy-paste the

`visualize_ttest_pvalue`

function of the code cell above and replace `alpha05`

by `alpha01`

, see the solution code below.\n", " When comparing with the previous graph, we see that the hatched zone representing $p$ is larger than the red zone that represents alpha.\n", "\n", "

\n", "# Visualize graphically the result of a t-test of t=-1.702 and p=0.095\n", "visualize_ttest_pvalue(sample_size,alpha01, t, p)\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Thanks to the visualization above, we see that one important difference between the t-statistic and the p-value is that that $|t|$ and $p$ evolve in opposite directions:

\n",
" \n",
"\n",
"###### Iris Ensata (Credit: Laitche CC BY-SA 3.0)\n",
"\n",
"

\n",
"\n",
"Let's imagine we want to know how our Vullierens sample compares to another iris population, for instance a Japanese iris species called Iris Ensata with a mean petal length of $\\mu_{ensata} = 5.832$ cm. We can apply the hypothesis testing approach that we have just learned and use a one-sample t-test to do the comparison, which we do in 4 steps:\n",
"\n",
"1. Get an idea about how the sample compares to the population: \n",
" At first sight, the sample petal mean of our Vullierens sample, $m= 5.646$ cm, is again quite close to the mean petal length of the Ensata population, $\\mu_{ensata} = 5.832$ cm. \n",
"\n",
"2. Formulate the hypotheses we want to test:\n",
" * First let's state our null hypothesis, which is that the Vullierens sample is similar to the Iris Ensata population, $H_0: m = \\mu_{ensata}$. \n",
" This is the hypothesis we want to know whether we can reject or not.\n",
" * And then state the alternate hypothesis $H_a: m \\neq \\mu_{ensata}$.\n",
"\n",
"\n",
"3. Choose a significance level: \n",
" Let's choose $\\alpha=0.05$ as previously, which means we want to be 95% sure.\n",
"\n",
"4. Compute the result of the t-test by using the predefined Python function `stats.ttest_1samp`

:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Define the mean petal length of the Ensata population\n",
"mu_ensata = 5.832\n",
"\n",
"# Compute the t-test comparing the Vullierens sample petal length to the Ensata population mean\n",
"t, p = stats.ttest_1samp(sample_data[\"petal_length\"], mu_ensata)\n",
"\n",
"# Display the result\n",
"print(\"t = {:.3f}\".format(t))\n",
"print(\"p = {:.3f}\".format(p))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The result of the t-test gives $t = -1.621$ and $p = 0.111$.\n",
"\n",
"With $\\alpha=0.05$, the cutoff value is 2.01. We see that $|t| < 2.01$ and $p > 0.05$. Therefore, the test tells us that the difference between the mean petal length of the Vullierens sample and the mean petal length of the Ensata population \n",
- " Question

\n", - " In the code cell below, use the function

\n", - " How do you interpret the results?

\n", - "

"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {},
- "outputs": [],
- "source": [
- "# Visualize graphically the result of the t-test with alpha05\n",
- "\n",
- "# Visualize graphically the result of the t-test with alpha01\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "jupyter": {
- "source_hidden": true
- }
- },
- "source": [
- "\n",
- "\n", - " In the code cell below, use the function

`visualize_ttest_pvalue`

to generate two visualizations of the result of this t-test: one with $\\alpha=0.05$ and then another with $\\alpha=0.01$.\n", - " How do you interpret the results?

\n", - "

You can check your answer by clicking on the \"...\" below.

\n", - "\n",
- " Solution

\n", - " We copy-paste the code of the cell above with the

\n", - " When comparing the two visualizations we see that, while $t$ is at the same place in both, the rejection zone is much smaller in the second visualization i.e. when $\\alpha=0.01$, reflecting the fact that we want to be sure that the sample is*extremely different* from the population (i.e. is part of the 1% most extreme possible samples for a population with this mean petal length) before rejecting our null hypothesis $H_0$.

\n", - " As a consequence, $t$ falls into the rejection zone in the first visualization but not in the second, which means that when choosing $\\alpha = 0.01$, we**cannot reject anymore our null hypothesis $H_0$**. The conclusion in this case is that, with $\\alpha=0.01$, the difference between the mean petal length of the Vullierens sample and the mean petal length the Ensata population cannot be considered as statistically significant.\n",
- "

\n",
- "\n",
- "\n", - " We copy-paste the code of the cell above with the

`visualize_ttest_pvalue`

function twice: in one we use `cutoff05`

and in the other we use `cutoff01`

, the other parameters remaining the same, see the code cell below.\n", - " When comparing the two visualizations we see that, while $t$ is at the same place in both, the rejection zone is much smaller in the second visualization i.e. when $\\alpha=0.01$, reflecting the fact that we want to be sure that the sample is

\n", - " As a consequence, $t$ falls into the rejection zone in the first visualization but not in the second, which means that when choosing $\\alpha = 0.01$, we

\n", - "# Visualize graphically the result of the t-test with alpha05\n", - "visualize_ttest_pvalue(sample_size, alpha05, t, p)\n", - "\n", - "# Visualize graphically the result of the t-test with alpha01\n", - "visualize_ttest_pvalue(sample_size, alpha01, t, p)\n", - "" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "For $\\alpha = 0.01$, the cutoff value which we get from the tables is 2.67. With this choice of $\\alpha$, we see that $|t| < 2.67$ and $p > 0.01$. This means that when choosing $\\alpha = 0.01$, the test tells us that the difference between the mean petal length of the Vullierens sample and the mean petal length the Ensata population