Interpreting P-values
tags: #hypothesis_testing
What are alpha values?
The alpha significance level (
- i.e., the alpha significance represents the threshold/cut-off point in determining the maximum allowable probability at which we are willing to accept the risk and reject the null.
When interpreting p-values against the the alpha (e.g., 0.05), we are essentially saying that, our conclusion only leads to a false positive 5% of the time when the truth is that there is no effect.
How do we interpret p-values?
P-values are the probability of observing a sample statistic that is at least as extreme as your sample statistic when you assume that the null hypothesis is true (i.e., the fraction of times you would observe a value as extreme as your statistic under the null).
P-values are used as is a measure of statistical significance (says nothing about the magnitude of the effect if there is an effect to be found. This is simply a question of whether a relationship appears to exist statistically or not.
If
- This means that the probability of observing the value if the null were true is low
- Relationship between variables does exist (i.e., not independent of each other)
- Observed relationship is not due to chance due to e.g., sampling error
If
- the probability of observing the value if the null were true is high
- variables are independent of each other
- insufficient evidence to suggest that the observed relationship is NOT due to chance
Misconceptions about rejecting the null hypothesis
A hypothesis test is a test to reject or not reject the null.
NOT whether not reject the null OR to reject the null AND accept the alternative.
i.e., when we reject a null hypothesis (such that the effect is unlikely to have occurred by chance/under if the null was true), this DOES NOT MEAN that the alternative is True.
HYPOTHESIS TESTING DOES NOT PROVE ANYTHING!!