Hypothesis Testing
Introduction
Be sure to read through the definitions for this section before trying to make sense out of the following.The first thing to do when given a claim is to write the claim mathematically (if possible), and decide whether the given claim is the null or alternative hypothesis. If the given claim contains equality, or a statement of no change from the given or accepted condition, then it is the null hypothesis, otherwise, if it represents change, it is the alternative hypothesis.
The following example is not a mathematical example, but may help introduce the concept.
Example
"He's dead, Jim," said Dr. McCoy to Captain Kirk.Mr. Spock, as the science officer, is put in charge of statistically determining the correctness of Bones' statement and deciding the fate of the crew member (to vaporize or try to revive)
His first step is to arrive at the hypothesis to be tested.
Does the statement represent a change in previous condition?
- Yes, there is change, thus it is the alternative hypothesis, H1
- No, there is no change, therefore is the null hypothesis, H0
- H0 : Patient is alive.
- H1 : Patient is not alive (dead).
Possible states of nature (Based on H0)
- Patient is alive (H0 true - H1 false )
- Patient is dead (H0 false - H1 true)
Possible decisions (Based on H0 ) / conclusions (Based on claim )
- Reject H0 / "Sufficient evidence to say patient is dead"
- Fail to Reject H0 / "Insufficient evidence to say patient is dead"
Statisticians will never accept the null hypothesis, we will fail to reject. In other words, we'll say that it isn't, or that we don't have enough evidence to say that it isn't, but we'll never say that it is, because someone else might come along with another sample which shows that it isn't and we don't want to be wrong.
Statistically (double) speaking ...
State of Nature | ||
Decision | H0 True | H0 False |
Reject H0 | Patient is alive,Sufficient evidence of death | Patient is dead,Sufficient evidence of death |
Fail to reject H0 | Patient is alive,Insufficient evidence of death | Patient is dead,Insufficient evidence of death |
In English ...
State of Nature | ||
Decision | H0 True | H0 False |
Reject H0 | Vaporize a live person | Vaporize a dead person |
Fail to reject H0 | Try to revive a live person | Try to revive a dead person |
Were you right ? ...
State of Nature | ||
Decision | H0 True | H0 False |
Reject H0 | Type I Error alpha | Correct Assessment |
Fail to reject H0 | Correct Assessment | Type II Error beta |
Which of the two errors is more serious? Type I or Type II ?
Since Type I is the more serious error (usually), that is the one we concentrate on. We usually pick alpha to be very small (0.05, 0.01). Note: alpha is not a Type I error. Alpha is the probability of committing a Type I error. Likewise beta is the probability of committing a Type II error.
Conclusions
Conclusions are sentence answers which include whether there is enough evidence or not (based on the decision), the level of significance, and whether the original claim is supported or rejected.Conclusions are based on the original claim, which may be the null or alternative hypotheses. The decisions are always based on the null hypothesis
Original Claim | ||
Decision | H0 "REJECT" | H1 "SUPPORT" |
Reject H0 "SUFFICIENT" | There is sufficient evidence at the (alpha) level of significance to reject the claim that (restate original claim) | There is sufficient evidence at the (alpha) level of significance to support the claim that (restate original claim) |
Fail to reject H0 "INSUFFICIENT" | There is insufficient evidence at the (alpha) level of significance to reject the claim that (restate original claim) | There is insufficient evidence at the (alpha) level of significance to support the claim that (restate original claim) |
The process
There is one fundamental guideline when performing hypothesis testing.
All hypothesis testing is done under the assumption
|
- We make an assumption
- We then test that assumption by gathering data and looking at the results
- If the results are too unusual to have happened just by chance, then we reject our assumption
Examples
The math used to find these probabilities is explained in a later section. Just try to understand the process and the logic for now.Example 1: Is the coin fair?
- We make an assumption that the coin is fair, that is, that the probability of flipping a head on a single trial will be 1/2.
- We gather data and test that claim. Let's say that we flipped the coin 100 times and we came up with 55 heads. The probability of getting 55 heads in 100 tosses, if the probability of heads is really 1/2 is 0.3173.
- Remember that there is always sampling error. We should not expect to get exactly 50 heads every time that flip 100 coins, there will be some variation because the results are random. The question is whether there's too much variation to just be caused by random chance. What we have found is that we could get these results 31.73% of the time just by chance. Something that happens 31.73% of the time is not unusual at all. Normally, we reserve the "unusual" status for things that happen less than 5% of the time. So, we fail to reject our assumption that the probability of getting a head is 1/2 and don't have enough evidence to reject the claim that the coin is fair.
If the results are too unusual to happen just by chance,
|
No comments:
Post a Comment