# 有趣的P value(值) 知識6題小測驗，含解答。

• 0分或看不懂的（6題都沒有答對），建議再唸一次大二統計學

• 16-50分或猜的1-3題答對的，建議再唸一次碩士多變量分析 或進階統計學

• 51-83 1-2 題沒有答對，多讀幾次 Haller and Krauss (2002); Gigerenzer (2004); Cohen (1988,1992)

• 100分的，統計Master太厲害了.

Rigor & Relevance(嚴謹和相關)是研究的兩大支柱，在研究相關性上，研究人員大多都會研究自己領域相關的問題，相關(Relevance)較好，因此，我們聚焦在研究的嚴謹性(Rigor)，關於顯著性p value的解釋，常常有些誤解，例如，可能對P值的過度解讀或統計概念的誤解。多年前的1個統計研究，測試研究人員正確理解P值能力Gigerenzer 2004 ，英文原文如下：

Suppose you have a treatment that you suspect may alter performance on a certain task. You compare the means of your control and experimental groups (say 20 subjects in each sample). Further, suppose you use a simple independent means t-test and your result is significant (= 2.7, d.f. = 18, = 0.01). Please mark each of the statements below as “true” or “false.” “False” means that the statement does not follow logically from the above premises. Also note that several or none of the statements may be correct.

1.You have absolutely disproved the null hypothesis (that is, there is no difference between the population means).

[] true/false []

2. You have found the probability of the null hypothesis being true.

[] true/false []

3. You have absolutely proved your experimental hypothesis (that there is a difference between the population means).

[] true/false []

4. You can deduce the probability of the experimental hypothesis being true.

[] true/false []

5.You know, if you decide to reject the null hypothesis, the probability that you are

making the wrong decision.

[] true/false []

6. You have a reliable experimental finding in the sense that if, hypothetically, the experiment were repeated a great number of times, you would obtain a significant

result on 99% of occasions.

[] true/false []

Which statements are in fact true? Recall that a p-value is the probability of the observed data (or of more extreme data points), given that the null hypothesis H0 is true, defined in symbols as p(D|H0).This definition can be rephrased in a more technical form by introducing the statistical model underlying the analysis (Gigerenzer et al., 1989, chapter 3).

Statements 1 and 3 are easily detected as being false, because a significance test can never disprove the null hypothesis or the (undefined) experimental hypothesis. They are instances of the illusion of certainty (Gigerenzer, 2002).

Statements 2 and 4 are also false. The probability p(D|H0) is not the same as p(H0|D), and more generally, a significance test does not provide a probability for a hypothesis.

The statistical toolbox, of course, contains tools that would allow estimating probabilities of hypotheses, such as Bayesian statistics. Statement 5 also refers to a probability of a hypothesis. This is because if one rejects the null hypothesis, the only possibility of making a wrong decision is if the null hypothesis is true. Thus, it makes essentially the same claim as Statement 2 does, and both are incorrect.

Statement 6 amounts to the replication fallacy (Gigerenzer, 1993, 2000). Here, p=1% is taken to imply that such significant datawould reappear in 99% of the repetitions. Statement 6 could be made only if one knew that the null hypothesiswas true. In formal terms, p(D|H0) is confused with 1p(D).

To sum up, all six statements are incorrect. Note that all six err in the same direction of

wishful thinking: They make a p-value look more informative than it is.

P值的定義是：當虛無假設（零假設）為真，與樣本結果相同的概率，表示為PD/H0

-----------------------

1.     你完全否定了虛無假設（零假設）（虛無假設（零假設）指的是兩個總體的均數没有差異。） /錯。

2.     你找到了虛無假設（零假設）為真的概率。對/錯。

3.     你完全證明了實驗假說（實驗假說指的是兩個總體的均數存在差異）對/錯。

4.     可以推斷出實驗假說為真的概率。對/錯。

5.     假如你决定要拒絕零假設，你已經知道這個决定是錯的概率。對/錯。

6.     你得到了一個可靠的研究結果，原因為：假如這個實驗被重複了非常多次，你將有99%的可能獲得一個顯著的結果。對/錯。

1)     H0 is absolutely disproved

2)     Probability of H0 is found

3)     H1 is absolutely proved

4)     Probability of H1 is found

5)     Probability of Type I error

6)     Probability of replication

P值的定義：當零假設為真時，觀察值的概率用符號表示：P(D/H0)

1和題3是錯的，因為一個顯著的測試從未否定H0H1

2和題4是錯的，P值表示P(D/H0)，而題2和題4指的是P(H0/D)，是不一樣的意思。

5也是錯的，題5參考題3假設的幾率，結果和題2相同，概念都是錯的。

6是錯的，題6P = 1%是指顯著資料重複99%，也就是1-P(D)，而原始P值表示P(D/H0)，不等於1-P(D)

• Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.

• Cohen, J. (1992). A Power Primer. Psychological Bulletin, 112(1), 155-159.

• Haller, H., Krauss, S., 2002. Misinterpretations of significance: a problem students share with their teachers? Methods of Psychological Reseach.

• Gerd Gigerenzer, Mindless statistics, The Journal of Socio-Economics 33 (2004) 587–606

https://m.sciencenet.cn/blog-3444471-1322621.html

## 萧文龙

GMT+8, 2024-5-22 14:31