tag:blogger.com,1999:blog-4912713243046142041.post3334726229700234783..comments2024-03-28T23:22:41.774-05:00Comments on TYWKIWDBI ("Tai-Wiki-Widbee"): A detailed discussion of p valuesMinnesotastanhttp://www.blogger.com/profile/01382888179579245181noreply@blogger.comBlogger4125tag:blogger.com,1999:blog-4912713243046142041.post-45516218640832143482017-08-12T19:06:46.893-05:002017-08-12T19:06:46.893-05:00The other downside of this mindset is that it plac...The other downside of this mindset is that it places a lot of social science outside the bounds of "legitimacy." Available samples for hard-to-reach populations are necessarily smaller than those for large survey samples, making p-values lower than .05 more difficult to obtain.<br /><br />For example, I've been doing gang research. It'd be really difficult, and extremely expensive, to find a representative (e.g. non-convenience) sample of 1000 gang members, just to hit a higher threshold of p-value. Much of social science falls in this trap, because we work with underrepresented populations.<br /><br />Being held to that high standard of p-values would derail the legitimacy of our findings under a pretense that doesn't fully understand the meaning of p-values.Topherhttps://www.blogger.com/profile/03788630069714050544noreply@blogger.comtag:blogger.com,1999:blog-4912713243046142041.post-331578820688018022017-08-08T10:26:42.607-05:002017-08-08T10:26:42.607-05:00I've just added another link to the post that ...I've just added another link to the post that I think you will find i nteresting. Minnesotastanhttps://www.blogger.com/profile/01382888179579245181noreply@blogger.comtag:blogger.com,1999:blog-4912713243046142041.post-24390079642473802662017-08-05T21:08:16.132-05:002017-08-05T21:08:16.132-05:00Back when I was doing research in grad school, my ...Back when I was doing research in grad school, my advisor had a saying: "If you have to use statistics to defend yourself, it's probably not significant."Lois Tverberghttps://www.blogger.com/profile/05233392728601855385noreply@blogger.comtag:blogger.com,1999:blog-4912713243046142041.post-47523313742663552972017-08-05T11:27:35.591-05:002017-08-05T11:27:35.591-05:00This is a good article; thanks for sharing. As th...This is a good article; thanks for sharing. As the article says, this change would not solve any of the fundamental problems with using p-values to evaluate scientific merit. The American Statistical Association recently released a statement on p-values (available at http://amstat.tandfonline.com/doi/full/10.1080/00031305.2016.1154108?scroll=top&needAccess=true#_i27). The statement includes a few relevant points. Here are a few:<br />"Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold."<br /><br />"A p-value, or statistical significance, does not measure the size of an effect or the importance of a result.<br /><br />"By itself, a p-value does not provide a good measure of evidence regarding a model or hypothesis."<br /><br />Changing the cutoff from 0.05 to 0.005 would only serve to emphasize the importance of obtaining a small p-value even more; we should move away from this mindset and towards estimates of effect size.Anonymousnoreply@blogger.com