Huberman promoted a lot of studies that were clearly p-hacked. As it turns out, a large portion of all the papers in various fields are p-hacked. A clear indication of this fact is that the proportions of all significant results which are between 0.05 and 0.01 are enormous.
If you want to know more, go check out this post: cremieux.xyz/p/ranking-fiel…
@cremieuxrecueil It is impossible to eliminate p-hacking with the current publishing scheme and incentives though. Only if studies were preregistered which would also allow negative results to be published.
@cremieuxrecueil Studies are often powered to produce results in that range, so not too surprising.
@cremieuxrecueil can any LLM model reliably detect p-hacking?
@cremieuxrecueil Not necessarily the case that this implies p-hacking. Studies are deliberately sized (“powered”) such that the anticipated result should reach but not drastically exceed the p=0.05 threshold. If you’re getting p=0.000001, your study is larger than it needs to be.
@cremieuxrecueil What is the test for identifying p-hacking?