{"type":"rich","version":"1.0","provider_name":"Transistor","provider_url":"https://transistor.fm","author_name":"80,000 Hours Podcast","title":"#147 – Spencer Greenberg on stopping valueless papers from getting into top journals","html":"","width":"100%","height":180,"duration":9488,"description":"Can you trust the things you read in published scientific research? Not really. About 40% of experiments in top social science journals don't get the same result if the experiments are repeated. \r\n\r\nTwo key reasons are 'p-hacking' and 'publication bias'. P-hacking is when researchers run a lot of slightly different statistical tests until they find a way to make findings appear statistically significant when they're actually not — a problem first discussed over 50 years ago. And because journals are more likely to publish positive than negative results, you might be reading about the one time an experiment worked, while the 10 times was run and got a 'null result' never saw the light of day. The resulting phenomenon of publication bias is one we've understood for 60 years. \r\n\r\nToday's repeat guest, social scientist and entrepreneur Spencer Greenberg, has followed these issues closely for years. \r\n\r\nLinks to learn more, summary and full transcript. \r\n\r\nHe recently checked whether p-values, an indicator of how likely a result was to occur by pure chance, could tell us how likely an outcome would be to recur if an experiment were repeated. From his sample of 325 replications of psychology studies, the answer seemed to be yes. According to Spencer, \"when the original study's p-value was less than 0.01 about 72% replicated — not bad. On the other hand, when the p-value is greater than 0.01, only about 48% replicated. A pretty big difference.\" \r\n\r\nTo do his bit to help get these numbers up, Spencer has launched an effort to repeat almost every social science experiment published in the journals Nature and Science, and see if they find the same results. \r\n\r\nBut while progress is being made on some fronts, Spencer thinks there are other serious problems with published research that aren't yet fully appreciated. One of these Spencer calls 'importance hacking': passing off obvious or unimportant results as surprising and meaningful. \r\n\r\nSpencer suspects that importance...","thumbnail_url":"https://img.transistorcdn.com/VO1STE7hN95RRg9QdLo4soV2VhhbR9PF5ZZlRhDYcwE/rs:fill:0:0:1/w:400/h:400/q:60/mb:500000/aHR0cHM6Ly9pbWct/dXBsb2FkLXByb2R1/Y3Rpb24udHJhbnNp/c3Rvci5mbS9zaG93/LzQxNDAyLzE2ODM1/NDQ1NDAtYXJ0d29y/ay5qcGc.webp","thumbnail_width":300,"thumbnail_height":300}