tag:blogger.com,1999:blog-5747401412673787565.post7819287505716390825..comments2024-03-18T06:19:28.852+00:00Comments on NeuroChambers: Scientific publishing as it was meant to beChris Chambershttp://www.blogger.com/profile/10437328364681252945noreply@blogger.comBlogger3125tag:blogger.com,1999:blog-5747401412673787565.post-3345743929239710912013-05-03T05:35:29.320+01:002013-05-03T05:35:29.320+01:00The problem with reinventing the hypothesis of a s...The problem with reinventing the hypothesis of a study is that the study was not conducted again with the new hypothesis. It's fine to look at data and interpret it in different ways, but that's the start of the scientific process not the end.<br /><br />See Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological science, 22(11), 1359–66.I Am A Robotnoreply@blogger.comtag:blogger.com,1999:blog-5747401412673787565.post-28034535252603550732013-04-30T05:44:22.042+01:002013-04-30T05:44:22.042+01:00Also "posthoc storytelling: reinventng hypoth...Also "posthoc storytelling: reinventng hypotheses<br />to“predict” unexpected results is very similar to "inference to the best explanation" which has been the very heart of much of science. Without it "The Origin of the Species" would have no unifying argument. <br /><br />Perhaps the real problem is not with postrationalised hypotheses, but with a lack of discipline on what counts as a good explanation of the results. David Deutch has written and spoken about the dangers of explanationless science, and on the importance of good explanations and on the importance of good criteria for what counts as a good explanation. Even the arch-empiricist Karl Popper spent a great deal of time in his writing detailing what counts as an informative hypothesis/theory and what does not. Perhaps it isn't the post-hoc hypothesizing that is the matter, but on the quality of the hypotheses as explanations, and on the lack of consensus on what logical criteria scientists should use in considering an explanatory hypothesis a good one. Back to Popper, there is a word of difference between an ad hoc "saving the phenomena" type hypothesis, and post-hoc explanation and one that increases explanatory and informative content.<br /><br />Inference to best explanation as a principle in fact lurks underneath much of statistics. Many tests rely on Maximum likelyhood, which is actually a probablistic form of best explanation, and Bayesian statistics has similar assumptions. Without inference to best explanation (post-hoc explantations) science would grind to a halt.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-5747401412673787565.post-71560976680711188982013-04-30T05:34:31.266+01:002013-04-30T05:34:31.266+01:00Hi, I published a little rant in BJP last year bas...Hi, I published a little rant in BJP last year basically on how journals aren't prepared to publish the reality of data if it doesn't fit with a preconceived schema of how a successful experiment will turn out. I think you're scheme is an excellent idea - disciplining not only scientists to lift our game, but also editorial boards to accept the real results of registered experiments, even if "outcome knowledge' (i.e., hindsight) invalidates the original/registered experimental plan midway (indeed, that is my only concern, because outcome knowledge so often changes how we view our experimental design, will scientists find themselves under pressure to massage their results to fit better with how things were originally planned and registered? Will it just shunt the dishonesty to another level? I'm eager to see how it goes!)<br /><br />John AshtonAnonymousnoreply@blogger.com