"History of science teaches us that scientific endeavor has often in the past wasted effort in fields with absolutely no yield of true scientific information."
(Ioannidis, 2005; see handout (.docx))
What is evidence? Things to keep in mind:
Who pays for science?
Most scientific research is funded by government, companies doing research and development, and non-profit entities. Because science is attempting to get at some "truth," the source of research funding shouldn't have a significant effect on the outcome of scientific research, right?
» Read Industry sponsorship and research outcome Cochrane Database Syst Rev. 2012 Dec 12;12:MR000033).
What to consider when looking at survey or estimated data:Adopted from information on the UCSF Family Health Outcomes Project web site
Reliability and validity
Adopted from Chapter 3, Conducting research literature reviews: from the Internet to paper, by Arlene Fink; Sage, 2010.
Reliable data collection is relatively free from "measurement error"
» Is the survey written at a reading level too high for the people completing it?
» Is the device used to measure elapsed time in an experiment accurate?
Validity refers to how well a measure assesses what it claims to measure
» If the survey is supposed to measure "quality of life," how is that concept defined?
» How accurately can this animal study of drug metabolism be extrapolated to humans?