Skip to main content
It looks like you're using Internet Explorer 11 or older. This website works best with modern browsers such as the latest versions of Chrome, Firefox, Safari, and Edge. If you continue with this browser, you may see unexpected results.

You can still access the UC Berkeley Library's services and resources. Here's how.

PH 270B: Toxicology I: Evaluate What You Find

Critically Evaluating What You Find

What is evidence? Things to keep in mind:

  • All research is (potentially) "evidence" and there are no "perfect" studies.
  • Is there an agenda (bias)?
    » It's doubtful that any study of humans is totally without some kind of bias, either in the study design, or in the author's pre-existing beliefs, not to mention the source of the research funds. How bias in methodology was controlled and the significance of bias in any particular study is what's relevant.
  • Is qualitative research "evidence"?
    » If your goal is to understand beliefs and meanings in the group with whom you are working, then qualitative studies can be important.

Who pays for science?
Most scientific research is funded by government, companies doing research and development, and non-profit entities. Because science is attempting to get at some "truth," the source of research funding shouldn't have a significant effect on the outcome of scientific research, right?
  » Read Industry sponsorship and research outcome Cochrane Database Syst Rev. 2012 Dec 12;12:MR000033).
  » Read the Sept. 12, 2016 article Sugar Industry and Coronary Heart Disease Research: A Historical Analysis of Internal Industry Documents from JAMA Internal Medicine

What to consider when looking at survey or estimated data:

  • Look at sample sizes and survey response rates - representative of your population? Enough responses to be valid?
  • Who was surveyed? - representative of population being compared to? Include group you are interested in?
  • Were the survey respondants from heterogeneous groups? Do the survey questions have a similar meaning to members of different groups?
  • How was survey conducted? Via telephone? - Many people only have cell phones. Random selection or targeted group?
  • What assumptions and methods were used for extrapolating the data?
  • Look at definitions of characteristics - Does this match your own definitions?
  • When was the data collected?

Reliability and validity
Adopted from Chapter 3, Conducting research literature reviews: from the Internet to paper, by Arlene Fink; Sage, 2014.
Reliable data collection is relatively free from "measurement error"
  » Is the survey written at a reading level too high for the people completing it?
  » Is the device used to measure elapsed time in an experiment accurate?
Validity refers to how well a measure assesses what it claims to measure
  » If the survey is supposed to measure "quality of life," how is that concept defined?
  » How accurately can this animal study of drug metabolism be extrapolated to humans?