Skip to Main Content

PH 155C: War and Public Health: Evaluate What You Find

Critically Evaluating What You Find

What is evidence?
All research is (potentially) "evidence" and there are no "perfect" studies. 
Critically evaluating what you read will help any unearth biases or methodological shortcomings that may be present.

Things to consider when evaluating research for bias:

  • The question being addressed: What kind of research gets funded?
  • Publication bias: Research that shows no effect tends not to get published
  • Conflict of interest, author affiliation, source(s) of funding: Does the researcher (or the funder) have a vested interest in the outcome?
  • Documentation and assumptions: Are all stated "facts" referenced?
  • Peer review: Is the article peer-reviewed? Does it matter?
  • Authority: Does the researcher have the knowledge to work in this area?
  • Significance of a single study: Science is an incremental process; one study rarely "changes everything"

  ...Consider that how issues are framed is influenced by our assumptions and biases, and also, keep swimming upstream!

Who pays for science? Does it matter?  (There is evidence that it does matter).
Research may be funded by:

  • Government;
  • Industry/trade groups;
  • Private foundations/associations;
  • etc.

What to consider when looking at survey or estimated data:

  • Look at sample sizes and survey response rates - representative of your population? Enough responses to be valid?
  • Who was surveyed? - representative of population being compared to? Include group you are interested in?
  • Were the survey respondants from heterogeneous groups? Do the survey questions have a similar meaning to members of different groups?
  • How was survey conducted? Via telephone? - Most people only have cell phones (PDF). Random selection or targeted group?
  • What assumptions and methods were used for extrapolating the data?
  • Look at definitions of characteristics - Does this match your own definitions?
  • When was the data collected?