Evaluation is about determining the quality, value, or importance of something in order to take action. It is underpinned by the systematic collection of information and evidence.
What is evidence? Things to keep in mind:
Who pays for science?
Most scientific research is funded by government, companies doing research and development, and non-profit entities. Because science is attempting to get at some "truth," the source of research funding shouldn't have a significant effect on the outcome of scientific research, right?
Is it race? or is it racism?
Race is a sociological construct, yet most articles describing racial disparities ascribe them to race, not to racism.
Peer review refers to a process whereby scholarly work (ie, an article) is reviewed and critiqued by experts to ensure it meets some standards of acceptance before it is published.
Does this process make for better science?
What gets researched? What gets published? ("Publication bias"); What (or Who) gets funded?
Studies that report interventions that had no effect are less likely to get published. What does this mean in terms of the state of knowledge on a topic?
There is also evidence of disparities in grant funding for research:
as well as disparities in recruitment for trials:
Sometimes stuff gets researched because of "scientific inertia":
Oops! I made a mistake (or ... was it cheating..?)
Occasionally, researchers make mistakes, and sometimes those mistakes affect the conclusions of a published article. Articles may be retracted if the mistake is significant. This is a formal process where the author or journal publishes a statement outlining the error. Sometimes, however, retraction is the result of fraud, plagiarism, or other bad acts.
Opinion or fact?
Do the conclusions of the article follow the evidence that's presented? Are opinions or notions posited as facts?
CV boosting: Does this study add to the body of knowledge, or is it just something the author is doing to add to his/her list of publications?
(In)significance of a single study: Science is incremental. Beware of any study that's proclaimed to be a "breakthrough."
Reliable data collection: relatively free from "measurement error."
Validity refers to how well a measure assesses what it claims to measure .
Adapted from Chapter 3, Conducting research literature reviews : from the Internet to paper, by Arlene Fink, 2010.