13 Ways to Tell Good Research from Bad

People are inundated with news proclaiming the most recent research report, with findings that often fly in the face of accepted practices. Consumer media sources like to take full advantage of these reports, using sensational headlines to get page views and clicks. As a result, readers are left to their own devices for determining if a study is good or bad, and it's often difficult to differentiate between a quality study and a questionable one.

The Harvard Kennedy School Shorenstein Center on Media, Politics and Public Policy has a "Journalist's Resource" newsletter. The most recent newsletter carried an article that included a list of questions reporters should ask when selecting studies to guide

their coverage. As you're looking at research studies from universities or companies, this list is relevant to you, too. While there is no way to guarantee the quality of a study, the article pointed out, these questions can

help you avoid biased or otherwise flawed research.

The authors said it's important to note that many of these questions apply primarily to quantitative research that involves the analysis of data. Here is their list:


Is this research peer reviewed?

A study published in a peer-reviewed journal typically undergoes a detailed critique by a small number of qualified scholars. The peer-review process, while imperfect, is designed for quality control.


Is it published in a top-tier academic journal?

Top journals are more likely to feature high-quality research. They are more selective about the research they accept for publication. Also, their peer-review process tends to be more rigorous. A measure for gauging a journal's ranking is its Impact Factor, which can be found in the

Journal Citation Reports database. Impact Factor scores range from zero

to over 100.


Do other scholars trust this work?

One indicator of whether other scholars consider a study to be credible is the number of times they cite it in their own research. It can take years, however, for a study to generate a high citation count. You can use

Google Scholar, a free search engine, or

Web of Science, a subscription-based service,

to find

citation counts.

Journalists also can ask faculty

in the field their opinions.


Who funded the research?

It's important to know who sponsored the research and what role, if any, a sponsor played in the design of the study and its implementation or in decisions about how findings would be presented to the public. Authors of studies published in academic journals are required to disclose funding sources. Studies funded by organizations such as the

National Science Foundation

tend to be trustworthy because the funding process itself is subject to an exhaustive peer-review process.


What are the authors' credentials?

Knowing where the authors work and how often they have been published can help you assess their expertise in a field of study.


How old is the study?

In certain fields — for example, chemistry or public opinion — a study that is several years old may no longer be reliable.


Do the authors have a conflict of interest?

Be leery of research conducted by individuals or organizations that stand to gain from the findings.


What's the sample size?

For studies based on samples,

larger samples generally yield more accurate results than smaller samples.


Does the study rely on survey results?

Survey results can be biased if respondents were not chosen by random selection. Beware of any survey that relies on respondents who self-select (for example, many internet-based surveys).


Can you follow the methodology?

Scholars should explain how they approached their research questions, where they got their data and how they used it. They also should clearly define key concepts and describe the statistical methods used in their analyses. This level of detail is necessary to allow other people to check and replicate their work. Replicability is critical.


Is statistical data presented?

Authors should present details about the data they are examining and the numerical results of their analyses. This allows others to review their calculations. In some fields, authors make their data sets publicly available.


Are the study's findings supported by the data?

Good researchers are very cautious in describing their conclusions - because they want to convey exactly what they learned. Sometimes, however, researchers might exaggerate or minimize their findings or there will be a discrepancy between what an author claims to have found and what the data suggests.


Is it a meta-study?

Among the most reliable studies are meta-studies, also referred to as meta-analyses. Their conclusions are based on an analysis of multiple studies done on a particular topic.

Editor's Note: The following scholars helped create this Journalists' Resource tip sheet: Adam J. Berinsky, professor of political science at MIT; Marybeth Gasman, professor of higher education at the University of Pennsylvania; Morgan L. W. Hazelton, assistant professor of political science at Saint Louis University; Thomas E. Patterson, professor of government and the press at Harvard University; Eric A. Stewart, professor of criminology at Florida State University.


Latest News

How Important is U.S. Ag and Food to the Economy?

In celebration of National Ag Day and National Ag Week, the 2023 Feeding the Economy report shows just how vital the industry is to U.S. families, communities and the world.

Ferrie: Ready, Set, Whoops! A Fast Start To Fieldwork Could Cost You Big In Corn At V5

Caution can help you avoid creating compaction or density layers. Plus, if you're applying anhydrous now, allowing 14 days between the application and planting can prevent dead or damaged plants and costly yield dings.

Nebraska Farmland Values Jump 14% in 2023 — Up 30% in Two Years

This year marks the second-largest increase in the market value of agricultural land in Nebraska since 2014 and the highest non-inflation-adjusted statewide land value in the 45-year history of the survey. 

U.S. Milk Production and Cow Numbers Both Rise

The February 2023 USDA Milk Production report showed an 0.8% increase in year-over-year milk production with a total of 17.7 billion lbs. of milk. Also following suit, U.S. cow numbers also documented growth.

Crude Oil Prices Drop Below $70: What is the Outlook for Consumers at the Pump and Farmers Heading Into Spring Planting?

Oil prices are also off their highs of last year and gas and diesel prices are also sliding at the pump, but will that trend continue ahead of planting?   Energy experts are hoping the answer is yes.  

Can the $20-Billion Inflation Reduction Act Get Rolled Out Quickly Enough?

Industry experts say the new legislative package represents a 'generational opportunity' for conservation funding and needs to reach U.S. farmers and livestock producers sooner rather than later, starting this spring.