Why was my systematic review rejected? It's rarely the checklist

Everyone has a list of reasons systematic reviews get rejected: the search, PRISMA, the protocol, GRADE. Honestly, by the time a paper reaches a reviewer, that list has usually been checked already. What sends it back is judgement. Here's what that means, from someone who sits on both sides of the table.

By Tau Square Academy Research Team Published 1 November 2026 Updated 1 November 2026 10 min read

Key takeaways

Systematic reviews are rarely rejected or sent back at peer review for the checklist items (search strings, the PRISMA diagram, protocol registration, GRADE). Those are mostly caught at the editor's desk. What reviewers send papers back for is judgement.
The four judgement questions: did risk of bias actually change how you read the results, should those studies have been pooled at all, does the model fit the question, and do the conclusions match the evidence?
Learning the steps of a meta-analysis is easy; every course teaches PICO. Learning which question is worth six months of your life, and whether there's enough to pool, is the part that's rarely taught.
If you don't have a reputation yet, your proof is your record: log every decision and why you made it, so every step is defensible when a reviewer asks.
Plan long term, not paper by paper: a paper should lead to a conference, a group and the next project.
This guide draws on Dr. Jabez David John, Tau Square Academy's lead mentor, who has reviewed more than 70 manuscripts for 8 journals over the past two years, most of them systematic reviews and meta-analyses.

Why do systematic reviews really get rejected at peer review?

Systematic reviews get rejected or sent back at peer review for judgement, far more often than for reporting. By the time a paper reaches a reviewer, the checklist has usually been checked: the editor has already looked at the search, the PRISMA diagram and the protocol. What's left for the reviewer is whether the thinking holds up.

This guide comes from someone who sees the problem from both sides. Dr. Jabez David John is Tau Square Academy's lead mentor. Over the past two years he has reviewed more than 70 manuscripts for 8 journals, most of them systematic reviews and meta-analyses, and for the last year and a half he has taught at Tau Square, where he built the research curriculum. His own work was published in JACC: Heart Failure in 2026, and students he has mentored have presented at international meetings including DDW 2026 and ASBrS 2026.

So he answers the question twice: once as a peer reviewer, once as a mentor. The two answers are very different, and both matter if you're writing your first review.

Isn't it the search, PRISMA, the protocol and GRADE?

Not usually, at peer review. There's a list that comes up whenever people ask why systematic reviews get rejected, and every item on it is real. But most of them are caught before a paper ever reaches a reviewer. Here's the usual list, and where each item tends to get caught in Dr. Jabez's experience as a reviewer:

The usual suspect	Where it's usually caught	Why
Search strings that can't be reproduced	Rarely a reason to send a paper back	Unless a search is genuinely poor, it's usually good enough to stand
An incomplete PRISMA 2020 flow diagram	The editor's desk	Editors tend not to send those out for review at all
No registered protocol (PROSPERO, OSF, JBI)	The editor's desk	Many leading journals now expect registration, so it's checked at submission
No GRADE certainty assessment	Often not required	Reviewers tone down overstated claims either way
Risk of bias assessed but never used	The reviewer	This is judgement: the next section
Studies pooled that shouldn't have been	The reviewer	This is judgement too

So there's some survivorship bias in the usual list. The papers a reviewer sees have already survived the desk. That doesn't make the checklist optional: skip it and your paper may never reach a reviewer. It just means the checklist gets you through the door, and judgement decides what happens after.

Two panels. At the editor's desk, a stick figure ticks off a checklist: search string, PRISMA diagram, protocol registered, formatting. At peer review, a second stick figure with a magnifying glass reads the paper, with four questions around them: did risk of bias change the reading, should these be pooled, does the model fit the question, do the conclusions match the evidence. — The checklist is mostly checked at the desk. What's left for the reviewer is judgement.

What do reviewers actually push back on?

Four questions, and they're all about judgement. A review can tick every reporting box and still fail all four.

Did risk of bias actually change how you read the results?
Should those studies have been pooled at all?
Does the model fit the question?
Do the conclusions match the evidence?

Did risk of bias change how you read the results?

Plenty of reviews assess risk of bias carefully, fill in the traffic-light plot, and then never mention it again. A reviewer wants to see it used: does the effect hold when you leave out the studies at high risk, and does your discussion weigh the evidence by how trustworthy it is? If your risk of bias assessment changes nothing about how you read the results, a reviewer will ask why you did it.

Should those studies have been pooled at all?

A meta-analysis can always produce a pooled number. The question is whether that number means anything. The Cochrane Handbook separates clinical diversity (different patients, interventions or outcomes) and methodological diversity (different designs) from statistical heterogeneity, and warns against combining studies that are too different to answer one question. If you pooled anyway, a reviewer wants to see why it made sense, and what you did about the heterogeneity you found.

Does the model fit the question?

This is about matching the analysis to what you're actually asking: the effect measure, a fixed or random effects model, how you handled studies that report outcomes differently. A model chosen because it's the software's default, rather than because it fits the question, shows.

Do the conclusions match the evidence?

Conclusions that run ahead of the data are among the most common things a reviewer will tone down. Few studies, high risk of bias or wide confidence intervals call for careful conclusions, whether or not you used GRADE. The reverse is true too: if the evidence is strong, say so.

Why is the judgement so hard to learn?

Because courses teach the steps, and the judgement sits between them. There's no shortage of courses, and they all teach you to frame a question with PICO. But not every PICO question makes a good meta-analysis. Is there a real gap? Is there enough to pool? Which gap is actually worth six months of your life? Those questions are rarely taught.

Learning meta-analysis is easy. Learning the judgement to do a good one is what's missing. That's a skill-transfer problem, and good mentors in this space are rare.
Dr. Jabez David John, lead mentor, Tau Square Academy

That's the gap Tau Square Academy's method was built around: a topic is tested hard before anyone commits months to it, and the reason for every decision is written down as it's made. It's also why supervision matters so much for a first review: judgement is learned by watching someone use it on your own project.

Two panels. On the left, a stick figure climbs a neat staircase of taught steps: PICO, search, screen, extract, analyse. On the right, the same figure stands at a fork in the road with three signposts: is there a real gap, is there enough to pool, is it worth six months. A mentor stick figure points the way. — The steps are taught everywhere. The judgement at the fork is what a mentor passes on.

What if a reviewer asks for more than the guidelines require?

Answer from your record. When you don't have a reputation to lean on, what you need is proof, and the best proof is a log of every decision you made and why. With that, almost any question a reviewer asks already has an answer, written at the time, not reconstructed months later.

Dr. Jabez tells every beginner the same thing, and his students' problem usually turns out to be the opposite one: so much proof that it won't fit in the appendix. The extra can go on a preprint server such as medRxiv or a project page on OSF, with a link from the paper, so a reviewer can check anything they like.

A reviewer stick figure asks why a study was excluded. The author stick figure, standing beside a tall stack of logged decisions, hands over the exact page with the answer. The overflow of the log goes to a preprint server. — No reputation yet? Your log is your proof.

Ten things to do before your first systematic review

These are the ten things Dr. Jabez tells every student starting out:

See where this is going before you start.
Understand the question behind each type of study.
Test your topic hard before you commit to it.
Build your search on the giants' shoulders.
Register your protocol before you start screening.
Use tools, and keep the decisions for yourself.
If you're forcing studies together, narrow the question.
Log every decision and why you made it.
Earn your view, then state it clearly.
Talk to people who can help, and help someone else.

1. See where this is going before you start

If your goal is just one paper, it fails. It should be more like: this paper gets me to that conference, where I meet that group, which leads to the next project. Plan long term, not paper by paper, and be clear about what you're trying to achieve. It changes which topic you pick and how hard you work at it.

Four rising steps. On the first, a stick figure stands beside a paper; on the second, one presents at a conference podium; on the third, a group of three researchers; on the top step, a flag marks the next project. — One paper is a step. Plan the path it's on.

2. Understand the question behind each type of study

Every study design exists to answer a specific kind of question. Once you see that, research starts to make sense intuitively, and picking a topic gets much easier, because you can tell which questions a systematic review can answer and which need a different kind of study.

3. Test your topic hard before you commit to it

Is it actually new? Is there enough out there to pool? Most topics fail here, and that's okay. Usually a failed topic needs a different angle, not the bin. A few hours of testing now saves months on a review nobody needs. Tau Square Academy's guide to finding a topic that hasn't been done walks through the checks.

4. Build your search on the giants' shoulders

Look at previous meta-analyses in your field, or an adjacent one, and build on their searches. Trying to stand apart in everything isn't a strength; save the originality for your question. A search that has already worked for good reviewers is a far better starting point than one invented from scratch.

5. Register your protocol before you start screening

Registering on PROSPERO, OSF or with JBI keeps you honest, because your methods are fixed before you see any results, and it answers half of a reviewer's questions before they ask. The systematic review guide covers how registration works, country by country.

6. Use tools, and keep the decisions for yourself

A reference manager such as Zotero (free) saves hours every time. If your review is huge, AI screening platforms can take on the volume. Let the tools handle the volume while you make the decisions. The guide to meta-analysis software compares what's available.

7. If you're forcing studies together, narrow the question

If you find yourself bending the rules to get studies into one pooled analysis, your question is too broad. Let the data set your limits, and never bend the evidence to fit the result you want.

8. Log every decision and why you made it

Every inclusion and exclusion, every judgement call in data extraction, every change to the plan, with the reason. When a reviewer asks, you already have the answer. If you don't have a reputation yet, this record is your proof.

9. Earn your view, then state it clearly

Build a comprehensive, nuanced picture of the evidence first. But once you've earned your view, you don't always need to hedge it. As a reviewer, Dr. Jabez values a clear, well-supported position, because it stands apart from the hedging that fills so much writing today. Learn the craft of academic writing too, though: a strong view written badly still gets sent back.

10. Talk to people who can help, and help someone else

Research is easier with people who've done it before, so ask for help early. And most important, help someone else on their journey once you can.

Starting from zero?Tau Square Academy publishes this guide. Its free fundamentals series, Research From Zero, covers the basics this guide doesn't: 14 short lessons, about 2 hours, no card needed.

Frequently asked questions

What is the most common reason systematic reviews get rejected?

In peer review, it's usually judgement rather than reporting: risk of bias that was assessed but never used to interpret the results, studies pooled that shouldn't have been, a model that doesn't fit the question, or conclusions stronger than the evidence. Reporting gaps such as a missing search string or an incomplete PRISMA diagram are more often caught at the editor's desk before a paper reaches reviewers.

Do reviewers check the PRISMA flow diagram?

They read it, but in the experience of Tau Square Academy's lead mentor, who has reviewed more than 70 manuscripts, an incomplete PRISMA diagram is usually caught by the editor before the paper goes out for review. Complete it anyway: the PRISMA 2020 statement lists what the flow diagram should show, including the reasons full texts were excluded.

Does a systematic review need GRADE to be published?

Many journals don't require it. But GRADE or not, reviewers will tone down any claim that goes beyond the data, so it's worth judging the certainty of your evidence yourself. GRADE gives you a recognised way to do it, outcome by outcome, and makes your conclusions easier to defend.

What should I do if a peer reviewer asks for more than the reporting guidelines require?

Answer from your record. If you logged every decision and why you made it while doing the review, you already have the answer for almost anything a reviewer asks. Material that won't fit in the appendix can go on a preprint server or OSF, with a link in the paper.

How do I know if my systematic review topic is good enough?

Test it hard before you commit. Is it actually new: has someone already reviewed it recently? Is there enough out there to pool? Most topics fail here, and that's okay; usually the topic needs a different angle, not the bin. A topic that survives these tests is worth the months a review takes.

Should I register my systematic review protocol before screening?

Yes. Register it on PROSPERO, OSF or with JBI before you start screening. It keeps you honest, because your methods are fixed before you see the results, and it answers half of a reviewer's questions before they ask them.

Ready to start? Phase 0 is free: 14 short lessons, no card needed. If you’d rather talk first, a doctor from our team will map your next three steps on a free 15-minute call.

Start Phase 0 free or book a free call

Tau Square Academy Research Team

A team of doctors, research mentors and educational psychologists

Written and fact-checked by the Tau Square Academy research team: the team of doctors, research mentors and educational psychologists who designed the 100-Hour Research Protocol and teach every cohort. Every fact links to its source, and we update guides when things change.