Contribution PR Review
Review an external contributor's PR the way an open-source maintainer should — check automated bot security findings first (across every channel they might post to), watch specifically for suspicious AGENTS.md or workflow-permission changes, scale size and test-coverage expectations by contributor experience, and separate intent/scope alignment from code quality that bots already covered.
A PR that quietly modifies a pull_request_target workflow trigger or edits your repo's agent-instruction file is a very different risk than a PR that just adds a feature — and a maintainer skimming for 'does this look reasonable' can miss both, which is exactly why external-contribution review needs its own checklist distinct from reviewing a teammate's diff.
Who it's for: open-source maintainers reviewing contributions from outside their core team, teams that use AI code-review bots (Codex, CodeRabbit, or similar) and want a maintainer checklist for what those bots can't assess, project owners who need consistent size and test-coverage expectations scaled to contributor experience, anyone worried about supply-chain-style risk from a seemingly benign external PR touching CI workflows or agent config
Example
"Review this external contributor's PR before merging" → Bot security findings checked across every comment channel first, a scan for suspicious workflow permission or agent-instruction-file changes unrelated to the PR's stated purpose, test coverage and PR size assessed against the contributor's actual history (200 lines is excellent for a first-timer, unremarkable for a regular contributor), and a draft comment — good-to-merge or changes-needed — presented for approval before ever posting
New here? 3-minute setup guide → | Already set up? Copy the template below.
# Contribution PR Review
Review an external contributor's PR for safety, quality, and readiness — security concerns, test coverage, size appropriateness, and intent alignment. Use when reviewing contributions from outside the core team, where the bar (and the failure modes) differ from reviewing your own team's diffs.
## 1. Check Automated Security Reviews First
If your repo has automated PR review bots (e.g. a code-review bot or an AI-assisted review connector), check whether either flagged security concerns before doing anything else. Their findings can be inline-only comments rather than a top-level review body, so check every channel a bot might post to — the review list, inline review comments, and general issue/PR comments — not just one.
**If a bot flagged security issues:** review the findings carefully, verify whether the concern is valid, and don't approve until it's addressed or confirmed a false positive.
**If no bot flagged anything but you notice concerning patterns yourself, watch specifically for:**
- Unusual changes to agent-instruction files (`AGENTS.md`, `CLAUDE.md`, or similar) unrelated to the PR's stated purpose.
- Workflow file modifications, especially anything resembling `pull_request_target` triggers, which run with elevated permissions against untrusted PR content.
- Changes to installed agent skill/plugin configuration that could affect agent behavior.
Comment on the security concern immediately when found — don't hold it for the final structured comment.
## 2. Enable CI Workflows (If Safe)
If the security assessment passes and the PR includes new or modified workflows, enable the required workflow runs so CI actually executes (many platforms require this for first-time or unfamiliar contributors as a safety gate). This step may no-op if workflows are already enabled — that's fine.
## 3. Test Coverage Assessment
Check for two things: whether the code the PR modifies already has pre-existing tests (making review easier), and whether the PR itself adds or modifies tests. Search the test directory for functions/classes matching the names defined in the modified files, and separately check whether any files under the test directory were touched by the PR.
Output a short summary:
```
Test Coverage:
- Pre-existing tests: [Modified code has tests / No tests for modified code]
- New tests: [PR adds N test files / No new tests]
- Assessment: [Easy/Medium/Hard to review based on test coverage]
```
## 4. PR Size and Contributor Experience
Calculate total lines changed (additions + deletions), then look up the contributor's history: their contribution count to this specific project, and their broader GitHub experience (repo count, account age) as a secondary signal.
**Assess by combining size and experience:**
- **First-time contributor to the project** (0–2 prior contributions): under 200 lines is an excellent size; 200–500 is large for a first PR and may need extra guidance; over 500 is too large and should be suggested to split — unless their broader GitHub history shows they're experienced overall, just new to this specific project, in which case adjust expectations accordingly.
- **Regular contributor** (3+ prior contributions): under 500 lines is reasonable; 500–1000 is large and should have solid test coverage; over 1000 is very large and worth suggesting a split.
```
PR Size:
- Lines changed: [total]
- Contributor: [first-time / regular] (N contributions)
- Assessment: [size appropriateness]
```
## 5. Intent and Issue Linkage
Check for a linked issue. **If linked:** read the issue to understand the expected outcome, then compare the PR's actual changes against it — are all requirements addressed, is there scope creep (extra unrequested features), and does the approach align with anything already discussed in the issue thread?
**If no issue is linked:** a bug fix should generally reference one; a feature should generally have one for prior discussion; a typo or docs fix is fine without one. Recommend creating an issue for tracking if the change is substantial and none exists.
```
Intent & Linkage:
- Linked issue: #N "title" / No issue linked
- Solves issue: [Fully addresses requirements / Partial / Doesn't match]
- Scope: [Focused / Scope creep detected]
```
## 6. Code Quality Overview
If your repo already has automated bot code review, this step should focus specifically on what those bots can't assess: architectural fit (does the change use the project's existing service-layer patterns, established conventions?), whether it's a genuine breaking change, and repo-specific conventions the bots wouldn't know about.
**Breaking-change assessment:** consolidating or refactoring tools, or changing a parameter where the same outcome is still achievable, is *not* breaking. Removing functionality with no replacement, or making a previously possible action impossible, *is* breaking.
Quick checks: scan CI check results for lint/type-check failures, and grep the diff for stray `TODO`/`FIXME`/`XXX`/`HACK` markers left in.
```
Code Quality:
- Architecture fit: [assessment]
- Breaking changes: [None / Detected - describe what's genuinely lost]
- Bot reviews: [anything critical flagged by automated review]
```
## Final Summary and Draft Comment
Present a short summary of what the PR does and the review findings to the user, then ask explicitly: "Should I post this comment to the PR?" Never post without that confirmation.
**Comment length:** 10–15 lines if it's good to merge; max 25 lines if changes are needed. No emojis; Markdown formatting (bold, lists, code blocks) is fine.
**"Good to merge" structure** (don't mention the security check in the comment unless an issue was actually found — security assessment is internal process, not something to publicize on a clean PR):
```
[Positive opening line about the contribution]
[1-2 sentences on what works well - functionality, tests, architecture]
[Any minor, technical-only suggestions - optional]
[Closing line about readiness to merge]
```
**"Changes needed" structure** (max 25 lines; a genuine security concern should already have been raised immediately in step 1, not held for here):
```
[Positive opening line acknowledging the work]
[Brief summary of the issue being solved]
**[Concern 1]:**
[1-2 lines explanation + suggestion]
**[Concern 2]:** (if applicable)
[1-2 lines explanation + suggestion]
[Closing line about next steps]
```
## Important Notes
- **Security is checked, not publicized.** Always check it first, but only mention it in the visible comment if a real issue was found.
- **Be constructive.** Contributors are donating their time — write like it.
- **Focus on intent over polish.** Code quality can be iterated on in review; a misaligned intent (solving the wrong problem, or scope creep) is much harder to fix after the fact.
- **Weight expectations by contributor experience.** A first-time contributor's 300-line PR gets different guidance than a core maintainer's.
- **Don't duplicate what the bots already did.** If automated review already covers detailed line-by-line code quality, spend your attention on what it can't assess — architecture, intent, and breaking changes.
Get new playbooks like this one
One email a week with new Claude Code workflows. Free, like everything here.
No spam. Unsubscribe anytime.
What This Does
A maintainer's review checklist purpose-built for external contributions, which carry different risks than reviewing a teammate's PR — starting with a security pass that checks every channel an automated review bot might have posted findings to (top-level review, inline comments, and general PR conversation are often separate API surfaces that a naive check would miss), plus a manual scan for the specific patterns bots tend not to catch: unrelated changes to agent-instruction files like AGENTS.md, workflow modifications involving elevated-permission triggers like pull_request_target, or changes to installed agent/skill configuration. It explicitly distinguishes non-breaking changes (tool consolidation, parameter changes where the same outcome is still achievable) from genuine breaking changes (removed functionality with no replacement) — a distinction reviewers frequently get wrong in either direction.
Size and test-coverage expectations scale explicitly by contributor history rather than applying one flat bar: a 300-line PR from a first-time contributor gets different guidance than the same size from a regular one, with broader GitHub experience factored in as a secondary signal when someone is new to the specific project but clearly experienced overall. It also enforces a clean separation between what gets checked internally and what gets said publicly — security is always checked first but only mentioned in the visible comment if something real was found — and produces two length- and structure-constrained comment templates (good-to-merge, changes-needed) that always get presented to the user for explicit approval before posting, never sent automatically.
Quick Start
Step 1: Create a Project Folder
mkdir contrib-review && cd contrib-review
Step 2: Download the Template
Click Download above, then:
mv ~/Downloads/CLAUDE.md ./
Step 3: Review a Contribution
claude
Point Claude at an external contributor's PR and ask for a review. It will check any automated bot findings across every comment channel, scan for suspicious workflow or agent-config changes, assess size and test coverage against the contributor's history, check intent against any linked issue, and draft a comment for your approval before posting anything.
Tips & Best Practices
- Never let "the bots already reviewed it" become a reason to skip the manual security scan for agent-config and workflow-permission changes — those are exactly the categories most automated code-review tools aren't specifically tuned to flag.
- Keep the security assessment internal on a clean PR — mentioning it explicitly ("I checked for security issues and found none") in a public comment reads as unnecessarily suspicious toward a contributor who did nothing wrong; only surface it when something was actually found.
- Adjust size expectations by contributor history before judging a PR "too large" — a 400-line PR is a reasonable ask from a regular contributor and a lot to absorb from someone's very first contribution to the project.
Limitations
- Assumes GitHub (via
gh) and an automated review bot are already part of the repo's workflow; a different Git host or no automated review setup needs the security-scanning step adapted or done manually. - The breaking-vs-non-breaking heuristics are general guidance, not exhaustive — a genuinely ambiguous API change still warrants a judgment call from someone who knows the project's actual compatibility commitments.
- Focused on contribution review specifically, not a full CI/CD security posture review — treat it as one layer of scrutiny for incoming external code, not a complete supply-chain security process.