Data Analysis Validator
QA-review data analysis for methodology, accuracy, and presentation before sharing
You're about to present your analysis to the VP and you're not 100% sure the numbers are right. Did that join double-count records? Is the average of averages misleading? Is your chart accidentally deceptive? You need a second pair of eyes.
Who it's for: data analysts presenting to stakeholders, BI teams publishing dashboards, researchers validating statistical methods, managers reviewing team analyses before sharing up, anyone who's been burned by a wrong number in a meeting
Example
"Validate this revenue analysis before I present it" → Methodology review, calculation spot-checks, bias detection (survivorship, selection), visualization audit for misleading elements, and a confidence score with specific fixes
CLAUDE.md Template
New here? 3-minute setup guide → | Already set up? Copy the template below.
npx ccplaybooks add data-analysis-validator# Data Analysis Validator
## Your Role
You are a data analysis QA reviewer. You review analyses for accuracy, methodology, and potential biases before they are shared with stakeholders. You generate a confidence assessment and provide specific, actionable improvement suggestions.
## Workflow
### 1. Review Methodology and Assumptions
Examine:
- **Question framing**: Is the analysis answering the right question? Could the question be interpreted differently?
- **Data selection**: Are the right tables/datasets being used? Is the time range appropriate?Get new playbooks like this one
One email a week with new Claude Code workflows. Free, like everything here.
No spam. Unsubscribe anytime.
What This Does
Reviews your data analysis for accuracy, methodology, and potential biases before you share it with stakeholders. The assistant runs through a comprehensive QA checklist, checks for common analytical pitfalls (join explosions, survivorship bias, average of averages), spot-checks calculations, reviews visualizations for misleading elements, and produces a confidence assessment with specific improvement suggestions.
Quick Start
Step 1: Download the Template
Click Download above to get the CLAUDE.md file.
Step 2: Set Up Your Project
Create a project folder and place the template inside:
mkdir -p ~/Projects/data-validation
mv ~/Downloads/CLAUDE.md ~/Projects/data-validation/
Add your analysis files -- reports, notebooks, SQL queries, charts, or spreadsheets -- to the folder.
Step 3: Start Working
cd ~/Projects/data-validation
claude
Say: "Review this analysis before I send it to the exec team"
Common Pitfalls Detected
The assistant systematically checks for these analytical traps:
- Join Explosion -- Many-to-many joins silently inflating counts and sums
- Survivorship Bias -- Analyzing only entities that exist today, ignoring churned or deleted ones
- Incomplete Period Comparison -- Comparing a partial month to a full month
- Denominator Shifting -- The denominator definition changing between compared periods
- Average of Averages -- Averaging pre-computed averages when group sizes differ (produces wrong results)
- Timezone Mismatches -- Different data sources using different timezones
- Selection Bias -- Segments defined by the outcome being measured
- Simpson's Paradox -- Trends that reverse when data is aggregated vs. segmented
Confidence Assessment
Every validation produces a 3-level confidence rating:
- Ready to share -- Methodologically sound, calculations verified, caveats noted
- Share with noted caveats -- Largely correct but has specific limitations that must be communicated to stakeholders
- Needs revision -- Specific errors or missing analyses that should be addressed before sharing
Tips
- Run this review before any high-stakes presentation or decision. A few minutes of validation can save your credibility.
- Even quick analyses benefit from a sanity check -- common mistakes like join explosions or incomplete periods are easy to miss.
- If the validation finds issues, fix and re-validate before sharing.
- Consider sharing the validation output alongside your analysis to build stakeholder confidence in your methodology.
Example Prompts
"Review this quarterly revenue analysis before I send it to the exec team"
"Check my churn analysis -- I'm comparing Q4 rates to Q3 but Q4 has a shorter window"
"Here's a SQL query and its results for our conversion funnel. Does the logic look right?"
"Validate these charts for my board deck -- are there any misleading visualizations?"
"Spot-check the calculations in this cohort retention analysis"