ETL Pipeline Builder
Design and automate Extract, Transform, Load data pipelines for data integration and analytics
Your analytics dashboard shows stale data because someone forgot to run the export script on Monday. Data lives in 5 different systems, transformations happen in random Python scripts, and nobody knows which version of the pipeline is running.
Who it's for: data engineers building repeatable data pipelines, analysts tired of manual data exports, backend developers integrating multiple data sources, startups outgrowing spreadsheet-based data workflows, teams migrating from ad-hoc scripts to proper ETL
Example
"Build an ETL pipeline that pulls Stripe payments, HubSpot contacts, and Postgres orders into our data warehouse" → Pipeline with extraction configs for each source, transformation logic for deduplication and normalization, load scripts for your warehouse, and scheduling via cron or Airflow
CLAUDE.md Template
New here? 3-minute setup guide → | Already set up? Copy the template below.
npx ccplaybooks add etl-pipeline# ETL Pipeline
Comprehensive workflow for designing and automating Extract, Transform, Load data pipelines.
## Pipeline Architecture
### Core ETL Flow
```
DATA PIPELINE ARCHITECTURE:
┌─────────────────────────────────────────────────────────┐
│ EXTRACT │Get new playbooks like this one
One email a week with new Claude Code workflows. Free, like everything here.
No spam. Unsubscribe anytime.
What This Does
Comprehensive workflow for designing and automating Extract, Transform, Load data pipelines.
Quick Start
Step 1: Create a Project Folder
mkdir -p ~/Documents/EtlPipeline
Step 2: Download the Template
Click Download above, then:
mv ~/Downloads/CLAUDE.md ~/Documents/EtlPipeline/
Step 3: Start Working
cd ~/Documents/EtlPipeline
claude
Best Practices
- Incremental Loading: Use incremental extraction when possible
- Idempotency: Ensure pipelines can be re-run safely
- Partitioning: Partition large tables by date
- Monitoring: Track pipeline health metrics
- Documentation: Document all transformations
- Testing: Test with sample data before production
- Version Control: Track pipeline changes in git