Reporting Pipelines
by bobmatnyc
Reporting pipelines for CSV/JSON/Markdown exports with timestamped outputs, summaries, and post-processing.
Skill Details
Repository Files
2 files in this skill directory
name: reporting-pipelines description: Reporting pipelines for CSV/JSON/Markdown exports with timestamped outputs, summaries, and post-processing. version: 1.0.0 category: universal author: Claude MPM Team license: MIT progressive_disclosure: entry_point: summary: "Generate CSV/JSON/markdown reports with timestamped filenames and summary outputs." when_to_use: "Building reporting flows, exporting analytics results, or standardizing CSV/JSON/markdown outputs across projects." quick_start: "1. Run the CLI that produces base data 2. Export CSV/JSON/markdown with timestamps 3. Save to reports/" tags:
- reporting
- csv
- json
- markdown
- analytics
Reporting Pipelines
Overview
Your reporting pattern is consistent across repos: run a CLI or script that emits structured data, then export CSV/JSON/markdown reports with timestamped filenames into reports/ or tests/results/.
GitFlow Analytics Pattern
# Basic run
gitflow-analytics -c config.yaml --weeks 8 --output ./reports
# Explicit analyze + CSV
gitflow-analytics analyze -c config.yaml --weeks 12 --output ./reports --generate-csv
Outputs include CSV + markdown narrative reports with date suffixes.
EDGAR CSV Export Pattern
edgar/scripts/create_csv_reports.py reads a JSON results file and emits:
executive_compensation_<timestamp>.csvtop_25_executives_<timestamp>.csvcompany_summary_<timestamp>.csv
This script uses pandas for sorting and percentile calculations.
Standard Pipeline Steps
- Collect base data (CLI or JSON artifacts)
- Normalize into rows/records
- Export CSV/JSON/markdown with timestamp suffixes
- Summarize key metrics in stdout
- Store outputs in
reports/ortests/results/
Naming Conventions
- Use
YYYYMMDDorYYYYMMDD_HHMMSSsuffixes - Keep one output directory per repo (
reports/ortests/results/) - Prefer explicit prefixes (e.g.,
narrative_report_,comprehensive_export_)
Troubleshooting
- Missing output: ensure output directory exists and is writable.
- Large CSVs: filter or aggregate before export; keep summary CSVs for quick review.
Related Skills
universal/data/sec-edgar-pipelinetoolchains/universal/infrastructure/github-actions
Related Skills
Attack Tree Construction
Build comprehensive attack trees to visualize threat paths. Use when mapping attack scenarios, identifying defense gaps, or communicating security risks to stakeholders.
Grafana Dashboards
Create and manage production Grafana dashboards for real-time visualization of system and application metrics. Use when building monitoring dashboards, visualizing metrics, or creating operational observability interfaces.
Matplotlib
Foundational plotting library. Create line plots, scatter, bar, histograms, heatmaps, 3D, subplots, export PNG/PDF/SVG, for scientific visualization and publication figures.
Scientific Visualization
Create publication figures with matplotlib/seaborn/plotly. Multi-panel layouts, error bars, significance markers, colorblind-safe, export PDF/EPS/TIFF, for journal-ready scientific plots.
Seaborn
Statistical visualization. Scatter, box, violin, heatmaps, pair plots, regression, correlation matrices, KDE, faceted plots, for exploratory analysis and publication figures.
Shap
Model interpretability and explainability using SHAP (SHapley Additive exPlanations). Use this skill when explaining machine learning model predictions, computing feature importance, generating SHAP plots (waterfall, beeswarm, bar, scatter, force, heatmap), debugging models, analyzing model bias or fairness, comparing models, or implementing explainable AI. Works with tree-based models (XGBoost, LightGBM, Random Forest), deep learning (TensorFlow, PyTorch), linear models, and any black-box model
Pydeseq2
Differential gene expression analysis (Python DESeq2). Identify DE genes from bulk RNA-seq counts, Wald tests, FDR correction, volcano/MA plots, for RNA-seq analysis.
Query Writing
For writing and executing SQL queries - from simple single-table queries to complex multi-table JOINs and aggregations
Pydeseq2
Differential gene expression analysis (Python DESeq2). Identify DE genes from bulk RNA-seq counts, Wald tests, FDR correction, volcano/MA plots, for RNA-seq analysis.
Scientific Visualization
Meta-skill for publication-ready figures. Use when creating journal submission figures requiring multi-panel layouts, significance annotations, error bars, colorblind-safe palettes, and specific journal formatting (Nature, Science, Cell). Orchestrates matplotlib/seaborn/plotly with publication styles. For quick exploration use seaborn or plotly directly.