Audit Aggregator

by jasonmichaelbell78-creator

skill

Aggregate and deduplicate findings from multiple audit reports

Skill Details

Repository Files

1 file in this skill directory


name: audit-aggregator description: Aggregate and deduplicate findings from multiple audit reports

Audit Aggregator Agent

Purpose: Merge findings from 6+ domain-specific audit reports into a single comprehensive, priority-ranked report with cross-domain insights and deduplication.

Input: Multiple audit report files (markdown format) Output: COMPREHENSIVE_AUDIT_REPORT.md with executive summary, priority rankings, and cross-cutting analysis


Input Requirements

Expected Files:

  • docs/audits/comprehensive/audit-code-report.md
  • docs/audits/comprehensive/audit-security-report.md
  • docs/audits/comprehensive/audit-performance-report.md
  • docs/audits/comprehensive/audit-documentation-report.md
  • docs/audits/comprehensive/audit-refactoring-report.md
  • docs/audits/comprehensive/audit-process-report.md

Required Format (in each report):

Each individual audit report should contain findings in this structure:

## Findings

### [ID] [Title] (Severity: S0-S3, Effort: E0-E3, Confidence: High/Medium/Low)

**File:** `path/to/file.ts:123` **Category:** [Category Name] **Description:**
[Detailed description] **Evidence:** [Code snippets, metrics, etc.]
**Recommendation:** [How to fix]

Processing Steps

Step 1: Parse All Reports

Read all 6 audit reports and extract findings into structured format:

{
  id: "CODE-001",
  title: "Missing error handling",
  severity: "S1",
  effort: "E1",
  confidence: "High",
  file: "src/auth.ts",
  line: 45,
  category: "Code Quality",
  domain: "code",
  description: "...",
  evidence: "...",
  recommendation: "..."
}

Parsing Rules:

  • ID Format: {DOMAIN}-{NUMBER} (e.g., CODE-001, SEC-012, PERF-003)
  • Severity: S0 (Critical) > S1 (High) > S2 (Medium) > S3 (Low)
  • Effort: E0 (Trivial) < E1 (Hours) < E2 (Day) < E3 (Major)
  • Confidence: High > Medium > Low (affects priority weighting)
  • File:Line: Extract from "File: `path/to/file.ts:123`" format

Handle Missing Fields:

  • If severity missing → assume S2 (Medium)
  • If effort missing → assume E1 (Hours)
  • If confidence missing → assume Medium
  • If file:line missing → categorize as "General" finding

Step 2: Deduplicate Findings

Deduplication Logic:

Group findings by (file, line) pair:

// Pseudo-code
const groups = groupBy(findings, (f) => `${f.file}:${f.line}`);

for (const group of groups) {
  if (group.length > 1) {
    // Multiple audits flagged same location
    merged = {
      id: `COMP-${nextId++}`, // New composite ID
      domains: group.map((f) => f.domain), // ["code", "security"]
      severity: maxSeverity(group), // Take worst severity
      effort: maxEffort(group), // Take highest effort estimate
      confidence: maxConfidence(group), // Take highest confidence
      title: mergeTitles(group), // Combine titles
      description: mergeDescriptions(group), // Combine contexts
      category: "Cross-Domain", // Flag as spanning multiple domains
      ...
    };
  }
}

Merge Rules:

  • Severity: Take worst (S0 > S1 > S2 > S3)
  • Effort: Take highest (E3 > E2 > E1 > E0)
  • Confidence: Take highest (High > Medium > Low)
  • Title: Combine unique titles (e.g., "Missing error handling + Exception vulnerability")
  • Description: Concatenate all descriptions with domain labels
  • Category: Change to "Cross-Domain" if 2+ domains

Example Deduplication:

Before:
  CODE-001: Missing error handling (S1, E1) at auth.ts:45
  SEC-012: Exception vulnerability (S0, E1) at auth.ts:45

After:
  COMP-001: Missing error handling + Exception vulnerability (S0, E1, Domains: 2)
    Category: Cross-Domain (Code + Security)

Step 3: Identify Cross-Cutting Patterns

Pattern Detection:

A. Hotspot Files (appear in 3+ audits):

// Count how many audits mention each file
const fileCounts = {};
for (const finding of findings) {
  fileCounts[finding.file] = (fileCounts[finding.file] || 0) + 1;
}

// Files with 3+ mentions
const hotspots = Object.entries(fileCounts)
  .filter(([file, count]) => count >= 3)
  .sort((a, b) => b[1] - a[1]);

B. Related Findings (fixing one helps another):

Look for pairs where:

  • Same file, different concerns (e.g., performance + security in same function)
  • Dependency chain (e.g., missing docs for complex code)
  • Contradiction patterns (e.g., over-engineered simple function)

C. Domain Overlaps:

Count overlaps between domain pairs:

Security + Performance: 12 findings
Code + Refactoring: 18 findings
Documentation + Code: 8 findings

D. Category Patterns:

Group by category across domains:

Authentication: 5 findings (2 security, 2 code, 1 process)
Error Handling: 8 findings (3 code, 3 security, 2 performance)
Testing: 12 findings (6 code, 4 refactoring, 2 process)

Step 4: Priority Ranking

Priority Score Formula:

priority = severityWeight × crossDomainMultiplier × confidenceWeight / effortWeight;

// Weights
severityWeight = { S0: 100, S1: 50, S2: 20, S3: 5 };
crossDomainMultiplier = 1 + (domains.length - 1) * 0.5; // 1.0, 1.5, 2.0, 2.5...
confidenceWeight = { High: 1.0, Medium: 0.8, Low: 0.5 };
effortWeight = { E0: 0.5, E1: 1.0, E2: 2.0, E3: 4.0 };

Ranking Rules:

  1. S0 Critical always rank first (regardless of other factors)
  2. Cross-domain findings bubble up (multiplier effect)
  3. High confidence preferred over low confidence
  4. Low effort (quick wins) preferred when severity equal

Example Scores:

Finding A: S0 Critical, 1 domain, High confidence, E1 effort
  → 100 × 1.0 × 1.0 / 1.0 = 100 (Rank #1)

Finding B: S1 High, 3 domains, High confidence, E1 effort
  → 50 × 2.0 × 1.0 / 1.0 = 100 (Rank #2 - tie, resolved by severity S0 > S1)

Finding C: S2 Medium, 2 domains, High confidence, E0 trivial
  → 20 × 1.5 × 1.0 / 0.5 = 60 (Rank #3 - quick win)

Finding D: S1 High, 1 domain, Medium confidence, E2 day
  → 50 × 1.0 × 0.8 / 2.0 = 20 (Rank #4 - high effort lowers priority)

Step 5: Generate Executive Summary

Summary Components:

A. Statistics:

## Executive Summary

### Audit Overview

- **Raw Findings:** 142 (across 6 audits)
- **Unique Findings:** 97 (after deduplication)
- **Merged Findings:** 45 (appeared in 2+ audits)
- **Cross-Domain Findings:** 18 (3+ audits)

### Severity Breakdown

- **S0 Critical:** 3 (IMMEDIATE ACTION REQUIRED)
- **S1 High:** 24 (fix within sprint)
- **S2 Medium:** 42 (plan for next milestone)
- **S3 Low:** 28 (backlog/nice-to-have)

### Effort Estimate

- **Total estimated effort:** 127 hours
- **Quick wins (E0):** 15 findings (12 hours)
- **Short fixes (E1):** 42 findings (42 hours)
- **Medium tasks (E2):** 28 findings (56 hours)
- **Major refactors (E3):** 12 findings (96 hours)

B. Top Insights:

List 3-5 highest-impact insights:

### Key Insights

1. **Authentication Layer Needs Comprehensive Refactor**
   - 8 files appear in 4+ audits (security, code, performance, documentation)
   - High complexity with poor error handling and missing tests
   - Recommended: Dedicated sprint to harden auth module

2. **Security + Performance Overlap**
   - 12 findings where fixing security also improves performance
   - Example: Rate limiting prevents DoS (security) + reduces server load
     (performance)
   - Recommended: Bundle these fixes together

3. **Documentation Gaps Align with Code Complexity**
   - 5 files with S1 complexity issues also have missing/outdated docs
   - Difficult to maintain or extend without documentation
   - Recommended: Document complex areas first (highest ROI)

C. Recommended Fix Order:

### Recommended Action Plan

**Phase 1: Critical Fixes (Week 1)**

1. FIX S0 findings (3 items - 8 hours)
2. Address top 5 quick wins (E0 - 4 hours)

**Phase 2: High-Priority Fixes (Week 2-3)**

1. Cross-domain findings (18 items - 32 hours)
2. Hotspot files refactor (8 files - 48 hours)

**Phase 3: Systematic Improvements (Month 2)**

1. Remaining S1 findings (21 items - 36 hours)
2. S2 medium-effort fixes (28 items - 56 hours)

**Phase 4: Backlog (Ongoing)**

1. S3 low-priority (28 items - 24 hours)
2. Major refactors (E3 - 96 hours, spread over quarters)

Step 6: Generate Full Report

Output File: docs/audits/comprehensive/COMPREHENSIVE_AUDIT_REPORT.md

Structure:

# Comprehensive Audit Report

**Generated:** [Date] **Audits Included:** Code, Security, Performance,
Documentation, Refactoring, Process **Total Findings:** 97 unique (142 raw)

---

## Executive Summary

[Generated in Step 5]

---

## Priority-Ranked Findings (Top 20)

| Rank | ID       | Severity | Domains | File:Line           | Description                             | Effort | Score |
| ---- | -------- | -------: | ------: | ------------------- | --------------------------------------- | -----: | ----: |
| 1    | COMP-001 |       S0 |       3 | src/auth.ts:45      | Missing error handling + Exception vuln |     E1 |   100 |
| 2    | SEC-012  |       S0 |       1 | firestore.rules:102 | Missing auth check on delete            |     E0 |    95 |
| ...  | ...      |      ... |     ... | ...                 | ...                                     |    ... |   ... |

---

## Cross-Domain Insights

### Hotspot Files (3+ audits)

1. **src/auth.ts** (5 audits: code, security, performance, documentation,
   refactoring)
   - Complexity: 45 (high)
   - Missing tests: 8 functions
   - Security issues: 3 (auth bypass, exception handling, rate limiting)
   - Performance: N+1 queries
   - Documentation: Outdated

[Continue with other hotspots...]

### Domain Overlaps

- **Security + Performance:** 12 findings (bundling opportunity)
- **Code + Refactoring:** 18 findings (quality improvements)
- **Documentation + Code:** 8 findings (complex code needs docs)

### Category Patterns

- **Authentication:** 5 findings → Need dedicated hardening sprint
- **Error Handling:** 8 findings → Systematic improvement needed
- **Testing:** 12 findings → Coverage gaps in critical areas

---

## Full Findings (Deduplicated)

### S0 Critical (3 findings)

#### COMP-001: Missing error handling + Exception vulnerability

**Domains:** Code (CODE-001), Security (SEC-012), Performance (PERF-005)
**File:** `src/auth.ts:45` **Severity:** S0 (Critical) **Effort:** E1 (Hours)
**Confidence:** High

**Description:** [Merged description from all 3 audits...]

**Evidence:** [Merged evidence from all 3 audits...]

**Recommendation:** [Merged recommendations from all 3 audits...]

---

[Continue with all findings grouped by severity...]

---

## Appendix

### Individual Audit Reports

- [Code Quality Report](./audit-code-report.md) - 32 findings
- [Security Report](./audit-security-report.md) - 18 findings
- [Performance Report](./audit-performance-report.md) - 24 findings
- [Documentation Report](./audit-documentation-report.md) - 15 findings
- [Refactoring Report](./audit-refactoring-report.md) - 41 findings
- [Process/Automation Report](./audit-process-report.md) - 12 findings

### Baseline Metrics

- **Tests:** 287 passing
- **Lint:** 0 errors, 12 warnings
- **Pattern Compliance:** 4 violations
- **Stack:** Next.js 16.1.1, React 19.2.3, Firebase 12.6.0

### False Positives Excluded

- **Total excluded:** 8 findings
- **Reason:** Documented false positives in FALSE_POSITIVES.jsonl
- **Categories:** code (3), security (2), performance (2), documentation (1)

Output Validation

Before Finalizing Report:

  1. Verify deduplication worked:

    • Count unique findings vs raw findings
    • Ensure merged findings have correct domain lists
    • No duplicate file:line pairs
  2. Verify priority ranking:

    • Top 20 should have highest scores
    • S0 Critical should all be in top 20
    • Cross-domain findings should rank higher than single-domain
  3. Verify cross-cutting insights:

    • Hotspot files should have 3+ mentions
    • Domain overlaps should be accurate counts
    • Category patterns should group related findings
  4. Verify completeness:

    • All raw findings accounted for (unique + duplicates = raw total)
    • All severity levels represented
    • All 6 audit reports linked in appendix

Error Handling

If Audit Report Missing:

  • Note in report: "X audit not available"
  • Continue with available audits
  • Adjust statistics (e.g., "5 audits analyzed")

If Parsing Fails:

  • Log which report/finding failed to parse
  • Skip malformed finding
  • Note in report: "X findings skipped due to format issues"

If Deduplication Produces Empty Result:

  • Verify input reports aren't empty
  • Check parsing logic
  • Fall back to showing all findings without deduplication

Usage

Standalone Usage (you have existing audit reports):

/audit-aggregator

Integrated Usage (called by audit-comprehensive):

Automatically invoked after all 6 parallel audits complete.


Notes

  • Deduplication Benefits: Prevents same issue from appearing 6 times
  • Cross-Domain Value: Finds patterns individual audits miss
  • Priority Ranking: Helps focus on high-impact fixes first
  • Effort Estimation: Helps plan sprint capacity

Future Enhancements

  • Trend Analysis: Compare against previous comprehensive audits
  • Auto-Issue Creation: Create GitHub issues for top 20 findings
  • Confidence Calibration: Track how often Low confidence findings are false positives
  • Fix Chains: Automatically detect dependencies between findings
  • ROI Scoring: Prioritize fixes by impact/effort ratio

Related Skills

  • /audit-comprehensive - Orchestrator that calls this aggregator
  • Individual audit skills - Generate the input reports this aggregator processes

Related Skills

Attack Tree Construction

Build comprehensive attack trees to visualize threat paths. Use when mapping attack scenarios, identifying defense gaps, or communicating security risks to stakeholders.

skill

Grafana Dashboards

Create and manage production Grafana dashboards for real-time visualization of system and application metrics. Use when building monitoring dashboards, visualizing metrics, or creating operational observability interfaces.

skill

Matplotlib

Foundational plotting library. Create line plots, scatter, bar, histograms, heatmaps, 3D, subplots, export PNG/PDF/SVG, for scientific visualization and publication figures.

skill

Scientific Visualization

Create publication figures with matplotlib/seaborn/plotly. Multi-panel layouts, error bars, significance markers, colorblind-safe, export PDF/EPS/TIFF, for journal-ready scientific plots.

skill

Seaborn

Statistical visualization. Scatter, box, violin, heatmaps, pair plots, regression, correlation matrices, KDE, faceted plots, for exploratory analysis and publication figures.

skill

Shap

Model interpretability and explainability using SHAP (SHapley Additive exPlanations). Use this skill when explaining machine learning model predictions, computing feature importance, generating SHAP plots (waterfall, beeswarm, bar, scatter, force, heatmap), debugging models, analyzing model bias or fairness, comparing models, or implementing explainable AI. Works with tree-based models (XGBoost, LightGBM, Random Forest), deep learning (TensorFlow, PyTorch), linear models, and any black-box model

skill

Pydeseq2

Differential gene expression analysis (Python DESeq2). Identify DE genes from bulk RNA-seq counts, Wald tests, FDR correction, volcano/MA plots, for RNA-seq analysis.

skill

Query Writing

For writing and executing SQL queries - from simple single-table queries to complex multi-table JOINs and aggregations

skill

Pydeseq2

Differential gene expression analysis (Python DESeq2). Identify DE genes from bulk RNA-seq counts, Wald tests, FDR correction, volcano/MA plots, for RNA-seq analysis.

skill

Scientific Visualization

Meta-skill for publication-ready figures. Use when creating journal submission figures requiring multi-panel layouts, significance annotations, error bars, colorblind-safe palettes, and specific journal formatting (Nature, Science, Cell). Orchestrates matplotlib/seaborn/plotly with publication styles. For quick exploration use seaborn or plotly directly.

skill

Skill Information

Category:Skill
Last Updated:1/24/2026