Pipeline Auditor

Security and DevOps specialist focused on auditing CI/CD pipelines, deployment configurations, and infrastructure as code.

Quick Reference

Property	Value
Domain	DevOps
FORGE Stage	5 (VERIFY)
Version	1.0.0
Output Types	Audit reports (JSON, Markdown)

Overview

Use this agent when you need to:

Audit GitHub Actions workflows for security vulnerabilities
Review Railway deployment configurations
Identify performance inefficiencies in CI/CD pipelines
Ensure compliance with security best practices
Generate remediation checklists with prioritized actions
Validate supply chain security (pinned actions, dependencies)

The Pipeline Auditor performs comprehensive security and performance analysis of CI/CD infrastructure, delivering actionable findings with severity ratings and remediation steps.

Core Capabilities

Security Audit

Identify vulnerabilities in CI/CD configs (secrets, permissions, supply chain)

Performance Analysis

Find inefficiencies in build/deploy pipelines (caching, parallelization)

Best Practice Review

Check compliance with DevOps standards and industry recommendations

Remediation Planning

Provide prioritized, actionable fix recommendations with code examples

When to Use

Auditing security of GitHub Actions workflows

Reviewing Railway deployment configurations

Identifying exposed secrets or hardcoded credentials

Validating supply chain security (action pinning)

Finding performance optimization opportunities

Ensuring SOC 2 or SLSA compliance

Not suitable for:

Application code security audits (use SAST tools)
Infrastructure audits beyond CI/CD and deployments
Real-time monitoring (this is point-in-time auditing)

Usage Examples

Security Audit
Performance Analysis
Remediation Checklist

Comprehensive security audit of GitHub Actions workflows:

{
  "auditReport": {
    "generatedBy": "pipeline-auditor",
    "version": "1.0.0",
    "timestamp": "2024-01-15T10:30:00Z",
    "scope": {
      "repository": "so1-io/so1-control-plane-api",
      "auditType": "security",
      "filesAnalyzed": [
        ".github/workflows/ci.yml",
        ".github/workflows/deploy.yml"
      ]
    },
    "summary": {
      "critical": 1,
      "high": 2,
      "medium": 0,
      "low": 0,
      "totalFindings": 3
    },
    "findings": [
      {
        "id": "SEC-001",
        "severity": "critical",
        "category": "secrets",
        "title": "Hardcoded secret in workflow",
        "description": "API key appears hardcoded in deploy.yml line 45",
        "file": ".github/workflows/deploy.yml",
        "line": 45,
        "evidence": "API_KEY: \"sk-live-xxxx...\"",
        "impact": "Secret exposure in repository history",
        "remediation": "Remove hardcoded secret, use GitHub Secrets, rotate key",
        "cwe": "CWE-798"
      },
      {
        "id": "SEC-002",
        "severity": "high",
        "category": "permissions",
        "title": "Overly permissive workflow permissions",
        "description": "Workflow uses 'write-all' instead of least privilege",
        "file": ".github/workflows/ci.yml",
        "line": 8,
        "remediation": "Use granular permissions: contents: read, pull-requests: write",
        "cwe": "CWE-250"
      },
      {
        "id": "SEC-003",
        "severity": "high",
        "category": "supply-chain",
        "title": "Unpinned action versions",
        "description": "Actions use mutable tags instead of SHA pinning",
        "file": ".github/workflows/ci.yml",
        "line": 15,
        "evidence": "uses: actions/checkout@v4",
        "remediation": "Pin to SHA: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11",
        "cwe": "CWE-829"
      }
    ],
    "recommendations": [
      {
        "priority": "immediate",
        "action": "Rotate compromised API key (SEC-001)",
        "owner": "Security team",
        "deadline": "24 hours"
      }
    ]
  }
}

Result: Detailed security findings with CWE mappings, evidence, and immediate action items.

Identify CI/CD performance bottlenecks and optimization opportunities:

{
  "auditReport": {
    "scope": {
      "repository": "so1-io/so1-console",
      "auditType": "performance"
    },
    "summary": {
      "critical": 0,
      "high": 0,
      "medium": 3,
      "low": 2
    },
    "findings": [
      {
        "id": "PERF-001",
        "severity": "medium",
        "category": "performance",
        "title": "No dependency caching configured",
        "description": "pnpm dependencies installed without caching",
        "file": ".github/workflows/ci.yml",
        "line": 20,
        "impact": "~2 minutes added to each run, ~$50/month wasted",
        "remediation": "Add pnpm/action-setup with cache option",
        "estimatedSavings": "2 minutes per run, $50/month"
      },
      {
        "id": "PERF-002",
        "severity": "medium",
        "category": "performance",
        "title": "Sequential jobs that could run in parallel",
        "description": "Lint and test jobs run sequentially but are independent",
        "file": ".github/workflows/ci.yml",
        "line": 30,
        "impact": "~3 minutes wasted on sequential execution",
        "remediation": "Remove needs: lint from test job"
      },
      {
        "id": "PERF-003",
        "severity": "medium",
        "category": "performance",
        "title": "Turborepo cache not utilized",
        "description": "Monorepo not using Turbo remote caching",
        "impact": "Redundant build steps across services",
        "remediation": "Configure Turbo cache in workflow",
        "estimatedSavings": "1-2 minutes per run"
      }
    ]
  }
}

Result: Performance findings with estimated time and cost savings for each optimization.

Generate actionable remediation checklist in Markdown:

# Pipeline Audit Remediation Checklist

**Repository**: so1-io/so1-control-plane-api
**Audit Date**: 2024-01-15
**Auditor**: pipeline-auditor v1.0.0

## Critical Findings (Fix Immediately)

### SEC-001: Hardcoded Secret in Workflow

- [ ] Remove hardcoded secret from `.github/workflows/deploy.yml:45`
- [ ] Add secret to GitHub Secrets: `Settings > Secrets > Actions`
- [ ] Update workflow to use `${{ secrets.API_KEY }}`
- [ ] Rotate the compromised API key in provider dashboard
- [ ] Scan git history: `git log -S "sk-live-" --all`

## High Priority (Fix This Week)

### SEC-002: Overly Permissive Permissions

```yaml
# Before
permissions: write-all

# After
permissions:
  contents: read
  pull-requests: write
  checks: write

SEC-003: Unpinned Action Versions

# Before
- uses: actions/checkout@v4

# After (pin to SHA)
- uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1

Medium Priority (Fix This Sprint)

PERF-001: Add Dependency Caching

- uses: pnpm/action-setup@v3
  with:
    version: 9
- uses: actions/setup-node@v4
  with:
    node-version: 20
    cache: 'pnpm'  # Add this line

Verification

After applying fixes:

# Validate workflow syntax
gh workflow list

# Check for secrets
git secrets --scan

# Verify permissions
gh api repos/:owner/:repo/actions/permissions

Result: Step-by-step remediation guide with code examples and verification commands.

Outputs

Audit Report Structure

interface AuditReport {
  generatedBy: "pipeline-auditor";
  version: "1.0.0";
  timestamp: string;
  scope: {
    repository: string;
    auditType: "security" | "performance" | "full";
    filesAnalyzed: string[];
  };
  summary: {
    critical: number;
    high: number;
    medium: number;
    low: number;
    informational: number;
    totalFindings: number;
  };
  findings: Finding[];
  recommendations: Recommendation[];
  compliance?: ComplianceStatus;
}

Severity Definitions

Severity	Description	Response Time
Critical	Active exploit possible, data exposure	24 hours
High	Security weakness, requires action	1 week
Medium	Should be fixed, lower risk	2 weeks
Low	Best practice violation	1 month
Informational	Suggestion for improvement	As capacity allows

Compliance Status

{
  "compliance": {
    "soc2": {
      "status": "partial",
      "gaps": ["SEC-001", "SEC-002"],
      "notes": "Secrets management needs improvement for CC6.1"
    },
    "slsa": {
      "level": 1,
      "targetLevel": 2,
      "gaps": ["SEC-003"],
      "notes": "Need hermetic builds and pinned dependencies for Level 2"
    }
  }
}

FORGE Gate Compliance

Entry Gates (Pre-conditions)

Target repository identified

The repository or configuration to be audited must be specified with appropriate access permissions.

Audit scope defined

The audit type (security, performance, or full) and specific files/workflows to analyze must be determined.

Access to configurations granted

Read access to .github/workflows/, railway.json, Dockerfile, and related infrastructure files.

Exit Gates (Post-conditions)

Audit report generated

A complete audit report with findings, severity ratings, and evidence is produced in JSON format.

Vulnerabilities categorized

All security vulnerabilities are categorized by severity (critical, high, medium, low) with CWE mappings.

Remediation steps provided

Each finding includes specific, actionable remediation steps with code examples where applicable.

Compliance status summarized

If compliance frameworks (SOC 2, SLSA) are relevant, gaps and current status are documented.

Integration Points

Control Plane API

// Store audit results
POST /api/v1/audits
{
  "repository": "so1-io/so1-control-plane-api",
  "auditType": "security",
  "summary": {
    "critical": 1,
    "high": 2,
    "medium": 3
  },
  "findings": [...],
  "auditedAt": "2024-01-15T10:30:00Z"
}

// Retrieve audit history
GET /api/v1/audits?repository=so1-io/so1-control-plane-api

Veritas Prompts

Consumed Prompts
Produced Prompts

Prompt ID	Purpose
`vrt-i9j0k1l2`	Security audit checklist (comprehensive security review criteria)
`vrt-m3n4o5p6`	Performance optimization patterns (CI/CD efficiency techniques)

When novel security patterns or vulnerabilities are discovered:

{
  "id": "vrt-audit-finding-001",
  "category": "task",
  "status": "draft",
  "content": "New supply chain attack vector in GitHub Actions",
  "outputPath": "veritas/agent-prompts/devops/audit-finding-001.json"
}

Audit Checklists

GitHub Actions Security

Check	Severity	Description
Secrets in code	Critical	No hardcoded secrets in workflows
Permissions	High	Least-privilege permissions used
Action pinning	High	Actions pinned to SHA, not tags
Third-party actions	Medium	Trusted publishers only
Workflow triggers	Medium	No dangerous trigger combinations (e.g., `pull_request_target` with code execution)
Environment protection	Medium	Production requires approval
OIDC for cloud	Medium	Use OIDC instead of long-lived tokens
Artifact handling	Low	No sensitive data in artifacts

Railway Security

Check	Severity	Description
Variable exposure	High	No secrets in `railway.json`
Network policy	Medium	Internal services not publicly exposed
Resource limits	Medium	Memory/CPU limits set appropriately
Health checks	Medium	Health endpoints configured
Replica count	Low	HA (2+ replicas) for production services

Container Security

Check	Severity	Description
Base image	High	Official, minimal base image used
Root user	High	Non-root user configured
Secrets in image	Critical	No secrets baked into Dockerfile
Layer optimization	Low	Multi-stage builds used
Vulnerability scan	Medium	No critical CVEs in dependencies

Agent	Relationship	Use Case
GitHub Actions Engineer	Audits	Reviews generated workflows for security issues
Railway Deployer	Audits	Reviews deployment configs for vulnerabilities
Incident Commander	Escalates	Receives alerts for critical security findings
Runbook Writer	Documents	Creates remediation runbooks from audit findings

Source Files

View Agent Source

Repository: so1-io/so1-agents
Path: agents/devops/pipeline-auditor.md
Version: 1.0.0

Common Patterns

Finding Categories

Security (SEC-xxx):

Exposed secrets (hardcoded, in logs, in history)
Overly permissive permissions
Supply chain risks (unpinned actions, untrusted publishers)
Insecure workflow triggers

Performance (PERF-xxx):

Missing dependency caching
Sequential jobs that could be parallel
Inefficient resource allocation
Redundant build steps

Best Practice (BP-xxx):

Missing concurrency controls
No timeout configurations
Insufficient error handling
Missing documentation

Common Audit Failures

Error	Cause	Resolution
Access denied	Missing repo permissions	Request read access to repository
File not found	Workflow path incorrect	Verify `.github/workflows/` directory exists
Parse error	Invalid YAML syntax	Fix YAML syntax before running audit
Rate limit	Too many API calls	Implement rate limiting or use GitHub App

Success Metrics

Metric	Target	Measurement
Finding Accuracy	>95%	Valid findings / total findings
Coverage	100%	Files analyzed / files in scope
Remediation Rate	>80%	Fixed findings / total findings (30 days)
False Positive Rate	<5%	False positives / total findings

Next Steps:

Agents Overview

Orchestration

Automation

Engineering

DevOps

Documentation

Prompts

Incident

​Pipeline Auditor

​Quick Reference

​Overview

​Core Capabilities

Security Audit

Performance Analysis

Best Practice Review

Remediation Planning

​When to Use

​Usage Examples

​SEC-003: Unpinned Action Versions

​Medium Priority (Fix This Sprint)

​PERF-001: Add Dependency Caching

​Verification

​Outputs

​Audit Report Structure

​Severity Definitions

​Compliance Status

​FORGE Gate Compliance

​Entry Gates (Pre-conditions)

​Exit Gates (Post-conditions)

​Integration Points

​Control Plane API

​Veritas Prompts

​Audit Checklists

​GitHub Actions Security

​Railway Security

​Container Security

​Related Agents

​Source Files

View Agent Source

​Common Patterns

​Finding Categories

​Common Audit Failures

​Success Metrics

Pipeline Auditor

Quick Reference

Overview

Core Capabilities

When to Use

Usage Examples

SEC-003: Unpinned Action Versions

Medium Priority (Fix This Sprint)

PERF-001: Add Dependency Caching

Verification

Outputs

Audit Report Structure

Severity Definitions

Compliance Status

FORGE Gate Compliance

Entry Gates (Pre-conditions)

Exit Gates (Post-conditions)

Integration Points

Control Plane API

Veritas Prompts

Audit Checklists

GitHub Actions Security

Railway Security

Container Security

Related Agents

Source Files

Common Patterns

Finding Categories

Common Audit Failures

Success Metrics