Skip to main content

Security Breach Response

P0 - CRITICAL SECURITY INCIDENTSTOP. Before proceeding, ensure:
  • Legal/Compliance has been notified
  • Evidence preservation is in place
  • Communication is on secure channels only
  • Need-to-know principle is enforced

Immediate Actions (First 15 Minutes)

Containment Checklist

# DO IMMEDIATELY - in order
# 1. Capture current state BEFORE making changes
./scripts/capture_forensic_snapshot.sh

# 2. Isolate affected systems (choose based on severity)
# Option A: Network isolation (preferred)
aws ec2 modify-instance-attribute --instance-id {{INSTANCE_ID}} \
  --groups {{ISOLATION_SECURITY_GROUP}}

# Option B: Stop instance (if data loss acceptable)
aws ec2 stop-instances --instance-ids {{INSTANCE_ID}}

# 3. Rotate compromised credentials
./scripts/emergency_credential_rotation.sh

# 4. Enable enhanced logging
./scripts/enable_forensic_logging.sh

Phase 1: Detection & Initial Assessment

AI Prompt: Breach Classification

You are a senior security incident responder performing initial breach assessment.

**CONFIDENTIAL - NEED TO KNOW BASIS**

**Alert Source:** {{ALERT_SOURCE}} (SIEM, IDS, user report, external notification)

**Initial Indicators:**
{{ALERT_DETAILS}}

**Affected System(s):** {{AFFECTED_SYSTEMS}}
**Time of Detection:** {{DETECTION_TIME}}
**Estimated Compromise Time:** {{ESTIMATED_START}} (if known)

**Classify this incident:**

1. **Breach Type:**
   - [ ] Unauthorized Access (credential compromise)
   - [ ] Data Exfiltration (data leaving network)
   - [ ] Malware/Ransomware
   - [ ] Insider Threat
   - [ ] Supply Chain Compromise
   - [ ] Web Application Attack (SQLi, XSS, etc.)
   - [ ] API Abuse

2. **Severity Assessment:**
   - Data sensitivity: [public|internal|confidential|restricted]
   - Regulatory impact: [none|GDPR|HIPAA|PCI-DSS|SOC2|multiple]
   - Customer data exposed: [yes|no|unknown]
   - System criticality: [low|medium|high|critical]

3. **Attack Vector Hypothesis:**
   Based on the indicators, what's the most likely attack path?

4. **Immediate Containment Priority:**
   What must be isolated/disabled RIGHT NOW?

**Output as structured assessment with confidence levels.**

Breach Type Decision Tree


Phase 2: Evidence Collection

AI Prompt: Forensic Collection Plan

You are a digital forensics specialist creating an evidence collection plan.

**Incident Type:** {{BREACH_TYPE}}
**Affected Systems:** {{AFFECTED_SYSTEMS}}
**Time Window:** {{COMPROMISE_WINDOW}}

**Create a prioritized evidence collection plan.**

**Requirements:**

- Preserve chain of custody
- Capture volatile data first (memory, connections, processes)
- Non-destructive collection methods only
- Document everything

**For each evidence source, specify:**

1. What to collect
2. Collection command/method
3. Priority (critical/high/medium)
4. Storage location
5. Hash verification command

**Evidence categories to consider:**

- Memory dumps
- Disk images
- Log files (auth, application, system, network)
- Network captures
- Cloud audit logs
- Container/K8s logs
- Database audit logs

Evidence Collection Commands

#!/bin/bash
# CAPTURE VOLATILE EVIDENCE FIRST
# Run this BEFORE any containment actions

EVIDENCE_DIR="/forensics/$(date +%Y%m%d_%H%M%S)_{{INCIDENT_ID}}"
mkdir -p $EVIDENCE_DIR

# 1. Current connections
netstat -tulpan > $EVIDENCE_DIR/netstat.txt
ss -tulpan > $EVIDENCE_DIR/ss.txt

# 2. Running processes
ps auxf > $EVIDENCE_DIR/processes.txt
ps -ef --forest > $EVIDENCE_DIR/process_tree.txt

# 3. Open files
lsof > $EVIDENCE_DIR/open_files.txt

# 4. Memory dump (if possible)
sudo dd if=/dev/mem of=$EVIDENCE_DIR/memory.dump bs=1M

# 5. Current users
w > $EVIDENCE_DIR/current_users.txt
last -100 > $EVIDENCE_DIR/last_logins.txt

# 6. Network routing
ip route > $EVIDENCE_DIR/routes.txt
iptables -L -n -v > $EVIDENCE_DIR/iptables.txt

# 7. Environment variables (may contain secrets)
env > $EVIDENCE_DIR/environment.txt

# 8. Loaded kernel modules
lsmod > $EVIDENCE_DIR/kernel_modules.txt

# Generate hashes
find $EVIDENCE_DIR -type f -exec sha256sum {} \; > $EVIDENCE_DIR/hashes.txt

echo "Volatile evidence captured: $EVIDENCE_DIR"

Phase 3: Investigation & Root Cause

AI Prompt: Attack Timeline Reconstruction

You are a threat analyst reconstructing the attack timeline.

**Evidence Collected:**

**CloudTrail Events:**
{{CLOUDTRAIL_SAMPLE}}

**Authentication Logs:**
{{AUTH_LOGS}}

**Application Logs:**
{{APP_LOGS}}

**Network Connections (at time of detection):**
{{NETSTAT_OUTPUT}}

**Your task:**

1. Construct a detailed timeline of attacker actions
2. Identify the initial access vector
3. Map lateral movement (if any)
4. Identify data accessed or exfiltrated
5. Determine persistence mechanisms (if established)
6. Assess current attacker access (are they still in?)

**Output as:**

timeline:
  - timestamp: "ISO8601"
    action: "description"
    evidence: "source"
    confidence: high|medium|low

initial_access:
  vector: "string"
  timestamp: "ISO8601"
  evidence: ["string"]

lateral_movement:
  - from: "system"
    to: "system"
    method: "technique"

data_accessed:
  - type: "data type"
    sensitivity: "level"
    volume: "estimate"

persistence:
  mechanisms: ["string"]
  locations: ["string"]

current_status:
  attacker_present: boolean
  confidence: high|medium|low
  reasoning: "string"

Investigation Queries

Authentication Analysis

# Failed logins followed by success (credential stuffing)
grep "Failed password" /var/log/auth.log | \
  awk '{print $11}' | sort | uniq -c | sort -rn | head -20

# Successful logins from unusual IPs
grep "Accepted" /var/log/auth.log | \
  awk '{print $11, $9}' | sort | uniq

# SSH key additions
grep -r "authorized_keys" /var/log/

# Sudo usage
grep "sudo" /var/log/auth.log | grep -v "session"

# User creation/modification
grep -E "(useradd|usermod|passwd)" /var/log/auth.log

Network Analysis

# Unusual outbound connections
netstat -tulpan | grep ESTABLISHED | \
  awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -rn

# DNS queries (if logging enabled)
grep -E "query\[" /var/log/syslog | \
  awk '{print $NF}' | sort | uniq -c | sort -rn | head -50

# Large data transfers
iftop -t -s 10 -L 50

# Suspicious ports
netstat -tulpan | grep -E "(4444|5555|6666|8080|9001)"

# Connection to known bad IPs (check against threat intel)
grep -f /path/to/bad_ips.txt /var/log/*/access.log

Process Analysis

# Processes with network connections
for pid in $(lsof -i -P -n | grep ESTABLISHED | awk '{print $2}' | sort -u); do
  echo "=== PID $pid ==="
  ps -p $pid -o pid,ppid,user,cmd
  ls -la /proc/$pid/exe 2>/dev/null
done

# Processes running from /tmp or unusual locations
ps aux | grep -E "(\/tmp\/|\/dev\/shm\/|\/var\/tmp\/)"

# Hidden processes
ps aux | awk '{print $2}' | while read pid; do
  if [ ! -d "/proc/$pid" ]; then
    echo "Hidden PID: $pid"
  fi
done

# Process ancestry (find parent chain)
pstree -p -s {{SUSPECT_PID}}

# Deleted but running binaries
ls -la /proc/*/exe 2>/dev/null | grep deleted

File Analysis

# Recently modified files
find / -type f -mtime -1 -ls 2>/dev/null | grep -v proc

# Files in unusual locations
find /tmp /var/tmp /dev/shm -type f -ls 2>/dev/null

# Suspicious file permissions (SUID/SGID)
find / -perm -4000 -o -perm -2000 -type f 2>/dev/null

# Webshells (common patterns)
grep -r -l -E "(eval\(|base64_decode|shell_exec|system\(|passthru)" /var/www/

# Recently added cron jobs
ls -la /etc/cron* /var/spool/cron/*

# SSH authorized_keys changes
find /home -name "authorized_keys" -mtime -7 -ls

Phase 4: Containment & Eradication

AI Prompt: Containment Strategy

Based on the investigation findings, generate a containment and eradication plan.

**Attack Summary:**
{{INVESTIGATION_SUMMARY}}

**Current Attacker Status:** {{ATTACKER_STATUS}}

**Affected Systems:** {{AFFECTED_SYSTEMS}}

**Generate:**

1. Immediate containment actions (stop the bleeding)
2. Eradication steps (remove attacker access/artifacts)
3. System hardening requirements
4. Credential rotation scope
5. Verification steps for each action

**Prioritize by:**

- Stopping active compromise first
- Preventing lateral movement
- Preserving evidence where possible

**Format as executable runbook with commands.**

Containment Actions

containment_runbook:
  phase_1_immediate:
    - action: "Isolate compromised systems"
      commands:
        - "aws ec2 modify-instance-attribute --instance-id {{ID}} --groups {{ISOLATION_SG}}"
      verification: "Confirm no external connectivity"
      rollback: "Restore original security groups"

    - action: "Disable compromised accounts"
      commands:
        - "aws iam update-login-profile --user-name {{USER}} --no-password-reset-required"
        - "aws iam delete-access-key --user-name {{USER}} --access-key-id {{KEY_ID}}"
      verification: "Confirm user cannot authenticate"

    - action: "Revoke active sessions"
      commands:
        - "aws iam delete-user-policy --user-name {{USER}} --policy-name {{POLICY}}"
        - "Invalidate all JWT tokens for user"

  phase_2_eradication:
    - action: "Remove persistence mechanisms"
      locations:
        - "/etc/cron.d/*"
        - "/home/*/.ssh/authorized_keys"
        - "/etc/systemd/system/*"
        - "AWS Lambda functions"
        - "IAM roles/policies"

    - action: "Remove malicious files"
      commands:
        - "rm -rf {{MALWARE_PATHS}}"
      verification: "File hashes no longer present"

    - action: "Rotate all credentials"
      scope:
        - "Database passwords"
        - "API keys"
        - "Service account credentials"
        - "SSH keys"
        - "Secrets in vault"

Phase 5: Recovery & Communication

AI Prompt: Breach Notification Draft

You are drafting breach notification communications.

**Incident Details:**

- Type: {{BREACH_TYPE}}
- Data Affected: {{DATA_TYPES}}
- Number of Records: {{RECORD_COUNT}}
- Regulatory Requirements: {{REGULATIONS}}
- Discovery Date: {{DISCOVERY_DATE}}

**Generate:**

1. **Regulatory Notification** (for {{REGULATION}})
   - Include all legally required elements
   - Timeline compliance requirements
2. **Customer Notification**
   - Clear, non-technical language
   - What happened, what data, what we're doing
   - What customers should do
3. **Press Statement** (if needed)
   - Brief, factual
   - Demonstrates responsibility
4. **Internal All-Hands**
   - More technical detail
   - What employees should/shouldn't say

**Note: All communications must be reviewed by Legal before sending.**

Recovery Checklist

recovery_checklist:
  systems:
    - action: "Rebuild compromised systems from clean images"
      verify: "System hashes match known-good baseline"

    - action: "Restore data from pre-compromise backup"
      verify: "Data integrity checks pass"

    - action: "Apply all security patches"
      verify: "Vulnerability scan shows no critical issues"

    - action: "Re-enable monitoring and logging"
      verify: "Alerts firing correctly"

  access:
    - action: "Issue new credentials to all affected users"
      verify: "Old credentials rejected"

    - action: "Implement additional authentication controls"
      options: ["MFA enforcement", "IP restrictions", "session limits"]

    - action: "Review and restrict IAM permissions"
      verify: "Principle of least privilege applied"

  monitoring:
    - action: "Deploy additional detection rules"
      for: ["IOCs from this incident", "TTPs observed"]

    - action: "Increase logging verbosity temporarily"
      duration: "30 days"

    - action: "Schedule follow-up threat hunt"
      timing: "7 days post-recovery"

Phase 6: Post-Incident

AI Prompt: Security Post-Mortem

Generate a comprehensive security incident post-mortem.

**Incident Summary:** {{INCIDENT_SUMMARY}}
**Timeline:** {{FULL_TIMELINE}}
**Root Cause:** {{ROOT_CAUSE}}
**Impact:** {{IMPACT_ASSESSMENT}}

**Structure:**

1. Executive Summary
2. Incident Timeline
3. Technical Analysis
4. Impact Assessment
5. Detection Analysis (how we found it, how we should have found it sooner)
6. Response Analysis (what went well, what didn't)
7. Recommendations (prioritized by impact)
8. Appendices (IOCs, evidence references)

**Include:**

- MITRE ATT&CK mapping
- Indicators of Compromise (IOCs)
- Detection rule recommendations
- Architecture improvements
- Process improvements
- Training needs identified

Lessons Learned Framework

post_incident_actions:
  detection_improvements:
    - gap: "{{DETECTION_GAP}}"
      solution: "{{DETECTION_SOLUTION}}"
      owner: "Security Team"
      due: "{{DUE_DATE}}"
      jira: "SEC-XXX"

  architecture_changes:
    - weakness: "{{ARCH_WEAKNESS}}"
      remediation: "{{ARCH_FIX}}"
      owner: "Platform Team"
      due: "{{DUE_DATE}}"
      jira: "INFRA-XXX"

  process_updates:
    - issue: "{{PROCESS_ISSUE}}"
      improvement: "{{PROCESS_FIX}}"
      owner: "Security Team"
      due: "{{DUE_DATE}}"

  training:
    - topic: "{{TRAINING_TOPIC}}"
      audience: "{{AUDIENCE}}"
      deadline: "{{DEADLINE}}"

n8n Master Workflow

{
  "name": "DR-Security-Breach-Response",
  "nodes": [
    {
      "id": "siem-webhook",
      "type": "n8n-nodes-base.webhook",
      "parameters": {
        "path": "security-breach",
        "method": "POST"
      }
    },
    {
      "id": "classify-severity",
      "type": "n8n-nodes-base.httpRequest",
      "parameters": {
        "url": "https://api.anthropic.com/v1/messages",
        "method": "POST",
        "headers": {
          "x-api-key": "={{$env.ANTHROPIC_API_KEY}}",
          "anthropic-version": "2023-06-01"
        },
        "body": {
          "model": "claude-sonnet-4-20250514",
          "max_tokens": 2000,
          "system": "You are a security incident classifier. Respond with JSON only.",
          "messages": [{ "role": "user", "content": "Classify: {{$json.alert_details}}" }]
        }
      }
    },
    {
      "id": "page-security-team",
      "type": "n8n-nodes-base.pagerDuty",
      "parameters": {
        "operation": "trigger",
        "serviceId": "={{$env.SECURITY_SERVICE_ID}}",
        "title": "Security Incident - {{$json.classification}}",
        "urgency": "high"
      }
    },
    {
      "id": "create-war-room",
      "type": "n8n-nodes-base.slack",
      "parameters": {
        "operation": "channel.create",
        "name": "inc-sec-{{$json.incident_id}}"
      }
    },
    {
      "id": "start-evidence-collection",
      "type": "n8n-nodes-base.ssh",
      "parameters": {
        "command": "/opt/forensics/capture_volatile.sh {{$json.affected_host}}",
        "host": "={{$env.FORENSICS_HOST}}"
      }
    },
    {
      "id": "notify-legal",
      "type": "n8n-nodes-base.email",
      "parameters": {
        "to": "legal@company.com",
        "subject": "CONFIDENTIAL: Security Incident {{$json.incident_id}}",
        "body": "A security incident has been declared..."
      }
    }
  ]
}

Quick Reference: IOC Templates

ioc_template:
  file_hashes:
    - md5: ""
      sha256: ""
      filename: ""
      first_seen: ""

  ip_addresses:
    - ip: ""
      type: "c2|exfil|scanner"
      first_seen: ""
      last_seen: ""

  domains:
    - domain: ""
      type: "c2|phishing|malware"
      first_seen: ""

  email_addresses:
    - email: ""
      type: "sender|recipient"

  user_agents:
    - ua: ""
      associated_with: ""

  registry_keys: # Windows
    - key: ""
      value: ""

  mutex_names:
    - name: ""

contacts:
  internal:
    - role: "Legal Counsel"
      contact: "{{LEGAL_EMAIL}}"
      escalation: "Immediate for any data breach"

    - role: "CISO"
      contact: "{{CISO_EMAIL}}"
      escalation: "All security incidents"

    - role: "Privacy Officer"
      contact: "{{DPO_EMAIL}}"
      escalation: "Any PII exposure"

  external:
    - role: "Outside Counsel"
      firm: "{{LAW_FIRM}}"
      contact: "{{EXTERNAL_LEGAL}}"

    - role: "Forensics Firm"
      company: "{{FORENSICS_COMPANY}}"
      contact: "{{FORENSICS_CONTACT}}"

    - role: "Cyber Insurance"
      company: "{{INSURANCE_COMPANY}}"
      policy: "{{POLICY_NUMBER}}"
      hotline: "{{INSURANCE_HOTLINE}}"

Evidence PreservationDO NOT delete, modify, or clean up any systems until Legal and Forensics have cleared you to do so. Evidence preservation is legally required in many jurisdictions.
Regulatory Timelines
  • GDPR: 72 hours to notify supervisory authority
  • CCPA: “Expeditiously” (typically less than 72 hours)
  • HIPAA: 60 days to notify HHS, individuals
  • PCI-DSS: Immediately notify card brands
  • SEC: 4 business days (material cybersecurity incidents)

Source Reference

This documentation is derived from security-breach.mdx.