troubleshoot/docs/Person-2-PRD.md

Person 2 PRD: Collectors, Redaction, Analysis, Diff, Remediation

CRITICAL CODEBASE ANALYSIS UPDATE

This PRD has been updated based on comprehensive analysis of the current troubleshoot codebase. Key findings:

Current State Analysis

• API Schema: Current API group is troubleshoot.replicated.com (not troubleshoot.sh), with v1beta1 and v1beta2 available
• Binary Structure: Multiple binaries already exist (preflight, support-bundle, collect, analyze)
• CLI Structure: support-bundle root command exists with analyze and redact subcommands
• Collection System: Comprehensive collection framework in pkg/collect/ with 15+ collector types
• Redaction System: Functional redaction system in pkg/redact/ with multiple redactor types
• Analysis System: Mature analysis system in pkg/analyze/ with 60+ built-in analyzers
• Support Bundle: Complete support bundle system in pkg/supportbundle/ with archiving and processing

Implementation Strategy

This PRD now focuses on EXTENDING existing systems rather than building from scratch:

• Auto-collectors: NEW package pkg/collect/autodiscovery/ extending existing collection
• Redaction tokenization: ENHANCE existing pkg/redact/ system
• Agent-based analysis: WRAP existing pkg/analyze/ system with agent abstraction
• Bundle differencing: COMPLETELY NEW pkg/supportbundle/diff/ capability

Overview

Person 2 is responsible for the core data collection, processing, and analysis capabilities of the troubleshoot project. This involves implementing auto-collectors, advanced redaction with tokenization, agent-based analysis, support bundle differencing, and remediation suggestions.

Scope & Responsibilities

• Auto-collectors (namespace-scoped, RBAC-aware), include image digests & tags
• Redaction with tokenization (optional local LLM-assisted pass), emit redaction-map.json
• Analyzer via agents (local/hosted) and "generate analyzers from requirements"
• Support bundle diffs and remediation suggestions

Primary Code Areas

• pkg/collect - Collection engine and auto-collectors (extending existing collection system)
• pkg/redact - Redaction engine with tokenization (enhancing existing redaction system)
• pkg/analyze - Analysis engine and agent integration (extending existing analysis system)
• pkg/supportbundle - Bundle readers/writers and artifact management (extending existing support bundle system)
• examples/* - Reference implementations and test cases

Critical API Contract: All implementations must use ONLY the current API group troubleshoot.replicated.com/v1beta2 types and be prepared for future migration to Person 1's planned schema updates. No schema modifications allowed.

Deliverables

Core Deliverables (Based on Current CLI Structure)

1. support-bundle --namespace ns --auto - enhance existing root command with auto-discovery capabilities
2. Redaction/tokenization profiles - streaming integration in collection path, emit redaction-map.json
3. support-bundle analyze --agent claude|local --bundle bundle.tgz - enhance existing analyze subcommand with agent support
4. support-bundle diff old.tgz new.tgz - NEW subcommand with structured diff.json output
5. "Generate analyzers from requirements" - create analyzers from requirement specifications
6. Remediation blocks - surfaced in analysis outputs with actionable suggestions

Note: The current CLI structure has support-bundle as the root collection command, with analyze and redact as subcommands. The diff subcommand will be newly added.

Critical Implementation Constraints

• NO schema alterations: Person 2 consumes but never modifies schemas/types from Person 1
• Streaming redaction: Must run as streaming step during collection (per IO flow contract)
• Exact CLI compliance: Implement commands exactly as specified in CLI contracts
• Artifact format compliance: Follow exact naming conventions for all output files

---

Component 1: Auto-Collectors

Objective

Implement intelligent, namespace-scoped auto-collectors that enhance the current YAML-driven collection system with automatic foundational data discovery. This creates a dual-path collection strategy that ensures comprehensive troubleshooting data is always gathered.

Dual-Path Collection Strategy

Current System (YAML-only):

• Collects only what vendors specify in YAML collector specs
• Limited to predefined collector configurations
• May miss critical cluster state information

New Auto-Collectors System:

• Path 1 - No YAML: Automatically discover and collect foundational cluster data (logs, deployments, services, configmaps, secrets, events, etc.)
• Path 2 - With YAML: Collect vendor-specified YAML collectors PLUS automatically collect foundational data as well
• Always ensures comprehensive baseline data collection for effective troubleshooting

Requirements

• Foundational collection: Always collect essential cluster resources (pods, deployments, services, configmaps, events, logs)
• Namespace-scoped collection: Respect namespace boundaries and permissions
• RBAC-aware: Only collect data the user has permission to access
• Image metadata: Include digests, tags, and repository information for discovered containers
• Deterministic expansion: Same cluster state should produce consistent foundational collection
• YAML augmentation: When YAML specs provided, add foundational collection to vendor-specified collectors
• Streaming integration: Work with redaction pipeline during collection

Technical Specifications

1.1 Auto-Discovery Engine

Location: pkg/collect/autodiscovery/

Components:

• discoverer.go - Main discovery orchestrator
• rbac_checker.go - Permission validation
• namespace_scanner.go - Namespace-aware resource enumeration
• resource_expander.go - Convert discovered resources to collector specs

API Contract:

type AutoCollector interface {
    // Discover foundational collectors based on cluster state
    DiscoverFoundational(ctx context.Context, opts DiscoveryOptions) ([]CollectorSpec, error)
    // Augment existing YAML collectors with foundational collectors
    AugmentWithFoundational(ctx context.Context, yamlCollectors []CollectorSpec, opts DiscoveryOptions) ([]CollectorSpec, error)
    // Validate permissions for discovered resources
    ValidatePermissions(ctx context.Context, resources []Resource) ([]Resource, error)
}

type DiscoveryOptions struct {
    Namespaces         []string
    IncludeImages      bool
    RBACCheck          bool
    MaxDepth           int
    FoundationalOnly   bool   // Path 1: Only collect foundational data
    AugmentMode        bool   // Path 2: Add foundational to existing YAML specs
}

type FoundationalCollectors struct {
    // Core Kubernetes resources always collected
    Pods           []PodCollector
    Deployments    []DeploymentCollector
    Services       []ServiceCollector
    ConfigMaps     []ConfigMapCollector
    Secrets        []SecretCollector
    Events         []EventCollector
    Logs           []LogCollector
    // Container image metadata
    ImageFacts     []ImageFactsCollector
}

1.2 Image Metadata Collection

Location: pkg/collect/images/

Components:

• registry_client.go - Registry API integration
• digest_resolver.go - Convert tags to digests
• manifest_parser.go - Parse image manifests
• facts_builder.go - Build structured image facts

Data Structure:

type ImageFacts struct {
    Repository string            `json:"repository"`
    Tag        string            `json:"tag"`
    Digest     string            `json:"digest"`
    Registry   string            `json:"registry"`
    Size       int64             `json:"size"`
    Created    time.Time         `json:"created"`
    Labels     map[string]string `json:"labels"`
    Platform   Platform          `json:"platform"`
}

type Platform struct {
    Architecture string `json:"architecture"`
    OS           string `json:"os"`
    Variant      string `json:"variant,omitempty"`
}

Implementation Checklist

Phase 1: Core Auto-Discovery (Week 1-2)

• Discovery Engine Setup

  • Create pkg/collect/autodiscovery/ package structure
  • Implement Discoverer interface and base implementation
  • Add Kubernetes client integration for resource enumeration
  • Create namespace filtering logic
  • Add discovery configuration parsing

• RBAC Integration

  • Implement RBACChecker for permission validation
  • Add SelfSubjectAccessReview integration
  • Create permission caching layer for performance (5min TTL)
  • Add fallback strategies for limited permissions

• Resource Expansion

  • Implement resource-to-collector mapping via ResourceExpander
  • Add standard resource patterns (pods, deployments, services, configmaps, secrets, events)
  • Create expansion rules configuration with priority system
  • Add dependency graph resolution and deduplication

• Unit Testing ALL TESTS PASSING

  • Test Discoverer.DiscoverFoundational() with mock Kubernetes clients
  • Test RBACChecker.FilterByPermissions() with various permission scenarios
  • Test namespace enumeration and filtering with different configurations
  • Test ResourceExpander with all foundational resource types
  • Test collector deduplication and conflict resolution (YAML overrides foundational)
  • Test error handling and graceful degradation scenarios
  • Test permission caching and RBAC integration
  • Test collector priority sorting and dual-path logic

Phase 2: Image Metadata Collection (Week 3)

• Registry Integration

  • Create pkg/collect/images/ package
  • Implement registry client with authentication support (Docker Hub, ECR, GCR, Harbor, etc.)
  • Add manifest parsing for Docker v2 and OCI formats
  • Create digest resolution from tags

• Facts Generation

  • Implement ImageFacts data structure with comprehensive metadata
  • Add image scanning and metadata extraction (platform, layers, config)
  • Create facts serialization to JSON with FactsBundle format
  • Add error handling and fallback modes with ContinueOnError

• Integration

  • Integrate image collection into auto-discovery system
  • Add image facts to foundational collectors
  • Create facts.json output specification with summary statistics
  • Add Kubernetes image extraction from pods, deployments, daemonsets, statefulsets

• Unit Testing ALL TESTS PASSING

  • Test registry client authentication and factory patterns for different registry types
  • Test manifest parsing for Docker v2, OCI, and legacy v1 image formats
  • Test digest resolution and validation with various formats
  • Test ImageFacts data structure serialization/deserialization
  • Test image metadata extraction with comprehensive validation
  • Test error handling for network failures and authentication
  • Test concurrent collection with rate limiting and semaphores
  • Test image facts caching and deduplication logic with LRU cleanup

Phase 3: CLI Integration (Week 4)

Note: Current CLI structure has --namespace already available. Successfully added --auto flag and related options.

CLI Usage Patterns for Dual-Path Approach

Path 1 - Foundational Only (No YAML):

# Collect foundational data for default namespace
support-bundle --auto

# Collect foundational data for specific namespace(s)  
support-bundle --auto --namespace myapp

# Include container image metadata
support-bundle --auto --namespace myapp --include-images

# Use comprehensive discovery profile
support-bundle --auto --discovery-profile comprehensive --include-images

Path 2 - YAML + Foundational (Augmented):

# Collect vendor YAML specs + foundational data
support-bundle vendor-spec.yaml --auto

# Multiple YAML specs + foundational data  
support-bundle spec1.yaml spec2.yaml --auto --namespace myapp

# Exclude system namespaces from foundational collection
support-bundle vendor-spec.yaml --auto --exclude-namespaces "kube-*,cattle-*"

Current Behavior (Preserved):

# Only collect what's in YAML (no foundational data added)
support-bundle vendor-spec.yaml

New Diff Command:

# Compare two support bundles
support-bundle diff old-bundle.tgz new-bundle.tgz

# Output to JSON file
support-bundle diff old.tgz new.tgz --output json -f diff-report.json

# Generate HTML report with remediation
support-bundle diff old.tgz new.tgz --output html --include-remediation

• Command Enhancement

  • Add --auto flag to support-bundle root command
  • Implement dual-path logic: no args+--auto = foundational only
  • Implement augmentation logic: YAML args+--auto = YAML + foundational
  • Integrate with existing --namespace filtering
  • Add --include-images option for container image metadata collection
  • Create --rbac-check validation mode (enabled by default)
  • Add support-bundle diff subcommand with full flag set

• Configuration

  • Add discovery profiles (minimal, standard, comprehensive, paranoid)
  • Add namespace exclusion/inclusion patterns with glob support
  • Implement dry-run mode integration for auto-discovery
  • Create discovery configuration file support with JSON format
  • Add profile-based timeout and collection behavior configuration

• Unit Testing ALL TESTS PASSING

  • Test CLI flag parsing and validation for all auto-discovery options
  • Test discovery profile loading and validation logic
  • Test dry-run mode integration and output
  • Test namespace filtering with glob patterns
  • Test command help text and flag descriptions
  • Test error handling for invalid CLI flag combinations
  • Test configuration file loading, validation, and fallbacks
  • Test dual-path mode detection and routing logic

Testing Strategy

• Unit Tests ALL PASSING

  • RBAC checker with mock Kubernetes API
  • Resource expansion logic and deduplication
  • Image metadata parsing and registry integration
  • Discovery configuration validation and pattern matching
  • CLI flag validation and profile loading
  • Bundle diff validation and output formatting

• Integration Tests IMPLEMENTED

  • End-to-end auto-discovery workflow testing
  • Permission boundary validation with mock RBAC
  • Image registry integration with mock HTTP servers
  • Namespace isolation verification
  • CLI integration with existing support-bundle system

• Performance Tests BENCHMARKED

  • Large cluster discovery performance (1000+ resources)
  • Image metadata collection at scale with concurrent processing
  • Memory usage during auto-discovery with caching
  • CLI flag parsing and configuration loading performance

Step-by-Step Implementation

Step 1: Set up Auto-Discovery Foundation

1. Create package structure: pkg/collect/autodiscovery/
2. Define AutoCollector interface with dual-path methods in interfaces.go
3. Implement FoundationalDiscoverer struct in discoverer.go
4. Define foundational collectors list (pods, deployments, services, configmaps, secrets, events, logs)
5. Add Kubernetes client initialization and configuration
6. Create unit tests for basic discovery functionality

Step 2: Implement Foundational Collection (Path 1)

1. Create foundational.go with predefined essential collector specs
2. Implement namespace-scoped resource enumeration for foundational resources
3. Add RBAC checking for each foundational collector type
4. Create deterministic resource expansion (same cluster → same collectors)
5. Add comprehensive unit tests for foundational collection

Step 3: Implement YAML Augmentation (Path 2)

1. Create augmenter.go to merge YAML collectors with foundational collectors
2. Implement deduplication logic (avoid collecting same resource twice)
3. Add priority system (YAML specs override foundational specs when conflict)
4. Create merger validation and conflict resolution
5. Add comprehensive unit tests for augmentation logic

Step 4: Build RBAC Checking Engine

1. Create rbac_checker.go with SelfSubjectAccessReview integration
2. Add permission caching with TTL for performance
3. Implement batch permission checking for efficiency
4. Add fallback modes for clusters with limited RBAC visibility
5. Create comprehensive RBAC test suite

Step 5: Add Image Metadata Collection

1. Create pkg/collect/images/ package with registry client
2. Implement manifest parsing for Docker v2 and OCI formats
3. Add authentication support (Docker Hub, ECR, GCR, etc.)
4. Create ImageFacts generation from manifest data
5. Add error handling and retry logic for registry operations

Step 6: Integrate with Existing Collection Pipeline

1. Modify existing pkg/collect/collect.go to support auto-discovery modes
2. Add CLI integration for --auto flag (Path 1) and YAML+auto mode (Path 2)
3. Create seamless integration with existing collector framework
4. Add streaming integration with redaction pipeline
5. Create facts.json output format and writer
6. Implement progress reporting and user feedback
7. Add configuration validation and error reporting

---

Component 2: Advanced Redaction with Tokenization

Objective

Enhance the existing redaction system (currently in pkg/redact/) with tokenization capabilities, optional local LLM assistance, and reversible redaction mapping for data owners.

Current State: The codebase has a functional redaction system with:

• File-based redaction using regex patterns
• Multiple redactor types (SingleLineRedactor, MultiLineRedactor, YamlRedactor, etc.)
• Redaction tracking and reporting via RedactionList
• Integration with collection pipeline

Requirements

• Streaming redaction: Enhance existing system to work as streaming step during collection
• Tokenization: Replace sensitive values with consistent tokens for traceability (new capability)
• LLM assistance: Optional local LLM for intelligent redaction detection (new capability)
• Reversible mapping: Generate redaction-map.json for token reversal by data owners (new capability)
• Performance: Maintain/improve performance of existing system for large support bundles
• Profiles: Extend existing redactor configuration with redaction profiles

Technical Specifications

2.1 Redaction Engine Architecture

Location: pkg/redact/

Core Components:

• engine.go - Main redaction orchestrator
• tokenizer.go - Token generation and mapping
• processors/ - File type specific processors
• llm/ - Local LLM integration (optional)
• profiles/ - Pre-defined redaction profiles

API Contract:

type RedactionEngine interface {
    ProcessStream(ctx context.Context, input io.Reader, output io.Writer, opts RedactionOptions) (*RedactionMap, error)
    GenerateTokens(ctx context.Context, values []string) (map[string]string, error)
    LoadProfile(name string) (*RedactionProfile, error)
}

type RedactionOptions struct {
    Profile        string
    EnableLLM      bool
    TokenPrefix    string
    StreamMode     bool
    PreserveFormat bool
}

type RedactionMap struct {
    Tokens    map[string]string `json:"tokens"`    // token -> original value
    Stats     RedactionStats    `json:"stats"`     // redaction statistics
    Timestamp time.Time         `json:"timestamp"` // when redaction was performed
    Profile   string            `json:"profile"`   // profile used
}

2.2 Tokenization System

Location: pkg/redact/tokenizer.go

Features:

• Consistent token generation for same values
• Configurable token formats and prefixes
• Token collision detection and resolution
• Metadata preservation (type hints, length preservation)

Token Format:

***TOKEN_<TYPE>_<HASH>***
Examples:
- ***TOKEN_PASSWORD_A1B2C3***
- ***TOKEN_EMAIL_X7Y8Z9***
- ***TOKEN_IP_D4E5F6***

2.3 LLM Integration (Optional)

Location: pkg/redact/llm/

Supported Models:

• Ollama integration for local models
• OpenAI compatible APIs
• Hugging Face transformers (via local API)

LLM Tasks:

• Intelligent sensitive data detection
• Context-aware redaction decisions
• False positive reduction
• Custom pattern learning

Implementation Checklist

Phase 1: Enhanced Redaction Engine (Week 1-2)

• Core Engine Refactoring

  • Refactor existing pkg/redact to support streaming
  • Create new RedactionEngine interface
  • Implement streaming processor for different file types
  • Add configurableprocessing pipelines

• Tokenization Implementation

  • Create Tokenizer with consistent hash-based token generation
  • Implement token mapping and reverse lookup
  • Add token format configuration and validation
  • Create collision detection and resolution

• File Type Processors

  • Create specialized processors for JSON, YAML, logs, config files
  • Add context-aware redaction (e.g., preserve YAML structure)
  • Implement streaming processing for large files
  • Add error recovery and partial redaction support

• Unit Testing

  • Test RedactionEngine with various input stream types and sizes
  • Test Tokenizer consistency - same input produces same tokens
  • Test token collision detection and resolution algorithms
  • Test file type processors with malformed/corrupted input files
  • Test streaming redaction performance with large files (GB scale)
  • Test error recovery and partial redaction scenarios
  • Test redaction map generation and serialization
  • Test token format validation and configuration options

Phase 2: Redaction Profiles (Week 3)

• Profile System

  • Create RedactionProfile data structure and parser
  • Implement built-in profiles (minimal, standard, comprehensive, paranoid)
  • Add profile validation and testing
  • Create profile override and customization system

• Profile Definitions

  • Minimal: Basic passwords, API keys, tokens
  • Standard: + IP addresses, URLs, email addresses
  • Comprehensive: + usernames, hostnames, file paths
  • Paranoid: + any alphanumeric strings > 8 chars, custom patterns

• Configuration

  • Add profile selection to support bundle specs
  • Create profile inheritance and composition
  • Implement runtime profile switching
  • Add profile documentation and examples

• Unit Testing

  • Test redaction profile parsing and validation
  • Test profile inheritance and composition logic
  • Test built-in profiles (minimal, standard, comprehensive, paranoid)
  • Test custom profile creation and validation
  • Test profile override and customization mechanisms
  • Test runtime profile switching without state corruption
  • Test profile configuration serialization/deserialization
  • Test profile pattern matching accuracy and coverage

Phase 3: LLM Integration (Week 4)

• LLM Framework

  • Create LLMProvider interface for different backends
  • Implement Ollama integration for local models
  • Add OpenAI-compatible API client
  • Create fallback modes when LLM is unavailable

• Intelligent Detection

  • Design prompts for sensitive data detection
  • Implement confidence scoring for LLM suggestions
  • Add human-readable explanation generation
  • Create feedback loop for improving detection

• Privacy & Security

  • Ensure LLM processing respects data locality
  • Add data minimization for LLM requests
  • Implement secure prompt injection prevention
  • Create audit logging for LLM interactions

• Unit Testing

  • Test LLMProvider interface implementations for different backends
  • Test LLM prompt generation and response parsing
  • Test confidence scoring algorithms for LLM suggestions
  • Test fallback mechanisms when LLM services are unavailable
  • Test prompt injection prevention with malicious inputs
  • Test data minimization - only necessary data sent to LLM
  • Test LLM response validation and sanitization
  • Test audit logging completeness and security

Phase 4: Integration & Artifacts (Week 5)

• Collection Integration

  • Integrate redaction engine into collection pipeline
  • Add streaming redaction during data collection
  • Implement progress reporting for redaction operations
  • Add redaction statistics and reporting

• Artifact Generation

  • Implement redaction-map.json generation and format
  • Add redaction statistics to support bundle metadata
  • Create redaction audit trail and logging
  • Implement secure token storage and encryption options

• Unit Testing

  • Test redaction integration with existing collection pipeline
  • Test streaming redaction performance during data collection
  • Test progress reporting accuracy and timing
  • Test redaction-map.json format compliance and validation
  • Test redaction statistics calculation and accuracy
  • Test redaction audit trail completeness
  • Test secure token storage encryption/decryption
  • Test error handling during redaction pipeline failures

Testing Strategy

• Unit Tests

  • Token generation and collision handling
  • File type processor accuracy
  • Profile loading and validation
  • LLM integration mocking

• Integration Tests

  • End-to-end redaction with real support bundles
  • LLM provider integration testing
  • Performance testing with large files
  • Streaming redaction pipeline validation

• Security Tests

  • Token uniqueness and unpredictability
  • Redaction completeness verification
  • Information leakage prevention
  • LLM prompt injection resistance

Step-by-Step Implementation

Step 1: Streaming Redaction Foundation

1. Analyze existing redaction code in pkg/redact
2. Design streaming architecture with io.Reader/Writer interfaces
3. Create RedactionEngine interface and base implementation
4. Implement file type detection and routing
5. Add comprehensive unit tests for streaming operations

Step 2: Tokenization System

1. Create Tokenizer with hash-based consistent token generation
2. Implement token mapping data structures and serialization
3. Add token format configuration and validation
4. Create collision detection and resolution algorithms
5. Add comprehensive testing for token consistency and security

Step 3: File Type Processors

1. Create processor interface and registry system
2. Implement JSON processor with path-aware redaction
3. Add YAML processor with structure preservation
4. Create log file processor with context awareness
5. Add configuration file processors for common formats

Step 4: Redaction Profiles

1. Design profile schema and configuration format
2. Implement built-in profile definitions
3. Create profile loading, validation, and inheritance system
4. Add profile documentation and examples
5. Create comprehensive profile testing suite

Step 5: LLM Integration (Optional)

1. Create LLM provider interface and abstraction layer
2. Implement Ollama integration for local models
3. Design prompts for sensitive data detection
4. Add confidence scoring and human-readable explanations
5. Create comprehensive privacy and security safeguards

Step 6: Integration and Artifacts

1. Integrate redaction engine into support bundle collection
2. Implement redaction-map.json generation and format
3. Add CLI flags for redaction options and profiles
4. Create comprehensive documentation and examples
5. Add performance monitoring and optimization

---

Component 3: Agent-Based Analysis

Objective

Enhance the existing analysis system (currently in pkg/analyze/) with agent-based capabilities and analyzer generation from requirements. This addresses the overview requirement for "Analyzer via agents (local/hosted) and 'generate analyzers from requirements'".

Current State: The codebase has a comprehensive analysis system with:

• 60+ built-in analyzers for various Kubernetes resources and conditions
• Host analyzers for system-level checks
• Structured analyzer results (AnalyzeResult type)
• Analysis download and local bundle processing
• Integration with support bundle collection
• JSON/YAML output formatting

Requirements

• Agent abstraction: Wrap existing analyzers and support local, hosted, and future agent types
• Analyzer generation: Create analyzers from requirement specifications (new capability)
• Analysis artifacts: Enhance existing results to generate structured analysis.json with remediation
• Offline capability: Maintain current local analysis capabilities
• Extensibility: Add plugin architecture for custom analysis engines while preserving existing analyzers

Technical Specifications

3.1 Analysis Engine Architecture

Location: pkg/analyze/

Core Components:

• engine.go - Analysis orchestrator
• agents/ - Agent implementations (local, hosted, custom)
• generators/ - Analyzer generation from requirements
• artifacts/ - Analysis result formatting and serialization

API Contract:

type AnalysisEngine interface {
    Analyze(ctx context.Context, bundle *SupportBundle, opts AnalysisOptions) (*AnalysisResult, error)
    GenerateAnalyzers(ctx context.Context, requirements *RequirementSpec) ([]AnalyzerSpec, error)
    RegisterAgent(name string, agent Agent) error
}

type Agent interface {
    Name() string
    Analyze(ctx context.Context, data []byte, analyzers []AnalyzerSpec) (*AgentResult, error)
    HealthCheck(ctx context.Context) error
    Capabilities() []string
}

type AnalysisResult struct {
    Results     []AnalyzerResult  `json:"results"`
    Remediation []RemediationStep `json:"remediation"`
    Summary     AnalysisSummary   `json:"summary"`
    Metadata    AnalysisMetadata  `json:"metadata"`
}

3.2 Agent Types

3.2.1 Local Agent

Location: pkg/analyze/agents/local/

Features:

• Built-in analyzer implementations
• No external dependencies
• Fast execution and offline capability
• Extensible through plugins

3.2.2 Hosted Agent

Location: pkg/analyze/agents/hosted/

Features:

• REST API integration with hosted analysis services
• Advanced ML/AI capabilities
• Cloud-scale processing
• Authentication and rate limiting

3.2.3 LLM Agent (Optional)

Location: pkg/analyze/agents/llm/

Features:

• Local or cloud LLM integration
• Natural language analysis descriptions
• Context-aware remediation suggestions
• Multi-modal analysis (text, logs, configs)

3.3 Analyzer Generation

Location: pkg/analyze/generators/

Requirements-to-Analyzers Mapping:

type RequirementSpec struct {
    APIVersion string                 `json:"apiVersion"`
    Kind       string                 `json:"kind"`
    Metadata   RequirementMetadata    `json:"metadata"`
    Spec       RequirementSpecDetails `json:"spec"`
}

type RequirementSpecDetails struct {
    Kubernetes KubernetesRequirements `json:"kubernetes"`
    Resources  ResourceRequirements   `json:"resources"`
    Storage    StorageRequirements    `json:"storage"`
    Network    NetworkRequirements    `json:"network"`
    Custom     []CustomRequirement    `json:"custom"`
}

Implementation Checklist

Phase 1: Analysis Engine Foundation (Week 1-2)

• Engine Architecture

  • Create pkg/analyze/ package structure
  • Design and implement AnalysisEngine interface
  • Create agent registry and management system
  • Add analysis result formatting and serialization

• Local Agent Implementation

  • Create LocalAgent with built-in analyzer implementations
  • Port existing analyzer logic to new agent framework
  • Add plugin loading system for custom analyzers
  • Implement performance optimization and caching

• Analysis Artifacts

  • Design analysis.json schema and format
  • Implement result aggregation and summarization
  • Add analysis metadata and provenance tracking
  • Create structured error handling and reporting

• Unit Testing

  • Test AnalysisEngine interface implementations
  • Test agent registry and management system functionality
  • Test LocalAgent with various built-in analyzers
  • Test analysis result formatting and serialization
  • Test result aggregation algorithms and accuracy
  • Test error handling for malformed analyzer inputs
  • Test analysis metadata and provenance tracking
  • Test plugin loading system with mock plugins

Phase 2: Hosted Agent Integration (Week 3)

• Hosted Agent Framework

  • Create HostedAgent with REST API integration
  • Implement authentication and authorization
  • Add rate limiting and retry logic
  • Create configuration management for hosted endpoints

• API Integration

  • Design hosted agent API specification
  • Implement request/response handling
  • Add data serialization and compression
  • Create secure credential management

• Fallback Mechanisms

  • Implement graceful degradation when hosted agents unavailable
  • Add local fallback for critical analyzers
  • Create hybrid analysis modes
  • Add user notification for service limitations

• Unit Testing

  • Test HostedAgent REST API integration with mock servers
  • Test authentication and authorization with various providers
  • Test rate limiting and retry logic with simulated failures
  • Test request/response handling and data serialization
  • Test fallback mechanisms when hosted agents are unavailable
  • Test hybrid analysis mode coordination and result merging
  • Test secure credential management and rotation
  • Test analysis quality assessment algorithms

Phase 3: Analyzer Generation (Week 4)

• Requirements Parser

  • Create RequirementSpec parser and validator
  • Implement requirement categorization and mapping
  • Add support for vendor and Replicated requirement specs
  • Create requirement merging and conflict resolution

• Generator Framework

  • Design analyzer generation templates
  • Implement rule-based analyzer creation
  • Add analyzer validation and testing
  • Create generated analyzer documentation

• Integration

  • Integrate generator with analysis engine
  • Add CLI flags for analyzer generation
  • Create generated analyzer debugging and validation
  • Add generator configuration and customization

• Unit Testing

  • Test requirement specification parsing with various input formats
  • Test analyzer generation from requirement specifications
  • Test requirement-to-analyzer mapping algorithms
  • Test custom analyzer template generation and validation
  • Test analyzer code generation quality and correctness
  • Test generated analyzer testing and validation frameworks
  • Test requirement specification validation and error reporting
  • Test analyzer generation performance and scalability

Phase 4: Remediation & Advanced Features (Week 5)

• Remediation System

  • Design RemediationStep data structure
  • Implement remediation suggestion generation
  • Add remediation prioritization and categorization
  • Create remediation execution framework (future)

• Advanced Analysis

  • Add cross-analyzer correlation and insights
  • Implement trend analysis and historical comparison
  • Create analysis confidence scoring
  • Add analysis explanation and reasoning

• Unit Testing

  • Test RemediationStep data structure and serialization
  • Test remediation suggestion generation algorithms
  • Test remediation prioritization and categorization logic
  • Test cross-analyzer correlation algorithms
  • Test trend analysis and historical comparison accuracy
  • Test analysis confidence scoring calculations
  • Test analysis explanation and reasoning generation
  • Test remediation framework extensibility and plugin system

Testing Strategy

• Unit Tests

  • Agent interface compliance
  • Analysis result serialization
  • Analyzer generation logic
  • Remediation suggestion accuracy

• Integration Tests

  • End-to-end analysis with real support bundles
  • Hosted agent API integration
  • Analyzer generation from real requirements
  • Multi-agent analysis coordination

• Performance Tests

  • Large support bundle analysis performance
  • Concurrent agent execution
  • Memory usage during analysis
  • Hosted agent latency and throughput

Step-by-Step Implementation

Step 1: Analysis Engine Foundation

1. Create package structure: pkg/analyze/
2. Define AnalysisEngine and Agent interfaces
3. Implement basic analysis orchestration
4. Create agent registry and management
5. Add comprehensive unit tests

Step 2: Local Agent Implementation

1. Create LocalAgent struct and implementation
2. Port existing analyzer logic to agent framework
3. Add plugin system for custom analyzers
4. Implement result caching and optimization
5. Create comprehensive test suite

Step 3: Analysis Artifacts

1. Design analysis.json schema and validation
2. Implement result serialization and formatting
3. Add analysis metadata and provenance
4. Create structured error handling
5. Add comprehensive format validation

Step 4: Hosted Agent Integration

1. Create HostedAgent with REST API client
2. Implement authentication and rate limiting
3. Add fallback and error handling
4. Create configuration management
5. Add integration testing with mock services

Step 5: Analyzer Generation

1. Create RequirementSpec parser and validator
2. Implement analyzer generation templates
3. Add rule-based analyzer creation logic
4. Create analyzer validation and testing
5. Add comprehensive generation testing

Step 6: Remediation System

1. Design remediation data structures
2. Implement suggestion generation algorithms
3. Add remediation prioritization and categorization
4. Create comprehensive documentation
5. Add remediation testing and validation

---

Component 4: Support Bundle Differencing

Objective

Implement comprehensive support bundle comparison and differencing capabilities to track changes over time and identify issues through comparison. This is a completely NEW capability not present in the current codebase.

Current State: The codebase has support bundle parsing utilities in pkg/supportbundle/parse.go that can extract and read bundle contents, but no comparison or differencing capabilities.

Requirements

• Bundle comparison: Compare two support bundles with detailed diff output (completely new)
• Change categorization: Categorize changes by type and impact (new)
• Diff artifacts: Generate structured diff.json for programmatic consumption (new)
• Visualization: Human-readable diff reports (new)
• Performance: Handle large bundles efficiently using existing parsing utilities

Technical Specifications

4.1 Diff Engine Architecture

Location: pkg/supportbundle/diff/

Core Components:

• engine.go - Main diff orchestrator
• comparators/ - Type-specific comparison logic
• formatters/ - Output formatting (JSON, HTML, text)
• filters/ - Diff filtering and noise reduction

API Contract:

type DiffEngine interface {
    Compare(ctx context.Context, oldBundle, newBundle *SupportBundle, opts DiffOptions) (*BundleDiff, error)
    GenerateReport(ctx context.Context, diff *BundleDiff, format string) (io.Reader, error)
}

type BundleDiff struct {
    Summary      DiffSummary         `json:"summary"`
    Changes      []Change            `json:"changes"`
    Metadata     DiffMetadata        `json:"metadata"`
    Significance SignificanceReport  `json:"significance"`
}

type Change struct {
    Type        ChangeType         `json:"type"`        // added, removed, modified
    Category    string             `json:"category"`    // resource, log, config, etc.
    Path        string             `json:"path"`        // file path or resource path
    Impact      ImpactLevel        `json:"impact"`      // high, medium, low, none
    Details     map[string]any     `json:"details"`     // change-specific details
    Remediation *RemediationStep   `json:"remediation,omitempty"`
}

4.2 Comparison Types

4.2.1 Resource Comparisons

• Kubernetes resource specifications
• Resource status and health changes
• Configuration drift detection
• RBAC and security policy changes

4.2.2 Log Comparisons

• Error pattern analysis
• Log volume and frequency changes
• New error types and patterns
• Performance metric changes

4.2.3 Configuration Comparisons

• Configuration file changes
• Environment variable differences
• Secret and ConfigMap modifications
• Application configuration drift

Implementation Checklist

Phase 1: Diff Engine Foundation (Week 1-2)

• Core Engine

  • Create pkg/supportbundle/diff/ package structure
  • Implement DiffEngine interface and base implementation
  • Create bundle loading and parsing utilities
  • Add diff metadata and tracking

• Change Detection

  • Implement file-level change detection
  • Create content comparison utilities
  • Add change categorization and classification
  • Implement impact assessment algorithms

• Data Structures

  • Define BundleDiff and related data structures
  • Create change serialization and deserialization
  • Add diff statistics and summary generation
  • Implement diff validation and consistency checks

• Unit Testing

  • Test DiffEngine with various support bundle pairs
  • Test bundle loading and parsing utilities with different formats
  • Test file-level change detection algorithms
  • Test content comparison utilities with binary and text files
  • Test change categorization and classification accuracy
  • Test BundleDiff data structure serialization/deserialization
  • Test diff statistics calculation and accuracy
  • Test diff validation and consistency check algorithms

Phase 2: Specialized Comparators (Week 3)

• Resource Comparator

  • Create Kubernetes resource diff logic
  • Add YAML/JSON structural comparison
  • Implement semantic resource analysis
  • Add resource health status comparison

• Log Comparator

  • Create log file comparison utilities
  • Add error pattern extraction and comparison
  • Implement log volume analysis
  • Create performance metric comparison

• Configuration Comparator

  • Add configuration file diff logic
  • Create environment variable comparison
  • Implement secret and sensitive data handling
  • Add configuration drift detection

• Unit Testing

  • Test Kubernetes resource diff logic with various resource types
  • Test YAML/JSON structural comparison algorithms
  • Test semantic resource analysis and health status comparison
  • Test log file comparison utilities with different log formats
  • Test error pattern extraction and comparison accuracy
  • Test log volume analysis algorithms
  • Test configuration file diff logic with various config formats
  • Test sensitive data handling in configuration comparisons

Phase 3: Output and Visualization (Week 4)

• Diff Artifacts

  • Implement diff.json generation and format
  • Add diff metadata and provenance
  • Create diff validation and schema
  • Add diff compression and storage

• Report Generation

  • Create HTML diff reports with visualization
  • Add interactive diff navigation and filtering
  • Implement diff report customization and theming
  • Create diff report export and sharing capabilities

• Unit Testing

  • Test diff.json generation and format validation
  • Test diff metadata and provenance tracking
  • Test diff compression and storage mechanisms
  • Test HTML diff report generation with various diff types
  • Test interactive diff navigation functionality
  • Test diff report customization and theming options
  • Test diff visualization accuracy and clarity
  • Test diff report export formats and compatibility
  • Add text-based diff output
  • Implement diff filtering and noise reduction
  • Create diff summary and executive reports

Phase 4: CLI Integration (Week 5)

• Command Implementation

  • Add support-bundle diff command
  • Implement command-line argument parsing
  • Add progress reporting and user feedback
  • Create diff command validation and error handling

• Configuration

  • Add diff configuration and profiles
  • Create diff ignore patterns and filters
  • Implement diff output customization
  • Add diff performance optimization options

Step-by-Step Implementation

Step 1: Diff Engine Foundation

1. Create package structure: pkg/supportbundle/diff/
2. Design DiffEngine interface and core data structures
3. Implement basic bundle loading and parsing
4. Create change detection algorithms
5. Add comprehensive unit tests

Step 2: Change Detection and Classification

1. Implement file-level change detection
2. Create content comparison utilities with different strategies
3. Add change categorization and impact assessment
4. Create change significance scoring
5. Add comprehensive classification testing

Step 3: Specialized Comparators

1. Create comparator interface and registry
2. Implement resource comparator with semantic analysis
3. Add log comparator with pattern analysis
4. Create configuration comparator with drift detection
5. Add comprehensive comparator testing

Step 4: Output Generation

1. Implement diff.json schema and serialization
2. Create HTML report generation with visualization
3. Add text-based diff formatting
4. Create diff filtering and noise reduction
5. Add comprehensive output validation

Step 5: CLI Integration

1. Add diff command to support-bundle CLI
2. Implement argument parsing and validation
3. Add progress reporting and user experience
4. Create comprehensive CLI testing
5. Add documentation and examples

---

Integration & Testing Strategy

Integration Contracts (Critical Constraints)

Person 2 is a CONSUMER of Person 1's work and must NOT alter schema definitions or CLI contracts.

Schema Contract (Owned by Person 1)

CRITICAL UPDATE: Based on current codebase analysis:

• Current API Group: troubleshoot.replicated.com (NOT troubleshoot.sh)
• Current Versions: v1beta1 and v1beta2 are available (NO v1beta3 exists yet)
• Use ONLY troubleshoot.replicated.com/v1beta2 CRDs/YAML spec definitions until Person 1 provides schema migration plan
• Follow EXACTLY agreed-upon artifact filenames (analysis.json, diff.json, redaction-map.json, facts.json)
• NO modifications to schema definitions, types, or API contracts
• All schemas act as the cross-team contract with clear compatibility rules

CLI Contract (Owned by Person 1)

CRITICAL UPDATE: Based on current CLI structure analysis:

• Current Structure: support-bundle (root/collect), support-bundle analyze, support-bundle redact
• Existing Flags: --namespace, --redact, --collect-without-permissions, etc. already available
• NEW Commands to Add: support-bundle diff (completely new)
• NEW Flags to Add: --auto, --include-images, --rbac-check, --agent
• NO changes to existing CLI surface area, help text, or command structure
• Must integrate new capabilities into existing command structure

IO Flow Contract (Owned by Person 2)

• Collect/analyze/diff operations read and write ONLY via defined schemas and filenames
• Redaction runs as streaming step during collection (no intermediate files)
• All input/output must conform to Person 1's schema specifications

Golden Samples Contract

• Use checked-in example specs and artifacts for contract testing
• Ensure changes don't break consumers or violate schema contracts
• Maintain backward compatibility with existing artifact formats

Cross-Component Integration

Collection → Redaction Pipeline

// Example integration flow
func CollectWithRedaction(ctx context.Context, opts CollectionOptions) (*SupportBundle, error) {
    // 1. Auto-discover collectors
    collectors, err := autoCollector.Discover(ctx, opts.DiscoveryOptions)
    if err != nil {
        return nil, err
    }
    
    // 2. Collect with streaming redaction
    bundle := &SupportBundle{}
    for _, collector := range collectors {
        data, err := collector.Collect(ctx)
        if err != nil {
            continue
        }
        
        redactedData, redactionMap, err := redactionEngine.ProcessStream(ctx, data, opts.RedactionOptions)
        if err != nil {
            return nil, err
        }
        
        bundle.AddFile(collector.OutputPath(), redactedData)
        bundle.AddRedactionMap(redactionMap)
    }
    
    return bundle, nil
}

Analysis → Remediation Integration

// Example analysis to remediation flow
func AnalyzeWithRemediation(ctx context.Context, bundle *SupportBundle) (*AnalysisResult, error) {
    // 1. Run analysis
    result, err := analysisEngine.Analyze(ctx, bundle, opts)
    if err != nil {
        return nil, err
    }
    
    // 2. Generate remediation suggestions
    for i, analyzerResult := range result.Results {
        if analyzerResult.IsFail() {
            remediation, err := generateRemediation(ctx, analyzerResult)
            if err == nil {
                result.Results[i].Remediation = remediation
            }
        }
    }
    
    return result, nil
}

Comprehensive Testing Strategy

Unit Testing Requirements

• Coverage Target: >80% code coverage for all components
• Mock Dependencies: Mock all external dependencies (K8s API, registries, LLM APIs)
• Error Scenarios: Test all error paths and edge cases
• Performance: Unit benchmarks for critical paths

Integration Testing Requirements

• End-to-End Flows: Complete collection → redaction → analysis → diff workflows
• Real Cluster Testing: Integration with actual Kubernetes clusters
• Large Bundle Testing: Performance with multi-GB support bundles
• Network Conditions: Testing with limited/intermittent connectivity

Performance Testing Requirements

• Memory Usage: Monitor memory consumption during large operations
• CPU Utilization: Profile CPU usage for optimization opportunities
• I/O Performance: Test with large files and slow storage
• Concurrency: Test multi-threaded operations and race conditions

Security Testing Requirements

• Redaction Completeness: Verify no sensitive data leakage
• Token Security: Ensure token unpredictability and uniqueness
• Access Control: Verify RBAC enforcement
• Input Validation: Test against malicious inputs

Golden Sample Testing

• Reference Bundles: Create standard test support bundles
• Expected Outputs: Define expected analysis, diff, and redaction outputs
• Regression Testing: Automated comparison against golden outputs
• Schema Validation: Ensure all outputs conform to schemas

---

Documentation Requirements

User Documentation

• Collection Guide: How to use auto-collectors and namespace scoping
• Redaction Guide: Redaction profiles, tokenization, and LLM integration
• Analysis Guide: Agent configuration and remediation interpretation
• Diff Guide: Bundle comparison workflows and interpretation

Developer Documentation

• API Documentation: Go doc comments for all public APIs
• Architecture Guide: Component interaction and data flow
• Extension Guide: How to add custom agents, analyzers, and processors
• Performance Guide: Optimization techniques and benchmarks

Configuration Documentation

• Schema Reference: Complete reference for all configuration options
• Profile Examples: Example redaction and analysis profiles
• Integration Examples: Sample integrations with CI/CD and monitoring

---

Timeline & Milestones

Month 1: Foundation

• Week 1-2: Auto-collectors and RBAC integration
• Week 3-4: Advanced redaction with tokenization

Month 2: Advanced Features

• Week 5-6: Agent-based analysis system
• Week 7-8: Support bundle differencing

Month 3: Integration & Polish

• Week 9-10: Cross-component integration and testing
• Week 11-12: Documentation, optimization, and release preparation

Key Milestones

• M1: Auto-discovery working with RBAC (Week 2)
• M2: Streaming redaction with tokenization (Week 4)
• M3: Local and hosted agents functional (Week 6)
• M4: Bundle diffing and remediation (Week 8)
• M5: Full integration and testing complete (Week 10)
• M6: Documentation and release ready (Week 12)

---

Success Criteria

Functional Requirements

• support-bundle collect --namespace ns --auto produces complete bundles
• Redaction with tokenization works with streaming pipeline
• Analysis generates structured results with remediation
• Bundle diffing produces actionable comparison reports

Performance Requirements

• Auto-discovery completes in <30 seconds for typical clusters
• Redaction processes 1GB+ bundles without memory issues
• Analysis completes in <2 minutes for standard bundles
• Diff generation completes in <1 minute for bundle pairs

Quality Requirements

• >80% code coverage with comprehensive tests
• Zero critical security vulnerabilities
• Complete API documentation and user guides
• Successful integration with Person 1's schema and CLI contracts

---

Final Integration Testing Phase

After all components are implemented and unit tested, conduct comprehensive integration testing to verify the complete system works together:

End-to-End Integration Testing

1. Complete Workflow Testing

• Test full support-bundle collect --namespace ns --auto workflow
• Test auto-discovery → collection → redaction → analysis → diff pipeline
• Test CLI integration with real Kubernetes clusters
• Test support bundle generation with all auto-discovered collectors
• Test complete artifact generation (bundle.tgz, facts.json, redaction-map.json, analysis.json)

2. Cross-Component Integration

• Test auto-discovery integration with image metadata collection
• Test streaming redaction integration with collection pipeline
• Test analysis engine integration with auto-discovered collectors and redacted data
• Test support bundle diff functionality with complete bundles
• Test remediation suggestions integration with analysis results

3. Real-World Scenario Testing

• Test against real Kubernetes clusters with various configurations
• Test with different RBAC permission levels and restrictions
• Test with various application types (web apps, databases, microservices)
• Test with large clusters (1000+ pods, 100+ namespaces)
• Test with different container registries (Docker Hub, ECR, GCR, Harbor)

4. Performance and Reliability Integration

• Test end-to-end performance with large, complex clusters
• Test system reliability with network failures and API errors
• Test memory usage and resource consumption across all components
• Test concurrent operations and thread safety
• Test scalability limits and graceful degradation under load

5. Security and Privacy Integration

• Test RBAC enforcement across the entire pipeline
• Test redaction effectiveness with real sensitive data
• Test token reversibility and data owner access to redaction maps
• Test LLM integration security and data locality compliance
• Test audit trail completeness across all operations

6. User Experience Integration

• Test CLI usability and help documentation
• Test configuration file examples and documentation
• Test error messages and user feedback across all components
• Test progress reporting and operation status visibility
• Test troubleshoot.sh ecosystem integration and compatibility

7. Artifact and Output Integration

• Test support bundle format compliance and compatibility
• Test analysis.json schema validation and tool compatibility
• Test diff.json format and visualization integration
• Test redaction-map.json usability and token reversal
• Test facts.json integration with analysis and visualization tools

---

MAJOR CHANGES FROM ORIGINAL PRD

This section documents all critical changes made to align the PRD with the actual troubleshoot codebase:

1. API Schema Reality Check

• CHANGED: API group from troubleshoot.sh/v1beta3 → troubleshoot.replicated.com/v1beta2
• REASON: Current codebase only has v1beta1 and v1beta2, using troubleshoot.replicated.com group

2. Implementation Strategy Shift

• CHANGED: From "build from scratch" → "extend existing systems"
• REASON: Discovered mature, production-ready systems already exist
• IMPACT: Faster implementation, better integration, lower risk

3. CLI Structure Alignment

• CHANGED: Command structure from support-bundle collect/analyze/diff → enhance existing support-bundle root + subcommands
• REASON: Current structure already has support-bundle (collect), support-bundle analyze, support-bundle redact
• NEW: Only support-bundle diff is completely new

4. Binary Architecture Reality

• DISCOVERED: Multiple binaries already exist (preflight, support-bundle, collect, analyze)
• IMPACT: Two-binary approach already partially implemented
• FOCUS: Enhance existing support-bundle binary capabilities

5. Existing System Capabilities

• Collection: 15+ collector types, RBAC integration, progress reporting
• Redaction: Regex-based, multiple redactor types, tracking/reporting
• Analysis: 60+ analyzers, host+cluster analysis, structured results
• Support Bundle: Complete archiving, parsing, metadata system

6. Removed All Completion Markers

• CHANGED: All ``, [ ], "" markers → [ ] (pending)
• REASON: Starting implementation from scratch despite existing foundation

7. Technical Approach Updates

• Auto-collectors: NEW package extending existing collection framework with dual-path approach
• Redaction: ENHANCE existing system with tokenization and streaming
• Analysis: WRAP existing analyzers with agent abstraction layer
• Diff: COMPLETELY NEW capability using existing bundle parsing

8. Auto-Collectors Foundational Data Definition

What "Foundational Data" Includes:

• Pods: All pods in target namespace(s) with full spec and status
• Deployments/ReplicaSets: All deployment resources and their managed replica sets
• Services: All service definitions and endpoints
• ConfigMaps: All configuration data (with redaction)
• Secrets: All secret metadata (values redacted by default)
• Events: Recent cluster events for troubleshooting context
• Pod Logs: Container logs from all pods (with retention limits)
• Image Facts: Container image metadata (digests, tags, registry info)
• Network Policies: Any network policies affecting the namespace
• RBAC: Relevant roles, role bindings, service accounts

This foundational collection ensures that even without vendor-specific YAML specs, support bundles contain the essential data needed for troubleshooting most Kubernetes issues.

This updated PRD provides a realistic, implementable roadmap that leverages existing production-ready code while adding the new capabilities specified in the original requirements. The implementation risk is significantly reduced, and the timeline is more achievable.