# Disaster Recovery Runbook ## Failure Recovery Protocols --- ## 1. Primary MySQL Outage 1. Monitoring detects primary database failure; triggers automated failover to healthy Read Replica. 2. Read Replica promoted to Primary. 3. FastAPI database connection pool automatically reconnects to new Primary. ## 2. OpenSearch Cluster Outage 1. OpenSearch Circuit Breaker trips automatically. 2. Search requests fall back to restricted simple ID/SKU lookups in MySQL. 3. Search Worker pauses indexing events in RabbitMQ queue. 4. Once OpenSearch cluster recovers, Celery trigger replays Outbox events to rebuild search indices.