Engineering · Reliability
Reliability
2 write-ups on reliability.
- The outage that logged at debug levelA feature degraded silently in production for weeks because three separate fallback paths turned a hard API failure into a quiet one. A postmortem on defensive code that defends the wrong thing.
- The one row that ate the queueA background worker with no terminal-failure state retried a single unprocessable item forever, starving every healthy item behind it and burning paid API quota continuously.