Auto Trader API Error
Incident Report for Auto Trader
Postmortem

Following a thorough investigation, we have determined the root cause of the recent incident stemmed from a missing index in one of our databases; leading to the failure of our services to establish connections and causing them to exhaust the connection pools within our API gateway due to repeated retries.

The oversight occurred during a code repository merge, where the absence of indexes went unnoticed. Once identified, we promptly addressed the issue, implementing necessary fixes to prevent its recurrence.

While the incident resulted in degraded access to our entire API suite, we are pleased with the swift resolution achieved. We remain committed to maintaining the reliability and performance of our services, prioritizing proactive measures to prevent similar occurrences in the future.

Posted Mar 20, 2024 - 15:05 GMT

Resolved
This incident has been resolved.
Posted Mar 18, 2024 - 17:04 GMT
Update
We are continuing to monitor for any further issues.
Posted Mar 18, 2024 - 17:01 GMT
Monitoring
We are seeing some stability come back to our APIs. We are continuing to monitor.
Posted Mar 18, 2024 - 17:01 GMT
Update
We have identified the cause of this and a fix is being deployed
Posted Mar 18, 2024 - 16:58 GMT
Identified
Our monitoring has detected a performance/stability issue impacting a number of Auto Trader's API. An Engineer has been assigned to investigate.
Posted Mar 18, 2024 - 16:54 GMT
This incident affected: AT Connect (Image API, Taxonomy API, Vehicle API, Stock API, Search API, Valuations API, Historic Valuations API, Future Valuations API, Vehicle Metrics API, Leads API) and Bespoke APIs (Consumer Disposal API, Insurance Valuation Service).