We had a cache server that crashed in a way where it received incoming requests but did not answer. This resulted in incoming requests queuing up and waiting for a response in the end creating a domino effect where no new requests could be received as everyone was waiting for the faulty cache server.
Normally such issues are detected automatically and faulty equipment is automatically taken out of production. We are looking into why this did not happen in this case and ensure that it will in the future.
We are sorry for the hiccup and hope you have a great day now that this has been resolved.
Posted Jul 07, 2021 - 14:36 CEST
Monitoring
We are back up and are monitoring the situation...
Posted Jul 07, 2021 - 14:25 CEST
Identified
We have identified the problem to be internal routing to our caching servers. We are resolving the issue now and will be back with updates shortly.
Posted Jul 07, 2021 - 14:20 CEST
Investigating
We are experiencing a full API outage and will get back shortly with further details...