The issue has been resolved. We will publish a post mortem as soon as we finalize our internal investigation and get more answers from the AWS team.
Posted Sep 09, 2021 - 10:20 UTC
Monitoring
The lag is gradually closing and is currently under 30 minutes. We are continuing to monitor what seems to be an issue with the new storage type (GP3) and are actively replacing some of the disks back to GP2.
Posted Sep 09, 2021 - 10:09 UTC
Update
We are continuing to investigate - currently unable to provide an accurate ETA. The index lag is ±1 hour at the moment for accounts over 500GB/day.
Posted Sep 09, 2021 - 09:47 UTC
Update
Currently only accounts over 500GB/day in EU region should be affected.
Posted Sep 09, 2021 - 08:52 UTC
Identified
Our team is working to resolve an issue that had occurred due to HW problems in AWS Ireland region - we are currently recovering our cluster and expecting to provide an accurate ETA within minutes. Alerts, Live Tail, Archive queries are working as expected.