SendGrid Platform Unavailability
Incident
Report for SendGrid
Resolved
Our engineers have monitored system performance and confirmed the issue with Mail Send and Statistics between the period of 7:40 AM PDT and 8:54 AM PDT has been resolved. All services are now operating normally at this time.
Update
Our engineering teams are continuing to monitor system performance. We will provide an update in 3 hours, or as soon as more information is available.
Update
Our engineering teams are continuing to monitor system performance. We will provide an update in 2 hours, or as soon as more information is available.
Monitoring
At 1:01 PM PDT, our engineering team confirmed that all delayed events between the period of 7:40 AM PDT and 8:54 AM PDT for mail send and statistics were fully dequeued and processed successfully. Another update will be provided in the next 30 minutes in order to confirm full recovery and stability of the SendGrid platform.
Update
At this time, our engineers have confirmed to have mitigated the majority of impact for degraded components. The impact time window is confirmed to have started at approximately 7:40 AM PDT. At 8:08 AM PDT, our engineering team deployed a change to mitigate degraded components. At 8:54 AM PDT, our team confirmed that this same deploy was fully in effect and allowed any new mail send and statistics to process properly from that time. However, mail send and statistics requests that were requested, scheduled or deferred between the times of 7:40 AM PDT and 8:54 AM PDT may still be impacted with delays until further notice. New mail send requests, Marketing Campaigns usage, SendGrid API and UI usage, statistics retrieval and email activity usage after 8:54 AM PDT is confirmed to be working as expected and fully functional. We will provide another update in 2 hours or as soon as possible.
Identified
Our engineers have identified the problems related to platform unavailability and problems will continue to be explored for resolution. Currently, users of the SendGrid system may experience delays in requested mail send, errors when using Marketing Campaigns, errors when using the SendGrid API and user interface, as well as delays in retrieval of statistics. This incident does impact mail send. We will provide another update in 1 hour or as soon as possible.
Update
Our engineers are still investigating various issues reported and known to be impacting the SendGrid platform. At this time, users of the SendGrid system may continue to experience issues using both the API and web console interface. We will continue to investigate and provide further details as they become known. We will provide another update in 1 hour or as soon as possible.
Investigating
Starting around 7:45 AM PDT on August 15th, our engineers began investigating issues with the SendGrid platform. As of now, users will experience issues logging into the platform and accessing the user interface. Other errors may be present. We are continuing to investigate the full scope of the issue and will update impact. We will provide another update in 60 minutes or as soon as more updates become available.
This incident affected: Marketing Campaigns (Marketing Campaigns), Mail Sending (API v3, SMTP, API v2), and API, Website, Statistics, Email Activity.