Sunday, September 29, 2019

How to setup an Alarm when RDS is running on low free disk space

Yea, that happened to me!

The Problem

My RDS instance suddenly ran out of space and some of our applications started failing left, right and centre. It was a disaster and a fair bit of fire fighting was involved.

I told to myself, how did this happen? I should have put checks in place to ensure this didn't happen. I should have added some sort of alarm to warn when free disk space is low.

To deal with this, we wanted to first setup an alarm to notify the team when RDS instance is running on low free disk space. Looked at AWS console to create the alarm, but - I must admit - we were a bit surprised to see that there isn't like a straightforward way to create this type of an alarm.

The Solution

After a little googling, we found the way to setup the Alarm. This post is to document the steps involved in getting this done so that, I do not forget them :D

We basically need to do the following

  • Create an SNS topic to that can send emails
  • Subscribe the team email address to the SNS topic
  • Confirm the email subscription by clicking on the link that AWS sends.
  • Create a cloud watch alarm to send the alert when the RDS free disk space is lesser than the chosen threshold

That's all there is to it!
Have some Fun!