Real Time Monitoring : Know Everything
Instagram released an awesome overview of how they handled the incredible scaling issues associated with 40+ million people using the service. Their theory of monitoring everything was exactly what I’ve been trying to do at @EggDropApp.
A great tool they highlighted was Statsd, an open source monitoring server from Etsy. It can collect basic stats from any service in your system with minimum overhead and generate easy-to-use graphs on the fly. The basic stats can be simple counters or timing that can be averaged over selectable periods. The interface is UDP-based and a breeze to use.
This is something that any startup can use. Given that, I tossed together an EC2 AMI that launches a standalone Statsd server ready for action. Launch it in your environment and all you need to worry about is integrating stats. Grab the link below and give it a whirl.
Check out Instagram’s incredible post and the original blog post from Etsy.
* Minor note on logging into the Graphite web:
Log into the box
$ cd /opt/graphite/webapp/graphite
$ sudo python manage.py createsuperuser stats
Follow the steps to create your new user
Open up the site in your browser and log in.