I gave a talk at Dreamforce a few weeks ago, titled “Create a Massively Scalable System with Force.com and Heroku” in which I presented a simple web application and showed how to make it painlessly scale. There was a great turn out so it seems as though the topic is quite popular.
“Discover how to integrate Force.com with a Heroku app in a scalable, spikable, loosely coupled way. We will use the Force.com API, Iron.io message queues and task queues, and the Boomi integration service to send data, transform, notify and connect events between Salesforce and an app running on Heroku. CIOs, architects, and developers will learn how to use cloud applications and technologies to augment existing resources and processes, and receive working code samples and best practices to balance agility, reliability, and scale.”
The main points I wanted to get across in the presentation are using queues are a key tool to achieve scalability.
- Use task queues to run jobs asynchronously in the background -- moving the load to systems that won't affect your users and can scale independently of your application
- Use queues to deal with spikes -- queues soak up spikes so you don't need to waste money on excess resources
- Use queues to keep database load consistent -- your database is almost always the bottleneck, you need to be able to control your database load
- Use queues for integration between systems -- keeps systems loosely coupled and allows you to stay within rate limits
I will follow up in the near future with focused posts on the above topics. And if you'd like to see the code for it, it's on GitHub here.
Dreamforce was an amazing conference and talking at it was a great experience. One thing that stuck with me in the days after was that we need to have IronCon and it's gotta be awesome like Dreamforce. 😉