Parallel Asynchronous Processing: Emails & SMS Notifications Best Practices
Emails, texts, and other notifications are critical for engaging with customers. No matter what form they take – alerts, daily summaries, weekly updates, special notices, or monthly newsletters – emails and notifications keep your customers informed, provide them with core services, and help drive revenue.
These notifications come in a growing number of formats delivered across an expanding set of client/message types. Many are triggered by user actions – whether based on a user’s schedule (2 days after signup) or their activity (after they bounce from a shopping cart). And they are increasingly being personalized, whether it’s a report of daily activity or recommendations and offers created specifically for each user.
Table of Contents
Flexibility, Horsepower, and Scale
Orchestrating and sending non-trivial notifications demands an agile framework, processing horsepower, and an easy way to scale the send and distribution. Using workers within IronWorker, it’s a simple matter to create a new types of notification without affecting existing workers or the current notification flows. All you need to do is create new worker objects, upload them to IronWorker, and then you're free to scale out the workload as you see fit. You can also use IronMQ to orchestrate a sequence of steps or interactions.
Sending out the same copy of a newsletter to thousands of customers isn't difficult, but customizing or personalizing them is another matter. Each email, text, or push notification entails pulling up information specific to each user and inserting it into a specialized message format or template. If it’s daily activity, the data might be in a ready form for inclusion. Other types of data, like personalized offers or billing summaries, might need heavier processing (which, in these cases, might mean continual tasks running in the background to prep the data for the message). A few servers or even dozens of servers aren’t likely going to cut it even with a modest amount of users.
This is where IronWorker excels. Just queue up thousands of tasks on IronWorker and they’ll run in parallel across tons of servers. Most apps send emails and notifications at set times (first thing in the morning or at the end of the day) which means the process can't take hours to do. Spreading the worker out across a set of elastic servers reduces the duration and lets you get them out in near real-time. As an example, one Iron.io customer converted from a few self-managed servers to IronWorker and was able to turn a 9-hour job sending emails into a 9-minute process.
Steps in the Process/Basic Notification Worker
Here are the steps for using IronWorker to send out a large number of notifications:
The notification worker itself might have three main actions to it – 1) initialize the message headers, 2) prepare and send the notification, and 3) signal exceptions/record the status. The first two parts are relatively self-explanatory. The third step, signaling and recording, consists of notifying the app of any errors that arise and logs the email/notification as part of the user record.
We generally recommend worker running times on the order of minutes to tens of minutes. This provides sufficient time to get work done but isn't so long that it becomes difficult to get insight into what's happening within a worker. Longer running processes also mean less parallelism and so the total duration of the work would extend the length of your longest worker plus some.
Iron.io Serverless Tools
Speak to us to learn how IronWorker and IronMQ are essential products for your application to send email and SMS notifications.
Thanks for subscribing! Please check your email for further instructions.
Running Thousands of Tasks at a Time
As with the image processing example (blog), the big benefit comes in being able to run a worker thousands of time – on a schedule at one or more set times or whenever app logic/circumstances dictate.
Here's a case where you might run through a list of users, slicing the work into batches of 100 message sends. (This example uses the iron_worker_ruby_ng library. The params contain the data payload that gets passed to the worker.)
To schedule the notification workers, you just need to schedule a master worker that then performs the steps above. (The code snippet below schedules the worker for 8pm and runs it every 24 hours.)
Using IronMQ to Orchestrate the Work
You can also use IronMQ to orchestrate the workers. Message items can be put on a queue containing user IDs or other identifiers. Scheduled workers can periodically run to queue up notification workers to take items from the queue and send the notifications.
In this situation, it can make sense to have the notification workers get the items from the queue directly. In this way, you're able to use the get-delete flow of IronMQ as a backstop to make sure the notifications get sent. The worker would get an item and then after a successful send, it would delete it from the queue. If the prep or sending process fails, the item can be explicitly put back on the queue or it can just be left to expire, in which case it also gets put back on the queue after the expiration time.
IronMQ and IronCache can also be used to extend the workflow. For example, say there are other actions that take place such as taking the daily report that's generated, converting it into a PDF file and posting a link to the user's account or even putting it into a user's DropBox account. Rather than extend the worker, you can create a queue, post items to the queue, and have others workers perform these actions. Makes for a simple, extensible – and highly scalable – process.
More on Email and Notifications
To learn more about how IronWorker, IronMQ, and IronCache can help scale your email, SMS, and notification jobs, visit our Dev Center and check out the examples in GitHub: IronWorker examples. There’s also a detailed blog post on IronWorker, ActionMailer, and SendGrid here.
Set Up Email & SMS Notifications With Iron
Find out how IronWorker and IronMQ can help your application obtain the cloud with fanatical customer support, reliable performance, and competitive pricing.