Task scheduling on Heroku
Task Scheduling Overview
Scheduling and automating background tasks are critical components for creating and maintaining a dependable and efficient product. Yet still, it may be challenging to get started and choose when and how to start the scheduling process.
Several applications require scheduled job execution. For instance, an app may be required to check a remote API every five minutes or deliver an email report. While the cron tool can be used for this purpose, it is unsuitable for horizontally scaled systems like Heroku. A work scheduler is a more robust and adaptable option.
One of the fundamental concepts you'll need to familiarize yourself with in Heroku is Dynos. Dynos are segregated, virtualized Linux containers optimized for rapid startup and execution of programs provided by the user. Consider them to be little servers optimized for the execution of a single task. Each application comprises one or more components that use several dynos: web dynos provide access to HTTP/S ports and are responsive to web requests. In contrast, worker dynos typically consume and process messages from a queue. Along with dynos that execute long-running components of your application, Heroku now supports one-off dynos that perform one-off operations that typically make use of your deployed application and its code. One-off dynos are advantageous for performing scheduled activities since you only pay for dyno uptime.
A task scheduler uses the process model to operate a lightweight, singular process that rises at a defined frequency and arranges tasks to be done. When combined with background workers, it creates a streamlined and adaptable task scheduling method.
The responsibilities of scheduling and performing the job are distinct. Separating the execution of a task from its scheduling means that each component's duties are well defined, resulting in a more manageable system.
Use a task scheduler to queue, not to perform background tasks. The queued tasks are subsequently received and executed separately by background workers.
By separating scheduling and execution, you can quickly expand your background worker count to match demand. You can scale your task scheduler to no more than one process if feasible for simplicity's purposes.
Otherwise, you may schedule the duplicate work unless you execute complicated locking logic.
Achieve Cloud Elasticity with Iron
Speak to us to find how you can achieve cloud elasticity with a serverless messaging queue and background task solution with free handheld support.
Simple Job Scheduling
A basic period such as daily, hourly, or every 10 minutes may be used to execute a task in the Scheduler add-on as a job scheduler.
Instead of providing simple execution intervals, such as "every two hours," "every three hours," or "every five seconds," it is preferable to use a custom clock process to run tasks with specified schedules.
Custom Clock Processes
Except for providing a custom schedule, a clock process has the advantage of being specified as part of the process model, which is consistent with the rest of the logical application components. This facilitates testing, eliminates additional dependencies, and ensures that the environment parity between development and production are more similar.
Defining custom clock processes
The implementations of custom clocks differ significantly per language. However, creating a clock process as part of an application's process model is very easy. This is a typical Node.js application's Procfile.
web: node web.js
worker: node worker.js
clock: node clock.js
The components of clock.js are useless conceptually. What is critical is that the clock.js process is primarily responsible for deciding which tasks should be performed at what intervals and scheduling them to run in the background.
The background worker created in worker.js is then capable of receiving and performing the scheduled task instantly.
Clock Processes on Heroku
As stated earlier, the clock component should be a single process to prevent duplicate scheduling tasks and requiring complex locking logic. You can Scale the clock operation to a single dyno once pushed to Heroku.
$ heroku ps:scale clock=1
Scaling 'clock' processes... done, now running 1
The clock process must be scaled up and running to identify appropriate times to schedule a task. Billing hours accumulate on the dyno continually as there is a running clock process, not only when it is time to schedule a task.
Additional dynos may be required for the worker process if the planned tasks need a significant increase in processing. At a minimum, one worker dyno must be operating to receive and execute scheduled tasks.
Iron.io Serverless Tools
Speak to us to learn how IronWorker and IronMQ are essential products for your application to become cloud elastic.
Thanks for subscribing! Please check your email for further instructions.
Add-ons
Heroku add-ons are components that provide additional functionality for your applications, such as data storage, management, statistics, and data processing. These are entirely managed for you by a third-party supplier like Heroku. Add-ons exist to free developers from the extra difficulty of maintaining supporting services operating at total capacity. IronMQ is a critical add-on that customers use for enormously scalable and hosted message queues. It simplifies data and event management inside your application by supporting numerous clients and a managed environment that relieves you of server management responsibilities, enabling you to concentrate on your application instead.
The Iron.io add-on is intended to address the issue of growing complexity in applications. As individuals develop more and more applications, the reaction times and connections need more processing. This is one reason Iron.io created a secure method to speed up that procedure. The add on is one of the quickest choices for interacting between components and services. It’s cloud-based and flexible, to guarantee it will suit your business at any size.
IronMQ Configuration on Heroku.
- Download the IronMQ Heroku addon.
heroku addons:create iron_mq:developer
-----> Adding iron_mq to cold-winter-5462... done, v10 (free)
- Include the IronMQ client library in your installation (Ruby Example)
gem install iron_mq
- Configuration: obtaining the Token and the Project ID
heroku config | grep IRON
IRON_WORKER_PROJECT_ID => 123456789
IRON_WORKER_TOKEN => aslkdjflaksuilaks
- Interact with your queues using the installed client library (create a queue, post/get/delete a message, etc.). The client library works as a wrapper over the rest of API.
Language Support
IronMQ supports a wide variety of programming languages and you can always utilise the REST API (or build your own!). This implies that your current Heroku stack should continue to function normally.
Ruby
For development reasons, we're going to need to install the Ruby gem:
$ gem install iron_mq
Fetching: iron_mq-3.0.2.gem (100%)
Successfully installed iron_mq-3.0.2
If you're developing a Rails application or something else that makes use of Bundler, also include following in your Gemfile:
gem 'iron_mq'
Now that you've created a basic helper, you may communicate with your queues:
# Create an IronMQ::Client object
@ironmq = IronMQ::Client.new()
# Get a queue (if it doesn't exist, it will be created when you first post a message)
@queue = @ironmq.queue("my_queue")
# Post a message
@queue.post("hello world!")
# Get a message
msg = @queue.get()
p msg
# Delete a message (you must delete a message when you're done with it or it will go back on the queue after a timeout)
msg.delete # or @queue.delete(msg.id)
Advantages of Using IronMQ
Rapid Configuration
IronMQ enables you to configure your message queue in a matter of minutes. You can configure, install, and begin your queues!
You just have to simply connect to IronMQ endpoints to get access to unlimited message queueing instantly.
Integrate Easily With Your Program
Utilize an extensive collection of IronMQ language libraries, including Ruby, PHP, Python, Java,.NET, and Go. IronMQ understands your native language.
Credible Queues
IronMQ ensures that your queues are durable and fault-tolerant, that they are elastic, and that you may use queues as a system monitoring tool.
No Additional Code To Understand
With IronMQ, you can simply set up individual queues (e.g., pull, push, or long polling), your queues will run at peak efficiency, and you won't need to write any more code to implement your queueing technique.
Tailored Solutions
If you need a customized solution for your application's message queue, you can contact Iron's support team, and they will create a solution tailored to your specific needs.
Trial Period
IronMQ is available for a 14-day free trial. The free trial provides access to our services, including up to 10 million API queries, infinite queues, and phone/chat/email support. When the trial time ends, your queues will be automatically paused unless you upgrade your subscription.
No Server Administration
Developing and managing servers is never a one-off task. IronMQ enables you to control traffic and system load increases without needing to maintain infrastructure. IronMQ releases you from the backend, allowing you to focus on front-end development.
Robust & Secure
IronMQ incorporates message durability and duplication from the outset, ensuring that your queues are highly available and durable. HTTPS and SSL offer a secure gateway and a user-friendly REST API, while OAuth enables robust, scalable authentication.
Maximize Reliability
Dedicated clusters ensure that your service level criteria are fulfilled, that your queues are as reliable, and that your queues are as quick as the cloud.
Maximize Availability
IronMQ's global availability zones and automatic region failover ensure that your queues are always available and that disruptions do not impact you.