Blog

Search This Blog

Loading...

Monday, September 30, 2013

IronCast 3: How to rapidly prototype locally with IronWorker - IronWorker 101 Part 3/4

In a series of four IronCasts, we will provide a high-level introduction to using IronWorker, an easy-to-use scalable task queue. In this series, we will be using an example application which is written in Rails. However, the same concept applies to every language or framework.

In this video, we will show you how to prototype and develop your IronWorker locally before sending it to the cloud.



We don't want to have to keep uploading our code to the cloud to test it. We want to be able to test it locally.

Essentially, there are 2 things that have changed between local environment and the cloud environment. 


1. Payload

Your workers are given payload when run on our servers and therefore you will need a way to mock that locally. You could look for command line arguments that are only present when the code is run on the server, such as "-payload", or "-id"

Ruby Example:
require_relative "./development/pygments_worker_dev.rb" unless ARGV.include?("-id")
def database_config
  YAML.load(File.open(File.expand_path('../../../config/database.yml', __FILE__)))
end

def params
  {
    "database" => database_config["development"],
    "request" => {"lang" => "ruby", "code" => "def hello\n puts 'hello'\n end"},
    "snippet_id" => 1
  }
end
Python Example:
def setup_env():
  for i in range(len(sys.argv)]:
    if sys.argv[i] == '-id':
      return
  # Do all your environment setup stuff here

2. Folder structure: 

This is the code package that has been uploaded to the cloud. And now compare that with the original repo. You will see that the folder structure has changed. For example, in the models folder was originally nested inside the app folder and now in the worker code package, it is on the root directory of your worker. 

To address this problem, you can specify you desired folder structure on the cloud using optional parameters in your Worker file. (See documentation). Or alternatively, in Ruby, you can try to change the LOADPATH of how you load files locally.

Regardless of the language or framework that you are using, the basic concept remains the same, you should aim to be able to always test your code locally. This saves you time continuously sending your code to the server to test.