Blog

Search This Blog

Loading...

Friday, January 17, 2014

Iron.io Drinkup in NYC

In a couple short weeks, Iron.io will be hosting an Iron.io Drinkup and would love to see developers in NYC there!


Who: One of our dev evangelists and YOU!
What: Beer and...more beer!
Why: To asynchronously chat and hangout
     ...and drink beers

Where: TBD – but definitely in the city
When: Thurs, Jan 30th – 8pm to whenever

Let us know you're coming here:
     http://iron.eventbrite.com


Thursday, January 16, 2014

IronHack Deep Dive: Iron TicTacToe

This past week we crowned a winner for our Iron Holiday Hack. It was a one week virtual hackathon where participants built apps that were useful, beautiful, informative, or just plain cool. 

David Jones built an awesome little project called Iron TicTacToe that hit the mark on the goals of the challenge. The application was very simple but focused on a complex algorithmic concept of game trees, a directed graph whose nodes are positions in a game and whose edges are moves.


"David was able to take a complex theory and create a very simple application. He integrated multiple API's and focused on a concept that is used in many different applications"
– Chad Arimura, CEO, Iron.io



Courtesy of Wikipedia


About the Winning Application

So, how does a game tree work? Take a look at the image on the right and you should get a good sense. In fact, Tic-Tac-Toe is often used to teach game tree search algorithms in undergraduate computer science classes.

David took this exercise one step further and harmoniously implemented all 3 Iron products, as well as SendGrid, to build his application.



"My goal was to solve an interesting problem that takes advantage of parallel execution, with a large number of IronWorkers performing tasks that are small pieces of a larger puzzle. The IronCache is used to keep track of intermediate results and IronMQ are used to keep track of tasks that will be executed later."


So, each product was tasked with taking on a different role – similar to service oriented architecture. But let's dig into the code a little more:


"The parallel algorithm has 2 main parts. The first part expands the game tree. The second part passes the results from finished games (at the leaves) back up the tree towards the root. In the middle of the game tree search, some IronWorkers are busy expanding one part of the game tree while other IronWorkers are busy passing results up another part of the game tree. 

The algorithm finishes when all the results are reported back to the root, and an email is sent with the final solution."


More Information on the App

There's more to the project than what we can list here but you can find a lot more information about it on the HackerLeague project page.


Iron TicTacToe


if you're interested in expanding on David's project, here are three suggestions he's provided:

  1. Taking into account the symmetries of tic-tac-toe boards, a large number of tic-tac-toe boards are actually "equivalent". By being a little smarter about how tic-tac-toe boards are represented in the IronCache, the size of the game tree (and the amount of work) could be significantly reduced.
  2. In serial game tree search, a popular optimization is Alpha-Beta pruning. It would be interesting to implement parallel Alpha-Beta pruning, to further reduce the amount of work.
  3. Tic-tac-toe is a simple game, and useful when the goal is understanding the game tree search algorithm. Once the algorithm is implemented, it would be interesting to generalize it to more complex games, such as checkers, chess, or go.


About the FIrst Prize Winner

David Jones is a resident of Ontario, Canada and when not hacking he’s keeping busy as a computational expert, entrepreneur, visual neuroscientist, and algorithmic artist.




Thank You to All

Again, thanks to all participants, New RelicSendGridGitHub, and Twilio! It was truly a great hackathon and we wish everyone a Hacky New Year!!


Need a Mobile Backend? Try OpenShift. (SendGrid article)

SendGrid on Mobile Backend Services
SendGrid put out a nice post on their blog the other day on what to look for with mobile backends.

Kunal Batra is a developer evangelist at-large for SendGrid and did some work with Redhat’s OpenShift platform recently.

In the post, Kunal lays out a simple case for why this platform plus services such as Iron.io, SendGrid, Twilio, and PubNub can make for a compelling mobile backend service.

Here's a few of the requirements that Kunal lists as important.

IronMQ is listed in the message queue section as the primary option.
  • Storage of Custom Objects
  • Spatial/Geo Support
  • Real-time Push Notifications
  • Message Queues
  • Communication Outside the App

Distributed Systems Point the Way
The movement away from developing a monolithic app to service-driven approaches is already happening. The emergence of mobile as a dominant client interface is only quickening the pace. 

Modern application – and mobile apps – in particular need to be built from the ground up as distributed asynchronous systems. IronMQ (and IronWorker) can serve as core components in these types of architectures, handling the stream of incoming events and processing all the work that these events trigger.

Kunal's post is a good read on this shift in mobile (and all app) development. Connect with us if you'd like to talk more about building distributed high-scale systems. It's a topic near to our hearts.

Tuesday, January 7, 2014

Holiday Hack Winners!

After a great holiday season and a minor delay in judging, the results are in!

1st:  Iron TicTacToe
2nd: HackerNews Sentiment Analysis
3rd: Iron Unfollow

Winner: Iron TicTacToe
Built by: David Jones

Simple hackathon project to implement "parallel game tree search" for TicTacToe using IronWorker, IronCache, IronMQ, and SendGrid.

The parallel algorithm has 2 main parts. The first part expands the game tree. The second part passes the results from finished games (at the leaves) back up the tree towards the root. In the middle of the game tree search, some IronWorkers are busy expanding one part of the game tree while other IronWorkers are busy passing results up another part of the game tree. The algorithm finishes when all the results are reported back to the root, and an email is sent with the final solution.

Try it out here: http://voxica.net/iron-tictactoe/


2nd Place: HN Sentiment Analysis
Built by: Martin Gingras

Searches Hacker News (https://news.ycombinator.com/news) for posts with a keyword that you enter, parses the text of post(if any) and comments on the post then analyzes them for positive/negative keywords to try to get a grasp of the communities sentiment towards a certain topic.

Try it out here: http://hn-sentiment.herokuapp.com/


3rd Place: IronUnfollow
Built by: Matthias Sieber

The purpose of IronUnfollow is to declutter the user's Twitter account. My goal was to get down the num
ber of people I follow down significantly. Before I started writing this useful script, I was following over 2,000 people. Now I'm following less than 150 actual people and can actually see some meaningful and interesting tweets that make me want to interact with my following.

Try it out here: https://github.com/manonthemat/IronUnfollow


Honorable Mention: Telephone-Pictionary 
Built By: Feather Knee

An online game of Telephone Pictionary. A player is texted by the previous player and prompted to submit a phrase or a drawing based on what the previous player wrote or said.

Try it out here: http://telephone-pictionary.herokuapp.com/


Congratulations to all and thanks again to our partners New Relic, SendGrid, GitHub, and Twilio!



Push Queues + Error Queues = Better Queue Processing

IronMQ has had push queue capabilities for a while now. Push queues allow messages/payloads to be pushed to one or more endpoints and is a very popular feature. Within IronMQ push queues, up to 100 subscribers can be added – with the routing being either unicast (pushing to 1 subscriber) or multi-cast (pushing to all subscribers).

Push queues eliminate the need to poll as well as provide access to messages/payloads more promptly. (As soon as the message comes in, it gets distributed to one or more subscribers.)

To make push queues even more effective for transaction processing and other message handling needs, we recently added an error queue capability to IronMQ. If error queues are enabled, messages that fail – after a set number of retries – will get put onto an error queue. This makes it so you can easily recover if the subscribers are not available or under significant load.

Push Queues + Error Queues Improve Message Processing

Retries Help...

Retries help...
With IronMQ, retries can be applied and so if a push fails, it’ll retry after a certain delay for up to n times (where n is set by the developer). A problem arises, however, if all the retries fails. You certainly don’t want lose messages and so with error queues, messages that fail on the push will be placed on an error queue.


... But Error Queues Pick Up after Retries Fail

Once a message hits an error queue, it can be dealt with as appropriate. The common approach with an error queue is to poll it on a periodic basis and run though a set of exception steps. This can include a combination of logging exceptions and possibly retrying (if the case of network outages or possibly api-related issues).
...but errors still happen

Error messaging can also include a completely different set of processing steps. Additionally, error queues could be turned into push queues but beware, because that can introduce the potential for dropping messages if pushes fail within that push queue.


Creating an Error Queue

Error queues are off by default. To create an error queue, simply post to your push queue a message with the error_queue option defined and a queue name provided.

{"push_type":"unicast", "subscribers": [ {"url": "http://thiswebsitewillthrowanerror.com"} ], "error_queue": "MY_EXAMPLE_ERROR_QUEUE"}

If a push queue is set with the error_queue parameter, then after the set number of retries, a message will be put in the named error queue and viewable via your account dashboard. By default the error queue will be a pull queue.

NOTE: An error queue will not appear in your dashboard until an initial error message is received.

The Error queue message will contain the following information:

You can look up the original message if needed via the GET message endpoint using the source_msg_id value.

To turn off/disable an error queue, just post to your push queue set the error queue option to an empty string e.g. "error_queue": "".

{"push_type":"unicast", "subscribers": [ {"url": "http://thiswebsitewillthrowanerror.com"} ], "error_queue": ""}

NOTE: Omitting the “error_queue” option does not disable the error queue. You need to send it an empty string.

Push Queues + Error Queues = Better Processing

Adding error queues is a great feature that we're happy to have invested in. Now, you'll never have to worry about trying to find a message that went AWOL as it'll be right in your error queue. Let us know what you think.


Update: No post on errors and queues would be complete without a video of a falling penguin. (It’s better with the sound on.)

Monday, January 6, 2014

A Reminder App using Ruby, Google Docs, SendGrid, and Iron.io (SendGrid repost)

Nick Quinlan from SendGrid put out a nice post the other day on a simple app he created to email him daily goals. It's a short post but with some solid code behind it which adds a lot of value in our minds. (It's one thing talk about tech but it's another to turn it into a code example.) Thank you for that, Nick.
SendGrid Code Example using Iron.io
Email Your Goals Using Ruby, Google Docs, SendGrid, and Iron.io 
The hack, which I’m calling Daily Goals, emails you every day with a selection of your goals for personal growth. It’s written in Ruby and relies on Iron.io to run it every day. 
All the code you need to get this working is on GitHub. I’ll highlight the key lines with links below. 
I decided to keep my goals in a spreadsheet in Google Drive so that I can easily change them as I need to. Luckily, there’s a gem to interact with the service. Using this gem we can log into Google, grab the spreadsheet, and read the list of goals. From there we select a random goal from each category. Once we have our goals from Google, all that’s left is to throw it into a nice email and send it off with SendGrid.
Read more >> 
About the Developer/Author 
Nick Quinlan is a SendGrid Developer Evangelist based out of San Francisco. He works to make developers lives easier by providing them with the help they need to grow their ideas. Give him a shout, @YayNickQ.

UPDATE: Nick will be one of the featured speakers at the January SFRails meetup that Iron.io organizes. He'll be talking about email best practices. It's a great group with some terrific speakers and so stop by if you're in the neighborhood.

Friday, January 3, 2014

Queuing and Processing Your Way To Scalability using Go and Iron.io (reposted)


We came across another great post by William Kennedy on running scaling out applications using Iron.io platform and Go programming language. He wrote an excellent post on using Go with IronWorker several months back.

William writes Going Go Programming and so he certainly knows what he's talking about when it comes to Go. We're big fans of William and his blog – maybe not so surprising since we're both fans of the Go programming language (you can hear about our experiences with go here, here, and here).

William Kennedy
Ardan Studios
The team here at Iron.io can talk at length about IronWorker and its ability to run dynamic languages such as Ruby, Python, Java, Python, Node.js, or .NET as well as binaries and compiled languages like Go. But we do that a lot and so it's nice to get contributions from people like William, especially when they're so well written and detailed.

––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
Queue Your Way to Scalability
William Kennedy

Introduction

Going Go: Queue Your Way to Scalability
The first thing I did when I started programming in Go was begin porting my Windows utilities classes and service frameworks over to Linux. This is what I did when I moved from C++ to C#. Thank goodness, I soon learned about Iron.io and the services they offered. Then it hit me, if I wanted true scalability, I needed to start building worker tasks that could be queued to run anywhere at any time. It was not about how many machines I needed, it was about how much compute time I needed.

Outcast Marine Forecast

The freedom that comes with architecting a solution around web services and worker tasks is refreshing. If I need 1,000 instances of a task to run, I can just queue it up. I don't need to worry about capacity, resources, or any other IT related issues. If my service becomes an instant hit overnight, the architecture is ready, the capacity is available.

My mobile weather application Outcast is a prime example. I currently have a single scheduled task that runs in Iron.io every 10 minutes. This task updates marine forecast areas for the United States and downloads and parses 472 web pages from the NOAA website. We are about to add Canada and eventually we want to move into Europe and Australia. At that point a single scheduled task is not a scalable or redundant architecture for this process.

Thanks to the Go Client from Iron.io, I can build a task that wakes up on a schedule and queues up as many marine forecast area worker tasks as needed. I can use this architecture to process each marine forecast area independently, in their own worker task, providing incredible scalability and redundancy. The best part, I don't have to think about hardware or IT related capacity issues.