There are numerous options for performing background processing in Rails.

• Spawn
• BackgrounDRB
• Backgroundjob

Here at Howcast, our method of choice is Backgroundjob (Bj).

“Backgroundjob (Bj) is a brain dead simple zero admin background priority queue for Rails. Bj is robust, platform independent (including windows), and supports internal or external management of the background runner process.”

Installing Bj

1. ./script/plugin install http://codeforpeople.rubyforge.org/svn/rails/plugins/bj
2. ./script/bj setup

This will create all the migrations you need to generate the job tables (note that there are also archive and configuration tables).

Distributing Bj

With Bj there is a persistent job queue in the database that workers can query to pick up pending jobs. This allows Bj to be easily distributed where workers can be run across various servers.

In your environment.rb simply add:

This will mean that the default Bj worker will not run on the web server. This means you can run various other workers on other servers/slices. Simply add this to the crontabs of the servers you want to distribute to.

Extending Bj

Although Bj is a full fledged solution for managing and running background processes, we needed some additional functionality on Howcast — dependency jobs and specialized workers. We needed some jobs to be run serially after one another and so we needed to specify a dependency job id that would need to be completed before the job submitted would be started. With this requirement also came another requirement of constraining specific workers to run specific types of jobs. To accomplish this we added a dependency_id to the bj_jobs table and added an option for running job works with specific tags (–only-tag parameter). This would allow you to run a worker with the following command:

This would cause this worker to only run jobs that were submitted with the ’specialized’ tag:

There is also an option to start a worker with –exclude-tag option to do the reverse of the example above.

With these two features combined you can create a pretty complex flow structure for jobs. Now you can split up larger jobs into smaller specialized jobs that specific workers will run in parallel or serially with the dependency_id set.

Installing the enhancements

Both these enhancements have been open sourced and are available here: http://github.com/howcast/backgroundjob/commits/dependencies_and_tags

We’ve found this to be an easy way to distribute background processing tasks and think some of you might find it to be useful as well.

At Howcast, we have a small engineering team that tries to accomplish a lot! Check out our about page to learn a bit more about who we are.

If you’re working with video on the web, you’re most likely transcoding to Flash Video format. Whether you use FFmpeg, On2, or another transcoding package, if you are FLV streaming via lighttpd or nginx, you’ll need to add keyframe metadata to the FLV in order to allow your player to seek to different points in the video that have yet to be downloaded.

Being that our site is built with Ruby on Rails, it was natural for us to leverage flvtool2 (which is written in Ruby) to add this metadata. Lately, we’ve been having intermittent failures with flvtool2 that appear to be related to choppy audio.

Although the ideal way to deal with this issue is for us to fix the problem in flvtool2, given some of the other things that we’re working on we may not be able to look at this near term. One alternative that we’re starting to experiment with is using flvtool++.

We’re in the process of setting this up in a linux environment, but for people interested in trying it out on OS X (we’re using Leopard), here are some instructions on how to install for folks that use MacPorts:

1. Grab the latest http://mirror.facebook.com/facebook/flvtool++/ (thanks Facebook!)

2. sudo port install scons

3. sudo port install boost

4. tar xzvf flvtool++-1.1.tar.gz

5. cd [flvtool++ directory]

6. Add a build path to SConscript:

5. Manually create byteswap.h:

6. scons

7. And voila!

./flvtool++

flvtool++ 1.0
Copyright (c) 2007 Dan Weatherford and Facebook, inc.
http://developers.facebook.com/opensource.php
Published under the BSD license.

usage: flvtool++ [-nodump] [-nomerge] [-tag name value] [input filename] [output filename]
-nodump: do not dump the metadata when done (kinda quiet)
-nomerge: do not keep existing metadata from the input file
-strip: remove all metadata (do not write it to the output file)
-tag name value: Set a metadata tag named 'name' to the (string) value 'value'
Note that manually set tags will override automatically generated tags.

8. To add keyframe metadata to an FLV simply run a command like:

./flvtool++ in.flv out.flv

We hope this is helpful. We’ll keep you posted on how things go as we’re just beginning to experiment on linux.

The Howcast Gem is a pure Ruby wrapper for the Howcast API. It aims to encapsulate all the functionality of the Howcast API in pure Ruby — now your Ruby programs can interact with Howcast videos and wiki guides.

You Will Need

• Ruby Installed
• RubyGems Installed

Step 1: Get the Gem

Jump right in and get the code. No better way to learn than experimenting.

sudo gem install howcast-howcast

Step 2: Get Acquainted

Familiarize yourself with the RDoc for the gem. The README file is a great starting point and outlines the functionality you can achieve with the gem. By following the RDoc documentation and the Howcast API Documentation, you’ll quickly find that the gem is incredibly simple to use in your own applications.

Step 3: Get Help

We have a support group for this gem available at: http://groups.google.com/group/howcast-developers/topics. Please use this group and mailing list for all your questions and advice.

Step 4: Stay Tuned

Join our Google Group to receive updates about new features in our API. Also subscribe to this blog to get announcements in your RSS reader about things going on at tech.Howcast.

Step 5: Requests

Something missing from the gem? Other interfaces into Howcast you would like to use? Please let us know by posting on our discussion boards. We will be checking these regularly, responding to your feedback and questions.

The Howcast Google Gadget and the Know How To? Facebook app were both developed using the gem.

For the past several months, we’ve been working hard on getting howcast.com up and running and launching new and improved versions of our product. Along the way we’ve learned a lot! We’d like to share some of that with you and hope that it’ll save you some time and provide you with some solution options for different projects. You’ll be seeing us post about different engineering problems that we’ve solved (Ruby, Rails, Flash, and Flex), how we use different commercial and open source packages and tools, and some general thoughts on different topics from time to time.

To get things started we’d like to announce the Howcast API v0.1! We are starting off small and providing a simple REST API to read video and wiki guide data. You can use the Howcast API to get lists of videos/guides with various filters; search for videos and guides, and retrieve information about a particular video or wiki guide.

For a full list of supported APIs, please refer to our API Documentation.

We are currently in active development of the Howcast API. To stay up to date with new features, or to share comments and ideas on how to make the API better, please join the discussions on our Google Group. We will notify members of the group as we release changes or enhancements.

Our Google gadget and Facebook application both use the API and we hope you find it useful as well! If you develop an application that leverages the API, please be sure email us at api+apps@howcast.com. We may feature your application on howcast.com!