To help get people started using the cucumber framework we presented tonight at the SF Ruby meetup. Here is the presentation and you can find all of the source code here:

http://github.com/howcast/cucumber_101_app/tree/master

Hope that everyone found it helpful and please feel free to ping us if you have any questions!

Due to popular request we’ve now added Category support into our API.

If you haven’t already installed the Howcast Gem you can do so with: “sudo gem install howcast”

To get you up and running here is some sample code to interact with the Category taxonomy.

What no posts about video transcoding?! Shocker, we know. We are always working hard to improve howcast.com, and despite what the droughts in blog posts may suggest, we actually do like to do things that do not involve work. Recently, I completed my first triathlon (the Santa Cruz Sentinel) which was a really fun experience. I am a complete hack, but for those of you that may be on the fence about giving it a go, I wrote a wiki guide that might help convince you to make the jump:

http://www.howcast.com/guides/2297-How-To-Not-So-Seriously-Train-For-a-Triathlon

We recently had to transcode a large number of videos for the iPhone. Rather than try to scale out our in-house infrastructure, we thought it would be prudent to check out different projects people had been working on. One stellar example is Panda (whom our friend Ezra Zygmuntowicz at Engine Yard mentioned to us). Panda leverages Amazon’s S3, EC2, and Simple DB web services to offer a very flexible, cost effective, and scalable option for integrating video transcoding into any site. We’re likely going to do build something a bit more tailored for howcast.com, but will definitely try to see if there are things that we can contribute back to that project. We humbly tip our hats to the New Bamboo team for their fantastic work.

OK so we have admittedly just about worn out this topic, however, we do have some recipe improvements to share with you. We ran into some issues with our previous recipe for some videos on the iPhone. Here is our latest incantation which seems to work quite well:

ffmpeg -y -i [input file] -v 1 -threads auto -vcodec libx264 -b 300k -r 15 -bt 175k -refs 1 -loop 1 -deblockalpha 0 -deblockbeta 0 -parti4x4 1 -partp8x8 1 -me full -subq 6 -me_range 21 -chroma 1 -slice 2 -bf 0 -level 30 -g 300 -keyint_min 30 -sc_threshold 40 -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.7 -qmax 51 -qdiff 4 -i_qfactor 0.71428572 -maxrate 768k -bufsize 2M -cmp 1 -s 480x320 -acodec libfaac -profile aac_low -ab 64k -ar 22050 -ac 2 -f mp4 [output file]

Note that you may need to adjust the resolution as appropriate. With respect to what changed, you will notice is that we lowered both the video and audio bit rate, are explicitly using the AAC low complexity audio profile (per Apple’s specs), and have also switched back to using stereo (vs. mono) for the audio output. You will need to ensure that you have a fairly recent version of FFmpeg installed with H.264 and AAC. We hope that you find this helpful and as always feel free to ping us with any questions.

A post or so ago we talked about podcast transcoding. Since then we’ve updated things a bit to support 640×480 video. The first thing that we did was update the FFmpeg recipe we use for transcoding:

ffmpeg -y -i [input file] -v 1 -threads auto -vcodec libx264 -b 500k -bt 175k -refs 1 -loop 1 -deblockalpha 0 -deblockbeta 0 -parti4x4 1 -partp8x8 1 -me full -subq 6 -me_range 21 -chroma 1 -slice 2 -bf 0 -level 30 -g 300 -keyint_min 30 -sc_threshold 40 -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.7 -qmax 51 -qdiff 4 -i_qfactor 0.71428572 -maxrate 768k -bufsize 2M -cmp 1 -s 640x480 -acodec libfaac -ab 96 -ar 48000 -ac 1 -f mp4 [output file]

This recipe was based on this Ubuntu community documentation. Note that we’re doing 1 pass versus 2 pass encoding and are using a single audio channel (still having interimittent audio issues if we use two).

The next challenge was setting the atom in the video such that the videos would sync properly from iTunes to video iPods and iPhones. As pointed out in Robert Swain’s post and also in an Apple forum thread, the development version of AtomicParsley has a feature to do this. We had to make some minor code tweaks to get things to compile properly (the changes were slightly different on OS X and Linux), but feel free to ping us if you get stuck. Once you have AtomicParsley compiled, here’s the command to run to set the atom:

AtomicParsley [output file] --DeepScan --iPod-uuid 1200 --overWrite

After that you should be set! Thanks to all of the contributors to these fantastic open source projects and let us know if you have any questions.

Recently we migrated our source to git, and in the process became fans of github. We had been using Trac as our bug tracking software of choice, and it integrated smoothly with Subversion. Unfortunately, the git support was a bit lacking. There were a couple of plugins out there, but they didn’t really suit our needs. Fortunately, we found a better solution.

Enter Lighthouse. The guys at ActiveReload have designed a simple hosted issue and project management application that is easy on the eyes and plays great with github. We decided that a switch was in order. Since there’s currently no bulk import built into Lighthouse, we needed to migrate over a year’s worth of data. Luckily they offer an API and even a Ruby wrapper, which several folks have used to roll their own importers.

We first attempted to use Shay Arnett’s cool trachouse importer, but ran into some issues with our Trac authentication scheme and plugin setup. Trachouse fetches data from a Trac install by spidering your Trac website using hpricot. Since we were hosting our own Trac install, we figured it might be easier to just parse Trac’s sqlite database.

We put together a quick script to parse the trac.db and import it into Lighthouse using the lighthouse-api Ruby wrapper, and we (not so) cleverly dubbed it “Traclight”. We thought others might find this handy, and so there’s now a repo for it over at github.

To use it, you will need to edit the traclight.rb script and add your own conversion information, so the script will know how to translate Trac users, milestones, and states to the Lighthouse equivalents. You should create any user accounts and milestones ahead of time.

Next, export your trac database. See the trac FAQ for more information.

Now you are ready to run the script:

traclight.rb PATH_TO_TRAC_DB_FILE

There are a couple of caveats:

• Due to Lighthouse API limitations, ticket comments will be merged into a formatted description of the ticket.
• The tickets will show up as being reported by the owner of the account you specify in the ticket, not the original reporter. The original Trac reporter will show up in the description for the ticket.
• It is not possible to set the original date the ticket was reported in Lighthouse. The original reporting date of the Trac ticket will appear in the Lighthouse ticket description.
• Attachments are _not_ imported, as support currently doesn’t exist in the API.
• Starting with an empty Lighthouse project is recommended, since traclight will import the tickets in sequential order. That means that your ticket numbers should match up (as long as you never deleted tickets from Trac).

We used this to import around 5000 tickets spread over 15 milestones into Lighthouse using Traclight. Perhaps it’ll be of use to your team.

There are a plethora of ways to transcode your videos into a podcast-compatible format. We wanted to give you a quick walkthrough of how we use FFmpeg and AtomicParsley to do it.

First things first — check out Apple’s technical specifications to understand more about the podcast feed format and iPod-compatible video specifications. This post is going to focus on creating 320×240 videos in M4V format (which is really MPEG-4 video format) with a single audio channel. Why not 640×480? There’s some additional work that you have to do to update the M4V metadata so that the video will sync properly to video iPods and we’re still experimenting with it. Why a single channel of audio? Technically we should be able to do things in stereo (two channels of audio), but we ran into issues in production with the audio cutting out midway through some of our videos, and using a single channel of audio seemed to resolve the problem. You may not run into this issue so we’ll point out where to tweak the transcoding recipe so that you can use one or two channels.

The rest of this post assumes that you have the following software installed:

• FFmpeg - Open source video transcoding tool
• AtomicParsley - Open source tool for manipulating metadata in MPEG-4 videos

If you have a Mac and use MacPorts, here’s the easy/lazy way to do it:

• sudo port install ffmpeg +lame +libogg +vorbis +faac +faad +xvid +libx264 +a52
• sudo port install AtomicParsley

If you prefer to download and compile the packages, there are a number of useful resources to help guide you based on the operating system you are using — so we’re going to skip diving into additional detail there.

Step 1 - Transcode the video

We basically follow the recipe in Robert Swain’s post:

ffmpeg -i [input file] -f mp4 -acodec libfaac -ab 128k -ac 1 -vcodec libx264 -b 768k -flags +loop -cmp +chroma -partitions +parti4x4+partp8x8+partb8x8 -flags2 +mixed_refs -me umh -subq 5 -trellis 1 -refs 5 -coder 0 -me_range 16 -g 250 -keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 -maxrate 768k -bufsize 2M -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.6 -qmin 10 -qmax 51 -qdiff 4 -level 13 -s 320x240 -y [output file].m4v

A couple notes here:

• If you use a Mac and are on Tiger vs. Leopard, you’ll need to use x264 versus libx264 for the -vcodec value
• Change -ac 1 to -ac 2 if you want to try with 2 audio channels

Step 2 - Embed an image in the video’s metadata

Want a specific image to appear when your podcast videos display in coverflow?

Here’s where AtomicParsley works its magic.

AtomicParsley [output file].m4v --artwork [image] --overWrite

Note that per the Apple specs, you should use a 600×600 jpg or png image.

That’s all there is to it. Double-click on your M4V file and it should play in iTunes. Sync to your iPod and enjoy your video!

There are numerous options for performing background processing in Rails.

• Spawn
• BackgrounDRB
• Backgroundjob

Here at Howcast, our method of choice is Backgroundjob (Bj).

“Backgroundjob (Bj) is a brain dead simple zero admin background priority queue for Rails. Bj is robust, platform independent (including windows), and supports internal or external management of the background runner process.”

Installing Bj

1. ./script/plugin install http://codeforpeople.rubyforge.org/svn/rails/plugins/bj
2. ./script/bj setup

This will create all the migrations you need to generate the job tables (note that there are also archive and configuration tables).

Distributing Bj

With Bj there is a persistent job queue in the database that workers can query to pick up pending jobs. This allows Bj to be easily distributed where workers can be run across various servers.

In your environment.rb simply add:

This will mean that the default Bj worker will not run on the web server. This means you can run various other workers on other servers/slices. Simply add this to the crontabs of the servers you want to distribute to.

Extending Bj

Although Bj is a full fledged solution for managing and running background processes, we needed some additional functionality on Howcast — dependency jobs and specialized workers. We needed some jobs to be run serially after one another and so we needed to specify a dependency job id that would need to be completed before the job submitted would be started. With this requirement also came another requirement of constraining specific workers to run specific types of jobs. To accomplish this we added a dependency_id to the bj_jobs table and added an option for running job works with specific tags (–only-tag parameter). This would allow you to run a worker with the following command:

This would cause this worker to only run jobs that were submitted with the ’specialized’ tag:

There is also an option to start a worker with –exclude-tag option to do the reverse of the example above.

With these two features combined you can create a pretty complex flow structure for jobs. Now you can split up larger jobs into smaller specialized jobs that specific workers will run in parallel or serially with the dependency_id set.

Installing the enhancements

Both these enhancements have been open sourced and are available here: http://github.com/howcast/backgroundjob/commits/dependencies_and_tags

We’ve found this to be an easy way to distribute background processing tasks and think some of you might find it to be useful as well.

At Howcast, we have a small engineering team that tries to accomplish a lot! Check out our about page to learn a bit more about who we are.