Recently we migrated our source to git, and in the process became fans of github. We had been using Trac as our bug tracking software of choice, and it integrated smoothly with Subversion. Unfortunately, the git support was a bit lacking. There were a couple of plugins out there, but they didn’t really suit our needs. Fortunately, we found a better solution.

Enter Lighthouse. The guys at ActiveReload have designed a simple hosted issue and project management application that is easy on the eyes and plays great with github. We decided that a switch was in order. Since there’s currently no bulk import built into Lighthouse, we needed to migrate over a year’s worth of data. Luckily they offer an API and even a Ruby wrapper, which several folks have used to roll their own importers.

We first attempted to use Shay Arnett’s cool trachouse importer, but ran into some issues with our Trac authentication scheme and plugin setup. Trachouse fetches data from a Trac install by spidering your Trac website using hpricot. Since we were hosting our own Trac install, we figured it might be easier to just parse Trac’s sqlite database.

We put together a quick script to parse the trac.db and import it into Lighthouse using the lighthouse-api Ruby wrapper, and we (not so) cleverly dubbed it “Traclight”. We thought others might find this handy, and so there’s now a repo for it over at github.

To use it, you will need to edit the traclight.rb script and add your own conversion information, so the script will know how to translate Trac users, milestones, and states to the Lighthouse equivalents. You should create any user accounts and milestones ahead of time.

Next, export your trac database. See the trac FAQ for more information.

Now you are ready to run the script:

traclight.rb PATH_TO_TRAC_DB_FILE

There are a couple of caveats:

• Due to Lighthouse API limitations, ticket comments will be merged into a formatted description of the ticket.
• The tickets will show up as being reported by the owner of the account you specify in the ticket, not the original reporter. The original Trac reporter will show up in the description for the ticket.
• It is not possible to set the original date the ticket was reported in Lighthouse. The original reporting date of the Trac ticket will appear in the Lighthouse ticket description.
• Attachments are _not_ imported, as support currently doesn’t exist in the API.
• Starting with an empty Lighthouse project is recommended, since traclight will import the tickets in sequential order. That means that your ticket numbers should match up (as long as you never deleted tickets from Trac).

We used this to import around 5000 tickets spread over 15 milestones into Lighthouse using Traclight. Perhaps it’ll be of use to your team.

There are a plethora of ways to transcode your videos into a podcast-compatible format. We wanted to give you a quick walkthrough of how we use FFmpeg and AtomicParsley to do it.

First things first — check out Apple’s technical specifications to understand more about the podcast feed format and iPod-compatible video specifications. This post is going to focus on creating 320×240 videos in M4V format (which is really MPEG-4 video format) with a single audio channel. Why not 640×480? There’s some additional work that you have to do to update the M4V metadata so that the video will sync properly to video iPods and we’re still experimenting with it. Why a single channel of audio? Technically we should be able to do things in stereo (two channels of audio), but we ran into issues in production with the audio cutting out midway through some of our videos, and using a single channel of audio seemed to resolve the problem. You may not run into this issue so we’ll point out where to tweak the transcoding recipe so that you can use one or two channels.

The rest of this post assumes that you have the following software installed:

• FFmpeg - Open source video transcoding tool
• AtomicParsley - Open source tool for manipulating metadata in MPEG-4 videos

If you have a Mac and use MacPorts, here’s the easy/lazy way to do it:

• sudo port install ffmpeg +lame +libogg +vorbis +faac +faad +xvid +libx264 +a52
• sudo port install AtomicParsley

If you prefer to download and compile the packages, there are a number of useful resources to help guide you based on the operating system you are using — so we’re going to skip diving into additional detail there.

Step 1 - Transcode the video

We basically follow the recipe in Robert Swain’s post:

ffmpeg -i [input file] -f mp4 -acodec libfaac -ab 128k -ac 1 -vcodec libx264 -b 768k -flags +loop -cmp +chroma -partitions +parti4x4+partp8x8+partb8x8 -flags2 +mixed_refs -me umh -subq 5 -trellis 1 -refs 5 -coder 0 -me_range 16 -g 250 -keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 -maxrate 768k -bufsize 2M -rc_eq 'blurCplx^(1-qComp)' -qcomp 0.6 -qmin 10 -qmax 51 -qdiff 4 -level 13 -s 320x240 -y [output file].m4v

A couple notes here:

• If you use a Mac and are on Tiger vs. Leopard, you’ll need to use x264 versus libx264 for the -vcodec value
• Change -ac 1 to -ac 2 if you want to try with 2 audio channels

Step 2 - Embed an image in the video’s metadata

Want a specific image to appear when your podcast videos display in coverflow?

Here’s where AtomicParsley works its magic.

AtomicParsley [output file].m4v --artwork [image] --overWrite

Note that per the Apple specs, you should use a 600×600 jpg or png image.

That’s all there is to it. Double-click on your M4V file and it should play in iTunes. Sync to your iPod and enjoy your video!

There are numerous options for performing background processing in Rails.

• Spawn
• BackgrounDRB
• Backgroundjob

Here at Howcast, our method of choice is Backgroundjob (Bj).

“Backgroundjob (Bj) is a brain dead simple zero admin background priority queue for Rails. Bj is robust, platform independent (including windows), and supports internal or external management of the background runner process.”

Installing Bj

1. ./script/plugin install http://codeforpeople.rubyforge.org/svn/rails/plugins/bj
2. ./script/bj setup

This will create all the migrations you need to generate the job tables (note that there are also archive and configuration tables).

Distributing Bj

With Bj there is a persistent job queue in the database that workers can query to pick up pending jobs. This allows Bj to be easily distributed where workers can be run across various servers.

In your environment.rb simply add:

This will mean that the default Bj worker will not run on the web server. This means you can run various other workers on other servers/slices. Simply add this to the crontabs of the servers you want to distribute to.

Extending Bj

Although Bj is a full fledged solution for managing and running background processes, we needed some additional functionality on Howcast — dependency jobs and specialized workers. We needed some jobs to be run serially after one another and so we needed to specify a dependency job id that would need to be completed before the job submitted would be started. With this requirement also came another requirement of constraining specific workers to run specific types of jobs. To accomplish this we added a dependency_id to the bj_jobs table and added an option for running job works with specific tags (–only-tag parameter). This would allow you to run a worker with the following command:

This would cause this worker to only run jobs that were submitted with the ’specialized’ tag:

There is also an option to start a worker with –exclude-tag option to do the reverse of the example above.

With these two features combined you can create a pretty complex flow structure for jobs. Now you can split up larger jobs into smaller specialized jobs that specific workers will run in parallel or serially with the dependency_id set.

Installing the enhancements

Both these enhancements have been open sourced and are available here: http://github.com/howcast/backgroundjob/commits/dependencies_and_tags

We’ve found this to be an easy way to distribute background processing tasks and think some of you might find it to be useful as well.

At Howcast, we have a small engineering team that tries to accomplish a lot! Check out our about page to learn a bit more about who we are.

If you’re working with video on the web, you’re most likely transcoding to Flash Video format. Whether you use FFmpeg, On2, or another transcoding package, if you are FLV streaming via lighttpd or nginx, you’ll need to add keyframe metadata to the FLV in order to allow your player to seek to different points in the video that have yet to be downloaded.

Being that our site is built with Ruby on Rails, it was natural for us to leverage flvtool2 (which is written in Ruby) to add this metadata. Lately, we’ve been having intermittent failures with flvtool2 that appear to be related to choppy audio.

Although the ideal way to deal with this issue is for us to fix the problem in flvtool2, given some of the other things that we’re working on we may not be able to look at this near term. One alternative that we’re starting to experiment with is using flvtool++.

We’re in the process of setting this up in a linux environment, but for people interested in trying it out on OS X (we’re using Leopard), here are some instructions on how to install for folks that use MacPorts:

1. Grab the latest http://mirror.facebook.com/facebook/flvtool++/ (thanks Facebook!)

2. sudo port install scons

3. sudo port install boost

4. tar xzvf flvtool++-1.1.tar.gz

5. cd [flvtool++ directory]

6. Add a build path to SConscript:

5. Manually create byteswap.h:

6. scons

7. And voila!

./flvtool++

flvtool++ 1.0
Copyright (c) 2007 Dan Weatherford and Facebook, inc.
http://developers.facebook.com/opensource.php
Published under the BSD license.

usage: flvtool++ [-nodump] [-nomerge] [-tag name value] [input filename] [output filename]
-nodump: do not dump the metadata when done (kinda quiet)
-nomerge: do not keep existing metadata from the input file
-strip: remove all metadata (do not write it to the output file)
-tag name value: Set a metadata tag named 'name' to the (string) value 'value'
Note that manually set tags will override automatically generated tags.

8. To add keyframe metadata to an FLV simply run a command like:

./flvtool++ in.flv out.flv

We hope this is helpful. We’ll keep you posted on how things go as we’re just beginning to experiment on linux.

The Howcast Gem is a pure Ruby wrapper for the Howcast API. It aims to encapsulate all the functionality of the Howcast API in pure Ruby — now your Ruby programs can interact with Howcast videos and wiki guides.

You Will Need

• Ruby Installed
• RubyGems Installed

Step 1: Get the Gem

Jump right in and get the code. No better way to learn than experimenting.

sudo gem install howcast

Step 2: Get Acquainted

Familiarize yourself with the RDoc for the gem. The README file is a great starting point and outlines the functionality you can achieve with the gem. By following the RDoc documentation and the Howcast API Documentation, you’ll quickly find that the gem is incredibly simple to use in your own applications.

Step 3: Get Help

We have a support group for this gem available at: http://groups.google.com/group/howcast-developers/topics. Please use this group and mailing list for all your questions and advice.

Step 4: Stay Tuned

Join our Google Group to receive updates about new features in our API. Also subscribe to this blog to get announcements in your RSS reader about things going on at tech.Howcast.

Step 5: Requests

Something missing from the gem? Other interfaces into Howcast you would like to use? Please let us know by posting on our discussion boards. We will be checking these regularly, responding to your feedback and questions.

The Howcast Google Gadget and the Know How To? Facebook app were both developed using the gem.

For the past several months, we’ve been working hard on getting howcast.com up and running and launching new and improved versions of our product. Along the way we’ve learned a lot! We’d like to share some of that with you and hope that it’ll save you some time and provide you with some solution options for different projects. You’ll be seeing us post about different engineering problems that we’ve solved (Ruby, Rails, Flash, and Flex), how we use different commercial and open source packages and tools, and some general thoughts on different topics from time to time.

To get things started we’d like to announce the Howcast API v0.1! We are starting off small and providing a simple REST API to read video and wiki guide data. You can use the Howcast API to get lists of videos/guides with various filters; search for videos and guides, and retrieve information about a particular video or wiki guide.

For a full list of supported APIs, please refer to our API Documentation.

We are currently in active development of the Howcast API. To stay up to date with new features, or to share comments and ideas on how to make the API better, please join the discussions on our Google Group. We will notify members of the group as we release changes or enhancements.

Our Google gadget and Facebook application both use the API and we hope you find it useful as well! If you develop an application that leverages the API, please be sure email us at api+apps@howcast.com. We may feature your application on howcast.com!