I’m currently involved in a new project for a client that involves a lot of user contributed video, that then gets viewed on the site. This means video encoding, and lots of it, and the budget and timeline involved mean that the client can’t afford to implement the dedicated hardware to do this themselves. Why not encode in the cloud you say? Why not.
First off I've got to say one thing: i usually don’t rave on about a new found internet widget, but things like this excite me. The internet has just gotten cooler, and “cloud services” are all the rage, but not very often is something that comes a long that seems to be one of those things that make your say:
“Why was it not always like this?”
Background
So as the project is currently still in development i am unable to name names, however i can happily talk about the premise.
The basic user flow goes something like this:
- Users upload their video
- Video is encoded into FLV
- Video is approved by an admin and streamed on the site for users to view using a CDN
As you can see from the above, this site, like YouTube, is going to be having to do a lot of number crunching in the background so that it can encode the submitted videos (potentially in a bunch of different allowed formats) into an FLV for streaming.
Add to that all of the drama that people who’ve encoded video before can vouch for, in the form of many headache inducing issue, the least of which:
- Updating encoding codec's and paying for the non-free ones
- Dealing with spreading load/splitting the video into parts to deliver an encoded video as fast as possible
Welcome to Encoding in the cloud
As i am a big supporter of the write once philosophy, i’m trying to not reinvent the wheel. I was very excited to learn that cloud encoding is offered by a few companies, each with varying levels of user experience (from ready users reviews on the interwebs). Encoding.com, FlixCloud, and Hey!Watch are just a few.
In my case i decided on using Encoding.com as they seem to offer the least friction when it comes to getting up and running.
They offer a free monthly trial to test them out with, so it is easy to run a quick proof of concept (like i did). Their pricing starts at US$20 a month, which is peanuts, in comparison to the time you’ll save setting up and maintaining an encoding farm. As well they offer other epically cool features like a fully documented developer XML API. Did i mention they encode into all the cool formats?
OOOOHHHHH… AHHHHHHHH…
This allows them to support having your application let them know to come fetch a video, encode it, send it to some FTP and let you know by a call-back when its complete. Pretty sweet all up.
Example Encoding.com API request
<?xml version="1.0"?> <query> <!-- Main fields --> <userid>[UserID]</userid> <userkey>[UserKey]</userkey> <action>[Action]</action> <mediaid>[MediaID]</mediaid> <source>[SourceFile]</source> <notify>[NotifyURL]</notify> <format> <!-- Format fields --> <output>[Output format]</output> <video_codec>[Video Codec]</video_codec> <audio_codec>[Audio Codec]</audio_codec> <bitrate>[Video bitrate]</bitrate> <audio_bitrate>[Audio bitrate]</audio_bitrate> <audio_sample_rate>[Audio quality]</audio_sample_rate> <audio_volume>[Volume]</audio_volume> <size>[Size]</size> <crop_left>[Crop Left]</crop_left> <crop_top>[Crop Top]</crop_top> <crop_right>[Crop Right]</crop_right> <crop_bottom>[Crop Bottom]</crop_bottom> <keep_aspect_ratio>[yes/no]</keep_aspect_ratio> <thumb_time>[Thumb time]</thumb_time> <thumb_size>[Thumb size]</thumb_size> <add_meta>[yes/no]</add_meta> <rc_init_occupancy>[RC Occupancy]</rc_init_occupancy> <minrate>[Min Rate]</minrate> <maxrate>[Max Rate]</maxrate> <bufsize>[RC Buffer Size]</bufsize> <keyframe>[Keyframe Period (GOP)]</keyframe> <start>[Start From]</start> <duration>[Result Duration]</duration> <!-- Destination fields --> <destination>[DestFile]</destination> <thumb_destination>[Thumb Dest]</thumb_destination> <!-- Logo fields (OPTIONAL) --> <logo> <logo_source>[LogoURL]</logo_source> <logo_x>[LogoLeft]</logo_x> <logo_y>[LogoTop]</logo_y> <logo_mode>[LogoMode]</logo_mode> <logo_threshold>[LogoTreshold]</logo_threshold> </logo> <!-- Video codec parameters (OPTIONAL, while only for libx264 video codec) --> <video_codec_parameters> <!-- Allowed values: flv: flv, libx264, vp6 ** fl9: libx264 wmv, zune: wmv2, msmpeg4 3gp: h263, libx264 m4v: mpeg4 mp4, ipod, iphone, ipad, appletv, psp: mpeg4, libx264 mp3, wma: none mpeg2: mpeg2video Default: flv: flv fl9, ipod, iphone, ipad: libx264 wmv, zune: wmv2 3gp: h263 mp4, m4v,appletv, psp: mpeg4 mpeg2: mpeg2video --> </video_codec_parameters> <!-- Profile (OPTIONAL, while only for libx264 video codec) --> <pofile>[high/main/baseline]</pofile> <!-- Turbo Encoding switch (OPTIONAL) --> <turbo>[yes/no]</turbo> </format> </query>
But wait there’s more
And before you start to think I'm a fully paid Encoding.com representative, on pay-roll, i have to tell you about another thing that made me decide on them. Encoding SLA’s.
Encoding.com guarantee that your encoding jobs will be completed in a set amount of time, depending on your account level.
Have your cake and eat it to!
You won’t have to wait hours for your videos to encode (time to completion ranges between 30 mins and 4 mins depending on your account level), independent of file size. For a client situation, this is everything i need.
So my new workflow (with Encoding.com in the middle) is:
- User uploads video
- Application dumps video file in FTP directory
- Application sends a new job to Encoding.com using their API, telling them to download from our FTP
- Encoding.com downloads, encodes, then FTPs the FLV file to my CDN and sends a call-back to my application
- Admin gets notified there is a video awaiting approval
- [Insert Rob Conery beer reference]
Preheat oven to 180 degrees…
Now i have my encoding setup, I'll need a CDN. In this case i am using Edgecast, as our company has worked very successfully with them before, and they have a POP in Australia, but the above use of Encoding.com allows a lot of flexibility, as it can either send to you using HTTP (a link emailed to you), FTP to server of your choice, or straight into Amazon’s S3 storage.
In my case Edgecast offers FTP support, so by adding that to my Encoding.com XML requests you’re done. It’s that easy.