"Broadcatching" with Automatic, Transmission, and Handbrake

Random chit-chat and anything that doesn't belong elsewhere
Post Reply
refulgentis
Bright Spark User
Posts: 342
Joined: Mon Apr 14, 2008 5:08 am

"Broadcatching" with Automatic, Transmission, and Handbrake

Post by refulgentis »

EDIT: please visit http://code.google.com/p/videoengine/ to keep track of the current progress of this software, this page hasn't been updated after nearly 4 months of development
With the advent of free daily video broadcasts being distributed by BitTorrent (i.e. Revision3), and the ubiquity of personal video players such at the iPod/iPod touch/iPhone, it's become advantageous to have these programs downloaded and converted for you automatically as they become available.

The magic of RSS, torrents, and shell scripting should make this easy, but up until recently it has been hard to get that working, at least on Mac. Approximately a year and a half ago, VisualHub implemented rudimentary conversion scripting, and there have been half-hearted attempts at automatic RSS downloaders (TED, TVShows), that were usually poorly maintained, unstable, and did not support feeds beyond a few sites.

Transmission, the most native Mac implementation of a torrent client available currently, has evolved over the last year or so to "watch" a certain folder and autoadd torrents downloaded to that folder. This is good and fine, but it needed an easy to use program to watch a torrent RSS feed and automatically download new torrents as they were added. The program developers didn't want to add this because they believed it (rightly so) that it would add bloat to the project.

At the beginning of October, a user on Transmission's forums posted a program called Automatic that was much easier to use than the few available torrent RSS watchers. It is now up to version 0.9 and very stable, and the author is responsive to any inquiries for help.
So, with Transmission becoming stable, Automatic appearing, and Handbrake's huge growth over the last year, combined with the scripting abilities of OS X, you can have essentially any program you'd like automatically downloaded and converted for you.

STEPS
1)Download Automatic from http://codingcurious.com/automatic/.
2)Download Transmission from http://www.transmissionbt.com
3)Download HandbrakeCLI
4) Configuring Automatic - Automatic is pretty simple to use, the basic instructions are included in the download, and when you start it up its easy to see how it works. Add a filter (ex. the show name you want to watch from a general channel feed), and then specify a URL source (a link to the RSS feed that has the links to the torrents in it). Here's an example of a feed you can try it with: http://jonsthoughtsoneverything.com/fee ... orrent.xml. Take note of what folder it says next to "Download enclosures to: ".
5) Configuring Transmission - If you haven't used torrents before, read up on forwarding ports so you can get maximum download and upload speed. To make sure your torrents from Automatic get added to Transmission, either check the "Open enclosures after downloading" box in Automatic, or go to Preferences -> Transfers -> Auto add: in Transmission and tell it to Watch for torrent files in: the folder you specified in Automatic. Also, uncheck Display options window, so that Transmission will just add the torrent silently rather than require further prompting from you to make sure it is added.
The key part to configuring Transmission is to make sure that the Default download location: and Keep incomplete files in: folders are separate. When torrents are downloading, they'll be in the Keep incomplete files: folder, when they're finished Transmission will move them to the Default download location folder. This is to make sure that incomplete torrents don't set off Handbrake converting a broken file, and along those lines make sure that both of these folders are on the same hard drive. If you have the incomplete folder on your internal hard drive and completed folder on an external, it might take awhile for the copy to finish, but Handbrake will start trying to convert the before its finished.
6) Configuring Handbrake/folder watching - This is the coolest part for me, because it shows off some of the power of OS X. A small 'program' written in Applescript will watch your completed torrents folder. If something gets added to it, it filters through all the files in the folder to see if they are video files that you want converted (by a list of extensions you specify), and it checks the label color of the files to see if they were already converted, so it doesn't waste time converting them again (it sets the color of processed files to green).

Second, you have to follow the directions in the fifth line of the post to adapt it for your use. All of variables that need to be adjusted for your system are located at the top of the script, and say something like "property extension_list". They're pretty much self-explanatory. Extension list is the list of file extensions you want converted, out folder is where you want your encodes and logs to be put (note the slash at the end!), handbrake_location is the path to your copy of HandbrakeCLI, encoding_options is...more complex. It's the settings that are passed to HandbrakeCLI. A description of HandbrakeCLI options is listed here: http://trac.handbrake.fr/wiki/CLIGuide. Note that built-in presets can be used, which makes it very simple (just use property encoding_options: "-Z 'preset name in GUI'"

The third and final step is to set up the folder watching so HandbrakeCLI is triggered when a torrent download is completed. Save your script in /Library/Scripts/Folder Action Scripts/. It's absolutely necessary to save it there or it acts wonky for reasons unkown to me. Then, go into the folder that your completed torrents will be downloaded into. Right click in empty space in it, select more, then Enable Folder Actions. Right click again, select more, select Configure folder actions, then add the folder your finished torrents will be in on the left pane. Select it, then click on the plus sign under the right pane, and select the name that you saved the script under in the Folder Actions directory.

Now, everything should be working. You should try dragging a file with an extension that will trigger the script to the folder you attached the script to make sure Handbrake is triggered properly, and set up a guaranteed match in Automatic to make sure that Automatic -> Transmission is working properly.

Enjoy!
Last edited by refulgentis on Tue Feb 24, 2009 9:45 pm, edited 10 times in total.
refulgentis
Bright Spark User
Posts: 342
Joined: Mon Apr 14, 2008 5:08 am

Re: "Broadcatching" with Automatic, Transmission, and Handbrake

Post by refulgentis »

old
Last edited by refulgentis on Tue Feb 24, 2009 9:44 pm, edited 4 times in total.
Northy
Regular User
Posts: 134
Joined: Fri May 16, 2008 1:06 pm

Re: "Broadcatching" with Automatic, Transmission, and Handbrake

Post by Northy »

refulgentis great work, keep it up.
isen
Posts: 1
Joined: Thu Feb 12, 2009 9:14 pm

Re: "Broadcatching" with Automatic, Transmission, and Handbrake

Post by isen »

Hi there:
Great post here for getting content to the ATV. I have Automatic and Transmission running successfully, but now I need to set up the Handbrake folder watching and I am not quite sure what of your scripts I need. I see your notes that you have a few things "FIXED IN MY CODE". Does that mean that the code you posted doesn't have the fixes? Or, is "your code" the code you have listed here?

I have my finished Transmission downloads put in a "finished" file. Now I would like to convert them to MP4 and import them into iTunes for syncing with my ATV.

Do you have a script or Automator flow to do this?

Thanks for all your great work!
refulgentis
Bright Spark User
Posts: 342
Joined: Mon Apr 14, 2008 5:08 am

Re: "Broadcatching" with Automatic, Transmission, and Handbrake

Post by refulgentis »

isen wrote:Hi there:
Great post here for getting content to the ATV. I have Automatic and Transmission running successfully, but now I need to set up the Handbrake folder watching and I am not quite sure what of your scripts I need. I see your notes that you have a few things "FIXED IN MY CODE". Does that mean that the code you posted doesn't have the fixes? Or, is "your code" the code you have listed here?

I have my finished Transmission downloads put in a "finished" file. Now I would like to convert them to MP4 and import them into iTunes for syncing with my ATV.

Do you have a script or Automator flow to do this?

Thanks for all your great work!
Usually it would mean the code here doesn't have fixes...right now specifically I forgot to update the log, so it does, but autotagging doesn't work unless you use my experimental script at the bottom of the post, name it autotag (no extension), make it executable (chmod 755 autotag), and put it in /usr/local/bin.

Right now, the AppleScript I posted above just has to be a Folder Action on your completed torrents folder, and everything should (and will, I can assure you) work properly, assuming you alter the AppleScript's default parameters at the top to match your system. The only odd caveat right now is that if you drop two files into completed, it converts both at the same time, rather than queueing them. I had some great ideas and now have queueing in my local code -- just a matter of testing it more before I unleash it. :)

I'm doing a big update either tonight or tomorrow afternoon hopefully, that will include a bunch of stuff for you to unzip and put in /Library/Application Support/Engine, this will include a current (0.9.3, it's against Handbrake policy to distribute current development code) version of HandbrakeCLI, mp4tags built on an intel Mac, and a couple shell scripts that allow queueing (i.e. one encode at a time, so there isn't 6 encodes). This allows me to assume a certain level of 'base' support, so I don't have to give complicated instructions on building this and that program, and you don't have to do that work, and also paves the way for me to support tagging without asking you to have a gazillion files compiled and ready.

I'm going to completely redo the guide at my earliest convenience as well. A *lot* has changed since the first post, many new features and I've learned a lot myself. Eventually, God willing, I'll learn enough about Objective C/Xcode to make this into a nicer GUI for everyone, but that's a bit off I'm afraid. :) I think I'll be moving this off to a separate page for formatting purposes eventually and use Google Code hosting if I can figure out how to use SVN.
refulgentis
Bright Spark User
Posts: 342
Joined: Mon Apr 14, 2008 5:08 am

Re: "Broadcatching" with Automatic, Transmission, and Handbrake

Post by refulgentis »

for people watching this thread -- please note http://code.google.com/p/videoengine/ is the place to visit for updates and information now. I just posted an extremely updated version of the code that used to be here, which has much less potential bugs and includes full tagging support.
GoodOmens
Regular User
Posts: 53
Joined: Thu Mar 22, 2007 8:30 pm

Re: "Broadcatching" with Automatic, Transmission, and Handbrake

Post by GoodOmens »

Will this unzip / unrar content? Some of the torrents I downloaded are compressed :-(
TorontoKev
Posts: 3
Joined: Wed Jul 08, 2009 5:38 pm

Re: "Broadcatching" with Automatic, Transmission, and Handbrake

Post by TorontoKev »

Hey!

I'm about to write an ap to do this for the PC, but I wanted to check if there was already one out there. If anyone knows of one, can you let me know? I don't want to reinvent the wheel, I just want to reinvent what already exists on the mac!!
refulgentis
Bright Spark User
Posts: 342
Joined: Mon Apr 14, 2008 5:08 am

Re: "Broadcatching" with Automatic, Transmission, and Handbrake

Post by refulgentis »

TorontoKev wrote:Hey!

I'm about to write an ap to do this for the PC, but I wanted to check if there was already one out there. If anyone knows of one, can you let me know? I don't want to reinvent the wheel, I just want to reinvent what already exists on the mac!!
I was told that you could tell utorrent to execute commands after a download finishes, unsure how much/if this will help you.
TorontoKev
Posts: 3
Joined: Wed Jul 08, 2009 5:38 pm

Re: "Broadcatching" with Automatic, Transmission, and Handbrake

Post by TorontoKev »

Hey, FYI as promised in the app above, I've written an app for Windows which will do the necessary for us non-Mac people !!

It's called ALERT & CONVERT. It will basically watch a folder (or search a target folder and recursive folders filled with AVI's), convert EVERYTHING it finds in it using HANDBRAKE CLI. As it goes, it will twitter and email you what it's up to. Then as a finishing move, adds that new file into the iTunes Library. If you have a playlist set up, you should be good to go with a sync.

You need this as badly as I did, head on over to http://pixeladdiction.co.uk/alertconvert/
Post Reply