Multiple queues with titles

[tubbyman]

FYI, I have a simple perl wrapper for handbrake/mediafork CLI that will process every title of every VIDEO_TS folder in any folder on your computer, all in one go. I used it to batch process all of my movie/tvshow DVD backups so that I could import them into itunes/frontrow/appletv. I've used it on mactel, macppc, and linux. You have to put MediaForkCLI into /usr/local/bin so that the script can find it.

For example, if your backups are in "/Volumes/MyDVDs" here are some ways to run it:
./mediaforker.pl /Volumes/MyDVDs (puts mp4s right next to backups)
./mediaforker.pl /Volumes/MyDVDs ~/Movies (puts mp4s in your Movies folder)
./mediaforker.pl /Volumes/MyDVDs ~/Movies -f avi ...etc (puts avi's into your movies folder)
./mediaforker.pl --help (prints help)

You can download it here:
http://www.crazywhiteman.net/mediaforke ... ker.pl.bz2

[tubbyman]

I updated the compression defaults from 2200k video with deinterlacing to constant quality 95% video without deinterlacing.

[rhester]

You realize 95% quality is very, very high and will produce a bitrate likely to exceed the ability for most devices to play the content, right?

Rodney

[tubbyman]

Yeah. That's ok with me. I only have 1 player device: a mini running front row connected to a projector. All I want is to get my DVD backups into frontrow at near original quality without relying on an external app to play them. I don't really care too much about processing time or file size.

I'm running some tests right now with these new settings, and I run into any problems, I'll update the script. Also, these are just the default compression settings, anyone running the script can override them if they want output for another device.

[tubbyman]

Holy cow was I wrong. An H264 transcode of 3 GB mpeg-2 title set to 95% quality resulted in a nearly 12 GB file, and you were right, the bitrates were prohibitively high. I expected that it would be, at most, 95% of the the size of the original 3 GB title, put probably much smaller due to H264's better encoding. I looked around in the src code to see what the "-q" flag actually does, and I see my mistake. It's a very unusual interpretation of a quality setting.

[jbrjake]

I looked around in the src code to see what the "-q" flag actually does, and I see my mistake. It's a very unusual interpretation of a quality setting.

Not really. The only thing unusual about it is that it's rescaled to 0%-100% instead of 0QP - 51QP like in x264's cli.

You're just misinterpreting what quality is referring to. It's saying what quantizer will be applied on the input x264 receives, which is not the same as what's on the DVD. The DVD has to be decoded from mpeg-2 to raw video before it can be passed over to x264 or any other codec. So the quantizer refers to the quality degradation from that decoded stream, not the quality degradation from the DVD. I do not believe this is any different from how setting a quantizer works in any other video or audio codec. QuickTime, for example, abstracts it even more...you just get Least, Low, Medium, High, and Best for quantizer choices.

[tubbyman]

You're just misinterpreting what quality is referring to.

I agree. I completely missed it. I thought it would set a 264 bitrate that was roughly 95% of the average compressed mpeg-2 bitrate, maybe with a weight on it for how much better x264 was than mpeg-2. I didn't expect a percentage of the bitrate of the uncompressed stream. No matter, though. Now that I understand, I can easily calculate a quality setting that I like.

Thanks again!

[jbrjake]

I can throw you another curve...

It's not really linear 0-100 % ;P

Instead, x264's quantizer scale is, I believe logarithmic.

So 95% isn't even going to necessarily be 95% of the quality of the raw stream. It's going to be a QP of about 3.

This has nothing to do with a percentage of the bitrate of the original. It's not like HB checks the original's bitrate and multiplies it by .95. It's telling x264 "Ok, apply a compression quantizer parameter of 3."

[tubbyman]

I apologize for including the script inline in my last post. Maybe a friendly admin will remove that post for me? I didn't see the code button on the editor. Anyway, here's an update of my perl wrapper for HB 0.9.3. It works exactly the same as before, except that it ignores titles shorter than 10 mins.
Requires HandBrakeCLI to be in /usr/local/bin, tested on macppc, macintel, and linux.

Code: Select all

#!/usr/bin/perl -w
# Finds all VIDEO_TS folders inside input folder and individually
# process all titles found. Please note: this script may run for
# weeks if you have a slow computer and/or a lot of VIDEO_TS 
# folders to process.
#
# David Colbert
# Sun Apr  8 18:06:57 EDT 2007
#
# Thu Jun  7 20:29:33 EDT 2007
# Renamed from mediaforker.pl to handbraker.pl to match handbrake's 
# schizophrenic project name. Also, now defaults to HandBrakeCLI, 
# but falls back to MediaForkCLI. 
#
# Fri Jan 25 15:20:27 EST 2008
# Updated for the latest HandBrakeCLI, removed switches that are 
# already in defaults. Added deintelacing back in, since HandBrake 
# now has better deinterlacing. Removed all references to mediafork.
#
# Sat Jan 31 16:34:52 EST 2009
# Updated for handbrake 0.9.3, changed defaults to virtually the 
# same as quicktime, added in new title scanner regexp, added time threshold
# so that only long titles (greater than 10 mins) are processed.
# TODO: Switch to GetOpt::Long for arg handling, but it should keep 
# pass-through HandBrakeCLI switch functionality

use strict;
use File::Find ();
# for the convenience of &wanted calls, including -eval statements:
use vars qw/*name *dir *prune/;
*name   = *File::Find::name;
*dir    = *File::Find::dir;
*prune  = *File::Find::prune;

# Declare subroutine for File::Find
sub wanted;

#
# Process input args
#
my $narg = scalar( @ARGV );

#
# Default Quality Settings (virtually identical to --preset QuickTime)
# Comment out any settings that you don't want
#

my $compressargs = ""; # Do not comment this out
# Video
$compressargs = $compressargs . " -f MP4 -e x264";	# Video format
$compressargs = $compressargs . " -b 2500";			# Video bitrate (atv=2500,iphone=1000)
#$compressargs = $compressargs . " -d";					# Deintelacing
$compressargs = $compressargs . " -m";					# Chapter Markers
$compressargs = $compressargs . " -2 -T";				# 2-pass turbo
$compressargs = $compressargs . " -p";					# Anamorphic

# Advanced x264 options
$compressargs = $compressargs . " -x ref=3:mixed-refs:bframes=3:weightb:direct=auto:me=umh:subme=7:analyse=all:8x8dct:trellis=1:no-fast-pskip=1:psy-rd=1,1";

# Audio
$compressargs = $compressargs . " -E faac";			# Audio format
$compressargs = $compressargs . " -B 384";			# Audio bitrate (384=64x6 for 5.1)
$compressargs = $compressargs . " -R auto";			# Audio sample rate
$compressargs = $compressargs . " -a 1";				# Audio Track
$compressargs = $compressargs . " -6 dpl2";			# Audio Mixdown



my $allargs = "";
my $infolder = "";
my $outfolder = "";
my $ext = ".m4v"; 

my $helpstr = "\nhandbraker.pl <infolder> <outfolder> <args>\n\
This script will recursively process all titles longer than 10 minutes in all VIDEO_TS folders inside \
<infolder>. <infolder> is required. \n\
<outfolder> is optional. If specified, this is where the movie files will be \
placed. If not specified, files will be placed at same level \
as the DVD folder that contains the VIDEO_TS folder. \n\
<args> are optional. If you choose to set args, you must also set <outfolder> \
Defaults: Pretty much the same as '--preset QuickTime', with minor tweaks. \n\
Examples:\n# Put MP4s right next to backups:\nprompt% handbraker.pl /Volumes/MyDVDs\n\
# Put MP4s in your Movies folder:\nprompt% handbraker.pl /Volumes/MyDVDs ~/Movies\n\
# Put AVIs into your movies folder:\nprompt% handbraker.pl /Volumes/MyDVDs ~/Movies -f avi ...\n\
# View the help:\nprompt% handbraker.pl --help\n\
Notes:\nHandbrake has a unique interpretation for \"quality\". I recommend that you read \
up on how it works before attempting the \"-q\" switch. Also, Handbrake will undoubtedly \
eventually make large scale processing easier, making this wrapper unnecessary.\n\
Use \"HandBrakeCLI --help\" to find out how to set your own compression args.\n\n";

if( $narg == 0 )
{
	die $helpstr;
}
elsif( $narg == 1 )
{
	$infolder = $ARGV[0];
	$allargs = $allargs . " $ARGV[0]";
}
elsif( $narg == 2 )
{
	$infolder = $ARGV[0];
	$outfolder = $ARGV[1];
	$allargs = $allargs . " $ARGV[0] $ARGV[1]";
}
elsif( $narg > 2 )
{
	# More switches used than just folder, so build up user-specified args from inputs
	$infolder = $ARGV[0];
	$outfolder = $ARGV[1];
	$allargs = $allargs . " $ARGV[0] $ARGV[1]";
	$compressargs = "";
	for( my $thisArg=2; $thisArg < $narg; $thisArg++ )
	{
		$compressargs = $compressargs . " $ARGV[$thisArg]";
		$allargs = $allargs . " $ARGV[$thisArg]";
	}
}

# Spit out some help if asked
die $helpstr if( $allargs =~ /(-h|-help|--help|--h)/ );

# Set filename extension for output movies
if( $compressargs =~ /avi/ )
{ $ext = ".avi"; }
elsif( $compressargs =~ /ogm/ )
{ $ext = ".ogm"; }
elsif( $compressargs =~ /mp4/ )
{ $ext = ".mp4"; }


#
# Find HandBrakeCLI
#
my $HandBrakeCLI = "/usr/local/bin/HandBrakeCLI";
my $HandBrakeMissing = "No HandBrakeCLI found in /usr/local/bin\n\
Please download HandBrakeCLI and place it in /usr/local/bin\n";
die $HandBrakeMissing if( !(-x $HandBrakeCLI ) );

#
# Find VIDEO_TS folders
#
my @allvidts = ();
# Traverse $infolder, push VIDEO_TS folders into allvidts
File::Find::find({wanted => \&wanted}, $infolder);
die "No VIDEO_TS folders found in $infolder\n" if( @allvidts == 0 );

#
# Processing loop
#

# Scan for all titles
my @hbcmds = ();
my $timeThreshold = 10*60; # Set a 10 minute time threshold to skip ads, trailers
foreach my $vidts (@allvidts)
{

	my @titles = ();
	my @durations = ();
	#  Find all titles in this video_ts 
	open( SCANNER, "$HandBrakeCLI -i \"$vidts\" -t 0 2>&1 |" );
	while( <SCANNER> )
	{
		chomp();
		push( @titles, $2) if( $_ =~ /(\s*\+\s+title\s+)(\d+)(:)*/ );
		push( @durations, (($2*3600)+($3*60)+$4) ) if( $_ =~ /(\s*\+\s*duration\s*\:)\s*(\d+):(\d+):(\d+)*/ );
	}
	close( SCANNER );

	# Set output file and output folder
	my $iTitle = 0;
	foreach my $title (@titles) 
	{
		if( $durations[$iTitle] > $timeThreshold )
		{
			my $outfile = "";
			if( $outfolder eq "" )
			{
				my @ts_split = split( /\/VIDEO_TS/, $vidts );
				$outfile = $ts_split[0] . ".title$title$ext";
			}
			else
			{
				my @slash_split = split( /\//, $vidts );
				$outfile = $outfolder . "\/" . $slash_split[-2] . ".title$title$ext";
			}

			# build system call strings
			push( @hbcmds, ( "$HandBrakeCLI -i \"$vidts\" -o \"$outfile\" -t $title $compressargs" ) );
		}
		$iTitle++;
	}
}


# Execution
foreach my $hbcmd (@hbcmds)
{
	# Grid stuff goes here
	system( $hbcmd );
}

#
# Subroutines
#

# generated by find2perl
sub wanted {
    my ($dev,$ino,$mode,$nlink,$uid,$gid);

    /^VIDEO_TS\z/s &&
    (($dev,$ino,$mode,$nlink,$uid,$gid) = lstat($_)) &&
    -d _ &&
	push( @allvidts, $name);
}

[TedJ]

I apologize for including the script inline in my last post. Maybe a friendly admin will remove that post for me? I didn't see the code button on the editor.

I'm guessing you missed the Edit button too... you didn't need to double post.