Multiple queues with titles

Discussion of the cross-platform HandBrake command line interface
Forum rules
Activity Log is required for support requests. Posts without this will be locked! See Must Read: How To Get Questions Answered on These Forums for details

Forum Rules can be found here
User avatar
tubbyman
Posts: 42
Joined: Mon Apr 09, 2007 5:39 am

Multiple queues with titles

Postby tubbyman » Mon Apr 09, 2007 5:52 am

FYI, I have a simple perl wrapper for handbrake/mediafork CLI that will process every title of every VIDEO_TS folder in any folder on your computer, all in one go. I used it to batch process all of my movie/tvshow DVD backups so that I could import them into itunes/frontrow/appletv. I've used it on mactel, macppc, and linux. You have to put MediaForkCLI into /usr/local/bin so that the script can find it.

For example, if your backups are in "/Volumes/MyDVDs" here are some ways to run it:
./mediaforker.pl /Volumes/MyDVDs (puts mp4s right next to backups)
./mediaforker.pl /Volumes/MyDVDs ~/Movies (puts mp4s in your Movies folder)
./mediaforker.pl /Volumes/MyDVDs ~/Movies -f avi ...etc (puts avi's into your movies folder)
./mediaforker.pl --help (prints help)

You can download it here:
http://www.crazywhiteman.net/mediaforke ... ker.pl.bz2

User avatar
tubbyman
Posts: 42
Joined: Mon Apr 09, 2007 5:39 am

Postby tubbyman » Thu Apr 12, 2007 3:25 am

I updated the compression defaults from 2200k video with deinterlacing to constant quality 95% video without deinterlacing.

rhester
Contributor
Posts: 2888
Joined: Wed Apr 19, 2006 12:24 am
Location: Brooklyn, NY
Contact:

Postby rhester » Thu Apr 12, 2007 3:32 am

You realize 95% quality is very, very high and will produce a bitrate likely to exceed the ability for most devices to play the content, right?

Rodney

User avatar
tubbyman
Posts: 42
Joined: Mon Apr 09, 2007 5:39 am

Postby tubbyman » Thu Apr 12, 2007 4:45 am

Yeah. That's ok with me. I only have 1 player device: a mini running front row connected to a projector. All I want is to get my DVD backups into frontrow at near original quality without relying on an external app to play them. I don't really care too much about processing time or file size.

I'm running some tests right now with these new settings, and I run into any problems, I'll update the script. Also, these are just the default compression settings, anyone running the script can override them if they want output for another device.

User avatar
tubbyman
Posts: 42
Joined: Mon Apr 09, 2007 5:39 am

Postby tubbyman » Thu Apr 12, 2007 9:34 pm

Holy cow was I wrong. An H264 transcode of 3 GB mpeg-2 title set to 95% quality resulted in a nearly 12 GB file, and you were right, the bitrates were prohibitively high. I expected that it would be, at most, 95% of the the size of the original 3 GB title, put probably much smaller due to H264's better encoding. I looked around in the src code to see what the "-q" flag actually does, and I see my mistake. It's a very unusual interpretation of a quality setting.

jbrjake
Contributor
Posts: 4806
Joined: Wed Dec 13, 2006 3:38 am
Contact:

Postby jbrjake » Thu Apr 12, 2007 10:15 pm

tubbyman wrote:I looked around in the src code to see what the "-q" flag actually does, and I see my mistake. It's a very unusual interpretation of a quality setting.

Not really. The only thing unusual about it is that it's rescaled to 0%-100% instead of 0QP - 51QP like in x264's cli.

You're just misinterpreting what quality is referring to. It's saying what quantizer will be applied on the input x264 receives, which is not the same as what's on the DVD. The DVD has to be decoded from mpeg-2 to raw video before it can be passed over to x264 or any other codec. So the quantizer refers to the quality degradation from that decoded stream, not the quality degradation from the DVD. I do not believe this is any different from how setting a quantizer works in any other video or audio codec. QuickTime, for example, abstracts it even more...you just get Least, Low, Medium, High, and Best for quantizer choices.

User avatar
tubbyman
Posts: 42
Joined: Mon Apr 09, 2007 5:39 am

Postby tubbyman » Fri Apr 13, 2007 1:26 am

You're just misinterpreting what quality is referring to.


I agree. I completely missed it. I thought it would set a 264 bitrate that was roughly 95% of the average compressed mpeg-2 bitrate, maybe with a weight on it for how much better x264 was than mpeg-2. I didn't expect a percentage of the bitrate of the uncompressed stream. No matter, though. Now that I understand, I can easily calculate a quality setting that I like.

Thanks again!

jbrjake
Contributor
Posts: 4806
Joined: Wed Dec 13, 2006 3:38 am
Contact:

Postby jbrjake » Fri Apr 13, 2007 3:11 am

I can throw you another curve...

It's not really linear 0-100 % ;P

Instead, x264's quantizer scale is, I believe logarithmic.

So 95% isn't even going to necessarily be 95% of the quality of the raw stream. It's going to be a QP of about 3.

This has nothing to do with a percentage of the bitrate of the original. It's not like HB checks the original's bitrate and multiplies it by .95. It's telling x264 "Ok, apply a compression quantizer parameter of 3."

User avatar
tubbyman
Posts: 42
Joined: Mon Apr 09, 2007 5:39 am

Re: Multiple queues with titles

Postby tubbyman » Wed Feb 04, 2009 3:20 am

I apologize for including the script inline in my last post. Maybe a friendly admin will remove that post for me? I didn't see the code button on the editor. Anyway, here's an update of my perl wrapper for HB 0.9.3. It works exactly the same as before, except that it ignores titles shorter than 10 mins.
Requires HandBrakeCLI to be in /usr/local/bin, tested on macppc, macintel, and linux.

Code: Select all

#!/usr/bin/perl -w
# Finds all VIDEO_TS folders inside input folder and individually
# process all titles found. Please note: this script may run for
# weeks if you have a slow computer and/or a lot of VIDEO_TS
# folders to process.
#
# David Colbert
# Sun Apr  8 18:06:57 EDT 2007
#
# Thu Jun  7 20:29:33 EDT 2007
# Renamed from mediaforker.pl to handbraker.pl to match handbrake's
# schizophrenic project name. Also, now defaults to HandBrakeCLI,
# but falls back to MediaForkCLI.
#
# Fri Jan 25 15:20:27 EST 2008
# Updated for the latest HandBrakeCLI, removed switches that are
# already in defaults. Added deintelacing back in, since HandBrake
# now has better deinterlacing. Removed all references to mediafork.
#
# Sat Jan 31 16:34:52 EST 2009
# Updated for handbrake 0.9.3, changed defaults to virtually the
# same as quicktime, added in new title scanner regexp, added time threshold
# so that only long titles (greater than 10 mins) are processed.
# TODO: Switch to GetOpt::Long for arg handling, but it should keep
# pass-through HandBrakeCLI switch functionality

use strict;
use File::Find ();
# for the convenience of &wanted calls, including -eval statements:
use vars qw/*name *dir *prune/;
*name   = *File::Find::name;
*dir    = *File::Find::dir;
*prune  = *File::Find::prune;

# Declare subroutine for File::Find
sub wanted;

#
# Process input args
#
my $narg = scalar( @ARGV );

#
# Default Quality Settings (virtually identical to --preset QuickTime)
# Comment out any settings that you don't want
#

my $compressargs = ""; # Do not comment this out
# Video
$compressargs = $compressargs . " -f MP4 -e x264";   # Video format
$compressargs = $compressargs . " -b 2500";         # Video bitrate (atv=2500,iphone=1000)
#$compressargs = $compressargs . " -d";               # Deintelacing
$compressargs = $compressargs . " -m";               # Chapter Markers
$compressargs = $compressargs . " -2 -T";            # 2-pass turbo
$compressargs = $compressargs . " -p";               # Anamorphic

# Advanced x264 options
$compressargs = $compressargs . " -x ref=3:mixed-refs:bframes=3:weightb:direct=auto:me=umh:subme=7:analyse=all:8x8dct:trellis=1:no-fast-pskip=1:psy-rd=1,1";

# Audio
$compressargs = $compressargs . " -E faac";         # Audio format
$compressargs = $compressargs . " -B 384";         # Audio bitrate (384=64x6 for 5.1)
$compressargs = $compressargs . " -R auto";         # Audio sample rate
$compressargs = $compressargs . " -a 1";            # Audio Track
$compressargs = $compressargs . " -6 dpl2";         # Audio Mixdown



my $allargs = "";
my $infolder = "";
my $outfolder = "";
my $ext = ".m4v";

my $helpstr = "\nhandbraker.pl <infolder> <outfolder> <args>\n\
This script will recursively process all titles longer than 10 minutes in all VIDEO_TS folders inside \
<infolder>. <infolder> is required. \n\
<outfolder> is optional. If specified, this is where the movie files will be \
placed. If not specified, files will be placed at same level \
as the DVD folder that contains the VIDEO_TS folder. \n\
<args> are optional. If you choose to set args, you must also set <outfolder> \
Defaults: Pretty much the same as '--preset QuickTime', with minor tweaks. \n\
Examples:\n# Put MP4s right next to backups:\nprompt% handbraker.pl /Volumes/MyDVDs\n\
# Put MP4s in your Movies folder:\nprompt% handbraker.pl /Volumes/MyDVDs ~/Movies\n\
# Put AVIs into your movies folder:\nprompt% handbraker.pl /Volumes/MyDVDs ~/Movies -f avi ...\n\
# View the help:\nprompt% handbraker.pl --help\n\
Notes:\nHandbrake has a unique interpretation for \"quality\". I recommend that you read \
up on how it works before attempting the \"-q\" switch. Also, Handbrake will undoubtedly \
eventually make large scale processing easier, making this wrapper unnecessary.\n\
Use \"HandBrakeCLI --help\" to find out how to set your own compression args.\n\n";

if( $narg == 0 )
{
   die $helpstr;
}
elsif( $narg == 1 )
{
   $infolder = $ARGV[0];
   $allargs = $allargs . " $ARGV[0]";
}
elsif( $narg == 2 )
{
   $infolder = $ARGV[0];
   $outfolder = $ARGV[1];
   $allargs = $allargs . " $ARGV[0] $ARGV[1]";
}
elsif( $narg > 2 )
{
   # More switches used than just folder, so build up user-specified args from inputs
   $infolder = $ARGV[0];
   $outfolder = $ARGV[1];
   $allargs = $allargs . " $ARGV[0] $ARGV[1]";
   $compressargs = "";
   for( my $thisArg=2; $thisArg < $narg; $thisArg++ )
   {
      $compressargs = $compressargs . " $ARGV[$thisArg]";
      $allargs = $allargs . " $ARGV[$thisArg]";
   }
}

# Spit out some help if asked
die $helpstr if( $allargs =~ /(-h|-help|--help|--h)/ );

# Set filename extension for output movies
if( $compressargs =~ /avi/ )
{ $ext = ".avi"; }
elsif( $compressargs =~ /ogm/ )
{ $ext = ".ogm"; }
elsif( $compressargs =~ /mp4/ )
{ $ext = ".mp4"; }


#
# Find HandBrakeCLI
#
my $HandBrakeCLI = "/usr/local/bin/HandBrakeCLI";
my $HandBrakeMissing = "No HandBrakeCLI found in /usr/local/bin\n\
Please download HandBrakeCLI and place it in /usr/local/bin\n";
die $HandBrakeMissing if( !(-x $HandBrakeCLI ) );

#
# Find VIDEO_TS folders
#
my @allvidts = ();
# Traverse $infolder, push VIDEO_TS folders into allvidts
File::Find::find({wanted => \&wanted}, $infolder);
die "No VIDEO_TS folders found in $infolder\n" if( @allvidts == 0 );

#
# Processing loop
#

# Scan for all titles
my @hbcmds = ();
my $timeThreshold = 10*60; # Set a 10 minute time threshold to skip ads, trailers
foreach my $vidts (@allvidts)
{

   my @titles = ();
   my @durations = ();
   #  Find all titles in this video_ts
   open( SCANNER, "$HandBrakeCLI -i \"$vidts\" -t 0 2>&1 |" );
   while( <SCANNER> )
   {
      chomp();
      push( @titles, $2) if( $_ =~ /(\s*\+\s+title\s+)(\d+)(:)*/ );
      push( @durations, (($2*3600)+($3*60)+$4) ) if( $_ =~ /(\s*\+\s*duration\s*\:)\s*(\d+):(\d+):(\d+)*/ );
   }
   close( SCANNER );

   # Set output file and output folder
   my $iTitle = 0;
   foreach my $title (@titles)
   {
      if( $durations[$iTitle] > $timeThreshold )
      {
         my $outfile = "";
         if( $outfolder eq "" )
         {
            my @ts_split = split( /\/VIDEO_TS/, $vidts );
            $outfile = $ts_split[0] . ".title$title$ext";
         }
         else
         {
            my @slash_split = split( /\//, $vidts );
            $outfile = $outfolder . "\/" . $slash_split[-2] . ".title$title$ext";
         }

         # build system call strings
         push( @hbcmds, ( "$HandBrakeCLI -i \"$vidts\" -o \"$outfile\" -t $title $compressargs" ) );
      }
      $iTitle++;
   }
}


# Execution
foreach my $hbcmd (@hbcmds)
{
   # Grid stuff goes here
   system( $hbcmd );
}

#
# Subroutines
#

# generated by find2perl
sub wanted {
    my ($dev,$ino,$mode,$nlink,$uid,$gid);

    /^VIDEO_TS\z/s &&
    (($dev,$ino,$mode,$nlink,$uid,$gid) = lstat($_)) &&
    -d _ &&
   push( @allvidts, $name);
}

User avatar
TedJ
Moderator
Posts: 5343
Joined: Thu Feb 21, 2008 1:25 am
Location: Brisbane, Australia

Re: Multiple queues with titles

Postby TedJ » Wed Feb 04, 2009 6:33 am

tubbyman wrote:I apologize for including the script inline in my last post. Maybe a friendly admin will remove that post for me? I didn't see the code button on the editor.


I'm guessing you missed the Edit button too... you didn't need to double post. ;)


Return to “CLI”