I've hacked together a version of HBTest that allows encoding H.264 at a constant rate factor, and uses a Dec. 9th 2006 snapshot of the latest x264 core (54).
It implements a --crf (short version: -Q) flag that changes Constant Quality encoding (-q) from using a constant quantizer rate control method to a constant rate factor method, something that's been available in x264 for over a year now but seems to have never made it into HandBrake.
According to the folks at doom9, it should provide better subjective constant quality than constant quantizer at the same -q value.
I'm a total n00b to programming and contributing to open source projects, so should I bother posting a patch? I probably don't follow proper coding methodology, so someone would have to clean it up a tad bit. For example, I used a string of if statements where a case switch probably would be more efficient.
I've got a diff -u for it against titer's last trunk SVN revision (not the more recently updated branch).
It only compiles if you replace your x264 core with a relatively current one (it needs the param.i_rc_method variable to exist and the constants for its values X264_RC_CRF, etc, defined).
Would the best thing to do be for me to just paste my diffs into a forum post, for anyone to look at at their leisure? It's just a few small changes to common.h, enc264.c, and test.c.
Here's the doom9 thread that led to the creation of the CRF option in x264:
http://forum.doom9.org/showthread.php?t=101551
It explains why and when you should use constant rate factor. Basically, CRF is the highest quality single-pass encoding because it takes motion into account. A CQP encode at Q=18 will stay at Q=18 regardless of the frame. CRF will increase the Q to, say, 20, for high motion frames (compressing them more) and lower it down to 16 for low motion. That means that while the average PSNR goes slightly down, the *perceptible* image quality goes up.
P.S.
Switching to the current core of x264 has awesome fringe benefits: encoding speed almost *doubles* on my G5 thanks to G. Poirier's recent Altivec enhancements to the codec. From 10.09 fps to 19.79 fps!
Constant rate factor encoding in HandBrake's cli
Forum rules
*******************************
Please be aware we are now using GitHub for issue tracking and feature requests.
- This section of the forum is now closed to new topics.
*******************************
*******************************
Please be aware we are now using GitHub for issue tracking and feature requests.
- This section of the forum is now closed to new topics.
*******************************
Yes, it would be nice to see what you have done here. Maybe you could pase in your changes just so we could see what you have done.
Are your changes based on Rev 70 from this trunk, or that last one titer checked in at the official site ? Either way, it would be great if you wanted to post your changes so we might take a look at what you have done. Sounds very interesting...
Are your changes based on Rev 70 from this trunk, or that last one titer checked in at the official site ? Either way, it would be great if you wanted to post your changes so we might take a look at what you have done. Sounds very interesting...
crf patches
The last one titer checked into the trunk, unfortunately. I downloaded from here: svn://svn.m0k.org/HandBrake/trunkdynaflash wrote:Are your changes based on Rev 70 from this trunk, or that last one titer checked in at the official site?
I got the x264 snapshot from here ftp://ftp.videolan.org/pub/videolan/x26 ... 45.tar.bz2 . Because the x264 team uses .tar.bz and the HandBrake jam rules want tar.gz, I had uncompress it, then tar and gzip it before HandBrake would accept it.
Here are the patches.
This is applied inside the test/ directory:
Code: Select all
diff -Naur HandBrake/test/test.c HandBrake-custom/test/test.c
--- HandBrake/test/test.c 2006-12-12 18:21:14.000000000 -0500
+++ HandBrake-custom/test/test.c 2006-12-12 18:26:52.000000000 -0500
@@ -37,6 +37,7 @@
static int acodec = 0;
static int chapter_start = 0;
static int chapter_end = 0;
+static int crf = 0;
/* Exit cleanly on Ctrl-C */
static volatile int die = 0;
@@ -404,6 +405,11 @@
}
job->file = strdup( output );
+ if( crf )
+ {
+ job->crf = 1;
+ }
+
if( twoPass )
{
job->pass = 1;
@@ -530,6 +536,7 @@
fprintf( stderr, " kHz)\n"
" -b, --vb <kb/s> Set video bitrate (default: 1000)\n"
" -q, --quality <float> Set video quality (0.0..1.0)\n"
+ " -Q, --crf Use with -q for CRF instead of CQP\n"
" -S, --size <MB> Set target size\n"
" -B, --ab <kb/s> Set audio bitrate (default: 128)\n"
" -w, --width <number> Set picture width\n"
@@ -575,7 +582,8 @@
{ "ab", required_argument, NULL, 'B' },
{ "rate", required_argument, NULL, 'r' },
{ "arate", required_argument, NULL, 'R' },
-
+ { "crf", no_argument, NULL, 'Q' },
+
{ 0, 0, 0, 0 }
};
@@ -583,7 +591,7 @@
int c;
c = getopt_long( argc, argv,
- "hvuC:f:i:o:t:c:a:s:e:E:2dgw:l:n:b:q:S:B:r:R:",
+ "hvuC:f:i:o:t:c:a:s:e:E:2dgw:l:n:b:q:S:B:r:R:Q",
long_options, &option_index );
if( c < 0 )
{
@@ -756,7 +764,10 @@
case 'B':
abitrate = atoi( optarg );
break;
-
+ case 'Q':
+ crf = 1;
+ break;
+
default:
fprintf( stderr, "unknown option (%s)\n", argv[optind] );
return -1;
Code: Select all
diff -Naur HandBrake/libhb/common.h HandBrake-custom/libhb/common.h
--- HandBrake/libhb/common.h 2006-12-12 18:21:14.000000000 -0500
+++ HandBrake-custom/libhb/common.h 2006-12-12 17:55:20.000000000 -0500
@@ -122,6 +122,7 @@
int vrate_base;
int pass;
int h264_13;
+ int crf;
/* Audio tracks:
Indexes in hb_title_t's audios list, starting from 0.
diff -Naur HandBrake/libhb/encx264.c HandBrake-custom/libhb/encx264.c
--- HandBrake/libhb/encx264.c 2006-12-12 18:21:14.000000000 -0500
+++ HandBrake-custom/libhb/encx264.c 2006-12-12 18:12:09.000000000 -0500
@@ -74,15 +74,29 @@
if( job->vquality >= 0.0 && job->vquality <= 1.0 )
{
+
+ if( job->crf==0)
+ {
/* Constant QP */
+ param.rc.i_rc_method= X264_RC_CQP;
param.rc.i_qp_constant = 51 - job->vquality * 51;
hb_log( "encx264: encoding at constant QP %d",
param.rc.i_qp_constant );
- }
- else
+ }
+
+ if( job->crf==1)
+ {
+ param.rc.i_rc_method = X264_RC_CRF;
+ param.rc.f_rf_constant = 51 - job->vquality * 51;
+ hb_log( "encx264: Encoding at constant RF %f",
+ param.rc.f_rf_constant );
+ }
+ }
+
+ if (!(job->vquality >= 0.0 && job->vquality <= 1.0))
{
/* Rate control */
- param.rc.b_cbr = 1;
+ param.rc.i_rc_method = X264_RC_ABR;
param.rc.i_bitrate = job->vbitrate;
switch( job->pass )
{
jbrjake:
I noticed that the new x264 is missing some of the files that are in the version the HB uses. Did you just replace the new ones and keep the old ones? Or did you just use only the new library and keep none of the old x264 library ?
After incorporating the new version of x264 I almost fell off my rocker when my fps almost doubled encoding at 1400 kbps with h.264 Baseline Level 3.0 on my experimental build.
Only to be let down when, despite what I asked for I only got a total of 170 kbps in my final movie!!!
But, I do think you are onto something here, I will keep trying to successfully implement the new x264 into this version. I have no idea if this will have any effect on the Intel Memory leak. But, if we can almost double the h.264 encoding speed for the new h.264 format, we have made huge gains !!!
After I get the new x264 version working, I will see if your patches can be applied to the Average Bitrate setting in the Mac OSX version.
I noticed that the new x264 is missing some of the files that are in the version the HB uses. Did you just replace the new ones and keep the old ones? Or did you just use only the new library and keep none of the old x264 library ?
After incorporating the new version of x264 I almost fell off my rocker when my fps almost doubled encoding at 1400 kbps with h.264 Baseline Level 3.0 on my experimental build.
Only to be let down when, despite what I asked for I only got a total of 170 kbps in my final movie!!!
But, I do think you are onto something here, I will keep trying to successfully implement the new x264 into this version. I have no idea if this will have any effect on the Intel Memory leak. But, if we can almost double the h.264 encoding speed for the new h.264 format, we have made huge gains !!!
After I get the new x264 version working, I will see if your patches can be applied to the Average Bitrate setting in the Mac OSX version.
Which files in particular? I was under the impression the only change would be that cbr mode (average bitrate) went from a boolean to the param.rc.i_method format. I believe I'm only using the new library. The oldest of any of the files in my HandBrake/contrib/x264 is Dec 9th 2006, the day of the snapshot I used. The x264 files in Handbrake/contrib/lib, contrib/bin, and contrib/include are dated the same time as my last build.dynaflash wrote:I noticed that the new x264 is missing some of the files that are in the version the HB uses. Did you just replace the new ones and keep the old ones? Or did you just use only the new library and keep none of the old x264 library ?
Hmm. I know I ran into that at one point (including the horribly painful letdown), and was able to fix it.After incorporating the new version of x264 I almost fell off my rocker when my fps almost doubled encoding at 1400 kbps with h.264 Baseline Level 3.0 on my experimental build.
Only to be let down when, despite what I asked for I only got a total of 170 kbps in my final movie!!!
Just for [Censored] and giggles, though, do a head -1 on the video that build produced in order to look at the x264 header. See if it's really using core 54 or not. I know that I had to struggle a bit to keep the jam rules from replacing the new x264 components with the old ones. Having any of the old bits around could screw it up.
Last night I took a clean copy of the source and applied my patches and substituted the new x264 core. Using the resultant build of HBTest, I'm able to successfully do ABR rips.
This statement:
Code: Select all
/Applications/HBTest-perfect -v -i /Users/jon/Movies/DVD_VIDEO/ -t 1 -c 4 -o /Users/jon/Movies/fear-perfectabr.mp4 -a none -r 23.976 -e x264 -b 1500 -w 704 -l 352 --crop 60:60:8:6Code: Select all
x264 - core 54 - H.264/MPEG-4 AVC codec - Copyleft 2005 - http://www.videolan.org/x264.html - options: cabac=1 ref=1 deblock=1:0:0 analyse=0x1:0x111 me=hex subme=4 brdo=0 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 chroma_qp_offset=0 slices=2 nr=0 decimate=1 mbaff=0 bframes=0 keyint=479 keyint_min=25 scenecut=40 rc=abr bitrate=1500 ratetol=1.0 rceq='blurCplx^(1-qComp)' qcomp=0.60 qpmin=10 qpmax=51 qpstep=4 ip_ratio=1.40Hmmm. Okay, I will have to get up to speed on this. Frankly I am not using the CLI but trying to implement the new x264 into the Mac OSX Gui. So, I am not really familiar with how to make sure it uses the new x264 on building it with the command line.
Did you have to modify the jam rules at all to make sure it used all of the new core you introduced.
Sorry, my command line skills are nominal at best.
Did you have to modify the jam rules at all to make sure it used all of the new core you introduced.
Sorry, my command line skills are nominal at best.
Hrm...I don't know anything about building the GUI. I think that it should stay the same, though. In either case the real work happens in libhb, right? If libhb is compiled with the new x264, the GUI should use it too. Since there's no way to set the --crf flag in the GUI (unless you wanted to add to the .nib), everything should work exactly the same except with the new x264 core.dynaflash wrote:Hmmm. Okay, I will have to get up to speed on this. Frankly I am not using the CLI but trying to implement the new x264 into the Mac OSX Gui. So, I am not really familiar with how to make sure it uses the new x264 on building it with the command line.
Did you have to modify the jam rules at all to make sure it used all of the new core you introduced.
Sorry, my command line skills are nominal at best.
Please check the header of the low bitrate video my patches produced, using the cli command
Code: Select all
header -1 filenameI didn't modify the jam rules at all.
First what I did was download a new snapshot of x264.
But that's in tar.bz2 format and the jamfile expects tar.gz. So then I uncompressed it (by double-clicking on it in the Finder) and renamed the resultant directory to "x264" from "x264-snapshot-20061209-2245".
I took this new x264 directory and I compressed it to the tar.gz format HandBrake's jam rules expect with this command:
Code: Select all
tar -cvzf x264.tar.gz x264At this point I applied my patches, and did ./configure and then jam and everything went well.
It did take me like 6 attempts before I got that routine down pat, though. I understand how frustrating it is. Forget one little thing, and the jam rules will download the old x264 core and overwrite your tarball.
Or, if you build it once with the old core, and then switch to the new, you might be forgetting to delete the objects that were already compiled the first time through. Can't tell you how many times I replaced that x264 directory and tarball, rebuilt, and still got "Core 43" on my video header.
Modifying the jam rules to expect a tar.bz2 and changing the contrib/version_x264.txt to point to the new x264 would probably be easier, but I wanted to keep my changes to a minimum.
A word of caution on the "contrib" directory - you want to be careful about upsetting the delicate balance there, since many of the contributed sources are hand-patched for a specific version and, as you noted, are automatically downloaded by jam during compile time.
My recommendation would be that you collectively settle on a single x264 source release and update the patches and Jamfile to match. I'd be happy to put the one you settle on up in the contrib directory on my site so the Jamfile will pull down the correct one automatically (follow the same naming convention as the release 71 Jamfile).
In general, the goal is to have every release compilable "out of the box" for all targeted platforms without manual intervention. I'm happy to do whatever is necessary to assist with this.
Rodney
My recommendation would be that you collectively settle on a single x264 source release and update the patches and Jamfile to match. I'd be happy to put the one you settle on up in the contrib directory on my site so the Jamfile will pull down the correct one automatically (follow the same naming convention as the release 71 Jamfile).
In general, the goal is to have every release compilable "out of the box" for all targeted platforms without manual intervention. I'm happy to do whatever is necessary to assist with this.
Rodney
Makes perfect sense.rhester wrote:My recommendation would be that you collectively settle on a single x264 source release and update the patches and Jamfile to match.
The only criterion I'll suggest is we go for one from after 11/18/2006. That's the day titer merged the faster Altivec code into the official x264 trunk, revision 601.
Okay, as jbrjake noted the newer ones are altivec enhanced which is great, but by themselves do not work with Rev 70 from here. I have tried manually using 601 and 604 from the svn over at vlc and everything builds but resulting movies are not right (as rhester noted) So, it seems to me it would be a great first step to integrate rhesters patches in x264 into the newer core version of either 601 or 604.
604 might be best as jbrjake noted.
rhester, the patches must be the additional files that your rev 70 has over the standar official hb distro.
604 might be best as jbrjake noted.
rhester, the patches must be the additional files that your rev 70 has over the standar official hb distro.
interestingly enough, even with the new x264, I do get 640 x 480 h.264 level 3.0 movies that transfer to the iPod. Its just that bitrate control is waaaay off.
rhester - how do you suggest we go about incorporating the 604 version with your patches ?
jbrjake - have you tried working with the rev 70 off of this svn yet ?
rhester - how do you suggest we go about incorporating the 604 version with your patches ?
jbrjake - have you tried working with the rev 70 off of this svn yet ?
I don't know anything about the 604 branch, but basically what needs to happen (at a high level) is:
- Fix the x264 patches (already in Handbrake Unofficial version 71) to match branch 604 of x264 - this has to be done by hand (and some trial-and-error)
- Put the 604 branch source tarball on this site
- Modify the Jamfile to point at it
- Test, test, test
Fixing up the x264 patches should be straightforward but tedious - if you guys need help with that, let me know. Beyond that, I just need someone to give me a source for the 604 tarball (or, alternately, e-mail me at handbrake@multics.dynalias.com and I'll set up an FTP account for you).
Rodney
- Fix the x264 patches (already in Handbrake Unofficial version 71) to match branch 604 of x264 - this has to be done by hand (and some trial-and-error)
- Put the 604 branch source tarball on this site
- Modify the Jamfile to point at it
- Test, test, test
Fixing up the x264 patches should be straightforward but tedious - if you guys need help with that, let me know. Beyond that, I just need someone to give me a source for the 604 tarball (or, alternately, e-mail me at handbrake@multics.dynalias.com and I'll set up an FTP account for you).
Rodney
Ok, I just grabbed the rev70 source.
I believe the problem is with how rate control methods are implemented in encx264.c with the new revision:
Some of that's wrong, I think. For example, max vbv bitrate shouldn't be vbitrate, afaik. I'll have some suggestions up later tonight, hopefully.
I believe the problem is with how rate control methods are implemented in encx264.c with the new revision:
Code: Select all
if( job->vquality >= 0.0 && job->vquality <= 1.0 )
{
/* Constant QP */
param.rc.i_qp_constant = 51 - job->vquality * 51;
hb_log( "encx264: encoding at constant QP %d",
param.rc.i_qp_constant );
}
else
{
/* Rate control */
/* no longer in x264 - see rc.i_rc_method in x264.h */
/* param.rc.b_cbr = 1; */
/* these were the only settings I could use to get accurate ending video bitrate */
param.rc.i_rc_method = X264_RC_CRF;
param.rc.i_vbv_max_bitrate = job->vbitrate;
param.rc.i_vbv_buffer_size = 224;
param.rc.i_rf_constant = 1;
param.rc.i_bitrate = job->vbitrate;
switch( job->pass )
{
case 1:
param.rc.i_rc_method = X264_RC_ABR;
param.rc.b_stat_write = 1;
param.rc.psz_stat_out = pv->filename;
break;
case 2:
param.rc.i_rc_method = X264_RC_ABR;
param.rc.b_stat_read = 1;
param.rc.psz_stat_in = pv->filename;
break;
}
Got ABR working fine with my changes to encx264.c. I ran into significant problems with the default x264 core used by rev. 70. It's core 50, when the current is core 54, and I think some important things happened in the interim to fix the vbv buffer (see: https://trac.videolan.org/x264/browser/ ... .c?rev=602 )
I compiled once with my changes applied to r70, and the default x264 core 50. I tried to do a one-pass at was should have been average bitrate 1500kbps. It produced 650kbps video with this header:
Same command with core54 produces 1500kbps video with this header:
Note how rc= goes from cqp to abr without any more changes to HandBrake code, just different x264 cores. I searched through all the code and I have absolutely no clue where it was getting that qp=26 value or even why it was using CQP instead of ABR. Maybe some default value that was specified in an x264 header for core 50?
I'll post revised patches for my changes against the r70 source later tonight.
I compiled once with my changes applied to r70, and the default x264 core 50. I tried to do a one-pass at was should have been average bitrate 1500kbps. It produced 650kbps video with this header:
Code: Select all
x264 - core 50 - H.264/MPEG-4 AVC codec - Copyleft 2005 - http://www.videolan.org/x264.html - options: cabac=1 ref=1 deblock=1:0:0 analyse=0x1:0x111 me=hex subme=5 brdo=0 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 chroma_qp_offset=0 slices=2 nr=0 decimate=1 bframes=0 keyint=479 keyint_min=25 scenecut=40 rc=cqp qp=26 ip_ratio=1.40Code: Select all
x264 - core 54 - H.264/MPEG-4 AVC codec - Copyleft 2005 - http://www.videolan.org/x264.html - options: cabac=1 ref=1 deblock=1:0:0 analyse=0x1:0x111 me=hex subme=4 brdo=0 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 chroma_qp_offset=0 slices=2 nr=0 decimate=1 mbaff=0 bframes=0 keyint=479 keyint_min=25 scenecut=40 rc=abr bitrate=1500 ratetol=1.0 rceq='blurCplx^(1-qComp)' qcomp=0.60 qpmin=10 qpmax=51 qpstep=4 ip_ratio=1.40I'll post revised patches for my changes against the r70 source later tonight.
patches for r70
Here are working patches for using the new x264 and crf, made against svn://multics.dynalias.com/HandBrake/trunk
I tested them by encoding a small sample video with ABR, CQP, and CRF methods. All work fine and were encoded properly.
patch for the libhb directory:
patch for the test directory:
In order to not have the x264.tar.gz I manually inserted with overwritten with an older, downloaded one from the version file, I commented out 2 lines in the contrib directory's Jamfile:
Sorry, rhester. I was in a rush to make sure these patches would work. Obviously, the proper way to do it is, like you said, to edit the Jamfile to unzip bz2, get the new core of x264 up on a server, and update the version_x264.txt.
I tested them by encoding a small sample video with ABR, CQP, and CRF methods. All work fine and were encoded properly.
patch for the libhb directory:
Code: Select all
diff -Naur HandBrake-r70-raw/libhb/common.h HandBrake-r70-custom/libhb/common.h
--- HandBrake-r70-raw/libhb/common.h 2006-12-13 20:04:49.000000000 -0500
+++ HandBrake-r70-custom/libhb/common.h 2006-12-13 17:21:53.000000000 -0500
@@ -124,7 +124,8 @@
int pass;
int h264_13;
int h264_level;
-
+ int crf;
+
/* Audio tracks:
Indexes in hb_title_t's audios list, starting from 0.
-1 indicates the end of the list */
diff -Naur HandBrake-r70-raw/libhb/encx264.c HandBrake-r70-custom/libhb/encx264.c
--- HandBrake-r70-raw/libhb/encx264.c 2006-12-13 20:04:49.000000000 -0500
+++ HandBrake-r70-custom/libhb/encx264.c 2006-12-13 17:39:43.000000000 -0500
@@ -75,36 +75,39 @@
/* Slightly faster with minimal quality lost */
param.analyse.i_subpel_refine = 4;
- if( job->vquality >= 0.0 && job->vquality <= 1.0 )
+ if( job->vquality >= 0.0 && job->vquality <= 1.0 )
{
+
+ if( job->crf==0)
+ {
/* Constant QP */
+ param.rc.i_rc_method= X264_RC_CQP;
param.rc.i_qp_constant = 51 - job->vquality * 51;
hb_log( "encx264: encoding at constant QP %d",
param.rc.i_qp_constant );
- }
- else
- {
+ }
+
+ if( job->crf==1)
+ {
+ param.rc.i_rc_method = X264_RC_CRF;
+ param.rc.f_rf_constant = 51 - job->vquality * 51;
+ hb_log( "encx264: Encoding at constant RF %f",
+ param.rc.f_rf_constant );
+ }
+ }
- /* Rate control */
- /* no longer in x264 - see rc.i_rc_method in x264.h */
- /* param.rc.b_cbr = 1; */
-
- /* these were the only settings I could use to get accurate ending video bitrate */
- param.rc.i_rc_method = X264_RC_CRF;
- param.rc.i_vbv_max_bitrate = job->vbitrate;
- param.rc.i_vbv_buffer_size = 224;
- param.rc.i_rf_constant = 1;
-
+ if (!(job->vquality >= 0.0 && job->vquality <= 1.0))
+ {
+ /* Rate control */
+ param.rc.i_rc_method = X264_RC_ABR;
param.rc.i_bitrate = job->vbitrate;
switch( job->pass )
{
case 1:
- param.rc.i_rc_method = X264_RC_ABR;
param.rc.b_stat_write = 1;
param.rc.psz_stat_out = pv->filename;
break;
case 2:
- param.rc.i_rc_method = X264_RC_ABR;
param.rc.b_stat_read = 1;
param.rc.psz_stat_in = pv->filename;
break;
Code: Select all
diff -Naur HandBrake-r70-raw/test/test.c HandBrake-r70-custom/test/test.c
--- HandBrake-r70-raw/test/test.c 2006-12-13 20:04:48.000000000 -0500
+++ HandBrake-r70-custom/test/test.c 2006-12-13 17:18:29.000000000 -0500
@@ -38,6 +38,7 @@
static int acodec = 0;
static int chapter_start = 0;
static int chapter_end = 0;
+static int crf = 0;
/* Exit cleanly on Ctrl-C */
static volatile int die = 0;
@@ -409,6 +410,11 @@
}
job->file = strdup( output );
+ if( crf )
+ {
+ job->crf = 1;
+ }
+
if( twoPass )
{
job->pass = 1;
@@ -535,6 +541,7 @@
fprintf( stderr, " kHz)\n"
" -b, --vb <kb/s> Set video bitrate (default: 1000)\n"
" -q, --quality <float> Set video quality (0.0..1.0)\n"
+ " -Q, --crf Use with -q for CRF instead of CQP\n"
" -S, --size <MB> Set target size\n"
" -B, --ab <kb/s> Set audio bitrate (default: 128)\n"
" -w, --width <number> Set picture width\n"
@@ -580,7 +587,8 @@
{ "ab", required_argument, NULL, 'B' },
{ "rate", required_argument, NULL, 'r' },
{ "arate", required_argument, NULL, 'R' },
-
+ { "crf", no_argument, NULL, 'Q' },
+
{ 0, 0, 0, 0 }
};
@@ -588,7 +596,7 @@
int c;
c = getopt_long( argc, argv,
- "hvuC:f:i:o:t:c:a:s:e:E:2dgw:l:n:b:q:S:B:r:R:",
+ "hvuC:f:i:o:t:c:a:s:e:E:2dgw:l:n:b:q:S:B:r:R:Q",
long_options, &option_index );
if( c < 0 )
{
@@ -766,7 +774,10 @@
case 'B':
abitrate = atoi( optarg );
break;
-
+ case 'Q':
+ crf = 1;
+ break;
+
default:
fprintf( stderr, "unknown option (%s)\n", argv[optind] );
return -1;
Code: Select all
--- HandBrake-r70-custom/contrib/Jamfile 2006-12-13 17:16:25.000000000 -0500
+++ HandBrake-r70-raw/contrib/Jamfile 2006-12-13 20:16:36.000000000 -0500
@@ -238,12 +238,15 @@
actions LibX264
{
cd `dirname $(>)` && CONTRIB=`pwd` &&
- rm -rf x264 && tar xzf x264.tar.gz && cd x264 && $(LIBX264_PATCH)
+ ## rm -rf x264 && tar xzf x264.tar.gz &&
+ cd x264 && $(LIBX264_PATCH)
./configure --prefix=$CONTRIB --enable-pthread && make &&
make install &&
strip -S $CONTRIB/lib/libx264.a
}
-Wget $(SUBDIR)/x264.tar.gz : $(SUBDIR)/version_x264.txt ;
+
+# Wget $(SUBDIR)/x264.tar.gz : $(SUBDIR)/version_x264.txt ;
+
LibX264 $(SUBDIR)/lib/libx264.a : $(SUBDIR)/x264.tar.gz ;
# libxvidcore
Will be interesting to see if we can get the new x264 integrated with the patch files on rev 70 of this trunk. I am unfortunately still wrapping my teeth around how all of this flows once it gets to libhb.
Up until now, I have primarily been working with the GUI and some very small changes. Hope to work on this more tomorrow.
Up until now, I have primarily been working with the GUI and some very small changes. Hope to work on this more tomorrow.
Okay, late last night I did try using the new x264 using jbrjakes patches to the libhb directory. Specifically common.h, encx264.c and his temporary patches to the contrib/Jamfile to force HB to use the new x264 (again, I am ONLY using rev 70 from this trunk and the new h.264 level 3.0 format).
I did NOT use any of your patches for the test directory as unless I am mistaken, the mac gui doesnt use any of those files (although please correct me if I am wrong, I didnt have time to chase that down).
I only had time to test one clip. I am only testing using Average Bitrate with the Mac GUI.
Results: iPod playable 640 x 480 h.264 Baseline Level 3.0 clip.
However, I did NOT notice the huge speed bump on encoding that I did previously, although, as I also noted, without the patches I was ony encoding at about 100 kbps video even though the gui asked for 1400, I believe that may caused the excessive speed, not necessarily the new x264.
So, now I will continue to test using the same version with different settings to try to determine what is happening here.
As rhester and jbrjake noted, we might want to incorporate all of rhesters patches to x264 to the new rev 604 of x264 (this is the rev I used,I may try rev601 as well) so we can move forward from where we are now to work toward compatibility with Intel macs, particularly the huge memory leak, etc.
As a side note, we might try to do this fairly soon if possible as according to SndChaser Chris Long will be getting his finished Intel mods to him later this week (he was able to get ahold of him by email!).
Please feel free to vett any of my thoughts here. Again, I am still getting up to speed on the real nuts and bolts of HB and certainly am grateful for everyone helping me walk through some of its intricacies.
I did NOT use any of your patches for the test directory as unless I am mistaken, the mac gui doesnt use any of those files (although please correct me if I am wrong, I didnt have time to chase that down).
I only had time to test one clip. I am only testing using Average Bitrate with the Mac GUI.
Results: iPod playable 640 x 480 h.264 Baseline Level 3.0 clip.
However, I did NOT notice the huge speed bump on encoding that I did previously, although, as I also noted, without the patches I was ony encoding at about 100 kbps video even though the gui asked for 1400, I believe that may caused the excessive speed, not necessarily the new x264.
So, now I will continue to test using the same version with different settings to try to determine what is happening here.
As rhester and jbrjake noted, we might want to incorporate all of rhesters patches to x264 to the new rev 604 of x264 (this is the rev I used,I may try rev601 as well) so we can move forward from where we are now to work toward compatibility with Intel macs, particularly the huge memory leak, etc.
As a side note, we might try to do this fairly soon if possible as according to SndChaser Chris Long will be getting his finished Intel mods to him later this week (he was able to get ahold of him by email!).
Please feel free to vett any of my thoughts here. Again, I am still getting up to speed on the real nuts and bolts of HB and certainly am grateful for everyone helping me walk through some of its intricacies.
I'd rather not integrate jbrjake's patches yet until everyone is satisfied that things are behaving as they should.
Ideally, what I'd like to see happen (and I'm willing to assist with this) is the creation of a revision 71, based on the revision 70 trunk where the _only_ changes are the 'upgrade' of x264 and possibly the inclusion of the ABR change. This revision should be compilable out-of-the-box and should form the bedrock of future enhancements.
Once that's done, everyone can sync their local repositories with revision 71 so we're all on the same platform and then start checking in unrelated changes (GUI, memory leak fixes, future feature enhancements, etc.).
Thoughts? Should revision 71 be "x264-only" without any changes to libhb or the CLI (the ABR stuff), and ABR be treated as a separate enhancement?
Opinions welcomed - I have thoughts of my own, but I'm just one guy, and I don't want to set the future direction of a project I can't meaningfully contribute to long-term. You guys are doing the heavy lifting - start thinking about a roadmap for releases and I'll do what I can to assist with coordinating the initial efforts and keeping everything flowing smoothly.
Rodney
Ideally, what I'd like to see happen (and I'm willing to assist with this) is the creation of a revision 71, based on the revision 70 trunk where the _only_ changes are the 'upgrade' of x264 and possibly the inclusion of the ABR change. This revision should be compilable out-of-the-box and should form the bedrock of future enhancements.
Once that's done, everyone can sync their local repositories with revision 71 so we're all on the same platform and then start checking in unrelated changes (GUI, memory leak fixes, future feature enhancements, etc.).
Thoughts? Should revision 71 be "x264-only" without any changes to libhb or the CLI (the ABR stuff), and ABR be treated as a separate enhancement?
Opinions welcomed - I have thoughts of my own, but I'm just one guy, and I don't want to set the future direction of a project I can't meaningfully contribute to long-term. You guys are doing the heavy lifting - start thinking about a roadmap for releases and I'll do what I can to assist with coordinating the initial efforts and keeping everything flowing smoothly.
Rodney
Ideally, what I'd like to see happen (and I'm willing to assist with this) is the creation of a revision 71, based on the revision 70 trunk where the _only_ changes are the 'upgrade' of x264 and possibly the inclusion of the ABR change. This revision should be compilable out-of-the-box and should form the bedrock of future enhancements.
Sounds good to me. Unfortunately, without the manual work and hack to the contrib/Jamfile jbrjake used, I am not familiar enough with things to incorporate x264 Rev604 with the changes to x264 that you and it appears chris had going on. I agree that it would be nice to have the "out of the box" functionality of building rev 70 but with the newer altivec and MMX enhanced x264.
Makes sense as well as we can at least all start from that basic platform consistently. As with most folk I suspect,I have my own interests in HB's direction, but as you said, I am just one guy. Probably be best to step up the features one by one.Should revision 71 be "x264-only" without any changes to libhb or the CLI (the ABR stuff), and ABR be treated as a separate enhancement?
Unfortunately my working knowledge of svn goes about as far as checking out. I am in no position to speak intelligently about checking in new revisions without the risk of screwing up the tree. So, for my part, I would definitely require assistance there.
But, as you, I am interested in what everyone else thinks.
Oh, please please don't even consider integrating my patches. Even though they're working correctly for me with all x264 encoding methods, my code is nasty and inefficient.rhester wrote:I'd rather not integrate jbrjake's patches yet until everyone is satisfied that things are behaving as they should.
Thoughts? Should revision 71 be "x264-only" without any changes to libhb or the CLI (the ABR stuff), and ABR be treated as a separate enhancement?
I totally agree about creating a rev71 that just upgrades the x264.
Unfortunately, it's impossible for an "x264-only" revision without any libhb changes. The current way rate control functions in rev70's encx264.c simply won't work with the new x264 core, because of changes in the way VBV is handled. At the same time, the old code from the official svn trunk won't work because of changes in the way rate control methods are handled.
If you want to hold back the crf feature for a later addition, I think we should use the same code titer has in his recent official svn branch to handle the new x264:
Code: Select all
75 if( job->vquality >= 0.0 && job->vquality <= 1.0 )
76 {
77 /* Constant QP */
78 param.rc.i_rc_method = X264_RC_CQP;
79 param.rc.i_qp_constant = 51 - job->vquality * 51;
80 hb_log( "encx264: encoding at constant QP %d",
81 param.rc.i_qp_constant );
82 }
83 else
84 {
85 /* Rate control */
86 param.rc.i_rc_method = X264_RC_ABR;
87 param.rc.i_bitrate = job->vbitrate;
88 switch( job->pass )
89 {
90 case 1:
91 param.rc.b_stat_write = 1;
92 param.rc.psz_stat_out = pv->filename;
93 break;
94 case 2:
95 param.rc.b_stat_read = 1;
96 param.rc.psz_stat_in = pv->filename;
97 break;
98 }
99 }
Code: Select all
if( job->vquality >= 0.0 && job->vquality <= 1.0 )
{
/* Constant QP */
param.rc.i_qp_constant = 51 - job->vquality * 51;
hb_log( "encx264: encoding at constant QP %d",
param.rc.i_qp_constant );
}
else
{
/* Rate control */
/* no longer in x264 - see rc.i_rc_method in x264.h */
/* param.rc.b_cbr = 1; */
/* these were the only settings I could use to get accurate ending video bitrate */
param.rc.i_rc_method = X264_RC_CRF;
param.rc.i_vbv_max_bitrate = job->vbitrate;
param.rc.i_vbv_buffer_size = 224;
param.rc.i_rf_constant = 1;
param.rc.i_bitrate = job->vbitrate;
switch( job->pass )
{
case 1:
param.rc.i_rc_method = X264_RC_ABR;
param.rc.b_stat_write = 1;
param.rc.psz_stat_out = pv->filename;
break;
case 2:
param.rc.i_rc_method = X264_RC_ABR;
param.rc.b_stat_read = 1;
param.rc.psz_stat_in = pv->filename;
break;
}
So, are you saying that an x264 only upgrade revision would have to include changes to rev 70's libhb ?
Second question: in your opinion is there a downside to this?
Not to sound stupid, but what were the reasons for these changes between titers rev69 and rhesters rev70? Is it to enable the h.264 baseline 3.0 that was added?
I am admittedly over my head on the rate control ins and outs here.
Second question: in your opinion is there a downside to this?
Not to sound stupid, but what were the reasons for these changes between titers rev69 and rhesters rev70? Is it to enable the h.264 baseline 3.0 that was added?
I am admittedly over my head on the rate control ins and outs here.
Yeah, upgrading x264 to the newest core would have to involve minor changes to libhb/encx264.c. How minor? Using the first code block in my last post rather than the second one.dynaflash wrote:So, are you saying that an x264 only upgrade revision would have to include changes to rev 70's libhb ?
Second question: in your opinion is there a downside to this?
Not to sound stupid, but what were the reasons for these changes between titers rev69 and rhesters rev70? Is it to enable the h.264 baseline 3.0 that was added?
Pretty sure it shouldn't affect baseline 3.0 support, unless it requires a vbv buffer of 224.
I didn't make the changes, so I can't say why...but I believe it was like this:
After core 43 (the version of x264 used by titer's svn trunk) the way to specify the rate control method changed.
It used to be that if you wanted an average bitrate encode, you'd tell libx264 that param.rc.b_cbr=1. But by core 50, the version of x264 used in rhester's svn, that doesn't work...Now, to make libx264 encode at average bitrate, you tell it param.rc.i_rc_method=X264_RC_ABR.
r70's code does this, but it does some other stuff as well that confuses me:
1. When vquality isn't set (constant quality isn't selected), it defaults the rate control method to CRF (?!).
2. Then it sets a VBV max bitrate and buffer size--I believe needlessly (x264's defaults should work fine, and if they don't with core 50, they should with core 54).
3. Then it sets the rate factor constant to an incredibly low value of 1. That would never be used. Common values are 18-24.
4. And then....then and only then, it uses a switch statement to reset the rate control method to ABR while it checks whether it's the first or second pass.
But that doesn't work. By default pass=0. The switch statement only changes the rate control method to ABR when the pass is 1 or 2, and pass is only set to 1 or 2 when you're in the 1st or 2nd pass of a 2-pass encode.
So if you try doing a single pass ABR encode, you end up using odd CRF settings instead, because the line that sets the rate control method to ABR is never called.
Sorry, I should have been more clear. Too many damn acronyms in this video encoding stuff, and they're all so similar.dynaflash wrote: By ABR you are referring to using Average BitRate, I presume.
ABR = Average Bit Rate
CQP = Constant Quantizer (I forget what the P stands for)
CRF = Constant Rate Factor