Please critique my high-fidelity setting.

[L255J]

I've been working to get smart high-fidelity encodes for some time now, and I've managed to find a well-working setting:

Code: Select all

+ encoder: x264 (r1376-3feaec2)
+ quality: 19.00 (RF)
+ options: ref=4:bframes=4:b-adapt=2:direct=auto:b-pyramid=1:rc-lookahead=60:me=umh:subq=9:merange=24:analyse=all:trellis=2:no-fast-pskip=1

Now, I do value space savings just a little, which is why DCT Decimation is allowed and the RF isn't any lower than it is, and I do value reasonable encoding time just a little, which is why bframes≠16 and why my motion estimation settings aren't any more comprehensive than umh + 24.

My target device will almost always be my own computer + VLC, so with that in mind, are there any other settings I could be taking advantage of that will help compressibility at a given RF that wouldn't be going overboard?

I want great quality, but again, without going overboard. Please critique any part of the setting you wish.

EDIT: I forgot to mention that I'm using the latest SVN of Handbrake, which has a build of x264 that doesn't have mbtree + b-pyramid compatibility problems.

[mduell]

Did you read your activity log after encoding?

mbtree and b-pyramid weren't compatible in the version of x264 used in the 0.9.4 release, so adding b-pyramid without disabling mbtree is pointless. When you have to choose between the two, mbtree is considered the better option so b-pyramid is discarded. You're warned about this in the log ("b-pyramid + mb-tree is not supported"). See this thread for more info.

[L255J]

I'm sorry. I forgot to mention that I'm using the latest SVN, which has the latest build of x264, which doesn't have that problem.

[Deleted User 11865]

I'm sorry. I forgot to mention that I'm using the latest SVN, which has the latest build of x264, which doesn't have that problem.

You did mention the x264 build.

Anyway, if your playback devices support the settings and you don't mind the time it takes to encode (those are very slow settings), there's nothing obviously wrong with them.

If I were to use settings this slow, I might opt for a higher number of ref frames (say 5) and b-frames (not sure) instead of using trellis 2 (vs the default value of 1), but I've never tested settings this slow so I'm not sure what would be more efficient.

[tlindgren]

Now, I do value space savings just a little, which is why DCT Decimation is allowed and the RF isn't any lower than it is, and I do value reasonable encoding time just a little, which is why bframes≠16 and why my motion estimation settings aren't any more comprehensive than umh + 24

I did some quick testing on a couple of different (but fairly small) files using both the standard High Profile preset, yours and some intermediate versions. I used RF=20 for all, it shouldn't matter for this.

Your settings seems to give some minor size gains compared to the regular High Profile preset, but only 1-4% and it took 2-3 times as long to encode. However, starting with the High Profile preset and adding just 4R, 4B and B-Pyramid settings gives a similar or slightly smaller result than your encode with little or no noticeable speed hit on this source, casting doubt that your settings are that good.

rc-lookahead=60 gave negligable gains above that of 50 (already in High Profile) in this case but on the other hand there was little or not speed hit on these trials either (heck, even a lookahead of 100 didn't reduce the file size nor slow it down in these cases). So it may be worthwhile on other sources.

IIRC the High Profile preset was actually UMH/subme9 (but no rc-lookahead) until shortly before the release of 0.9.4, it was changed at the last moment based on advice from one of the x264 developers, based on this and earlier tests I've done his advice makes a lot of sense (the size hit is very small and mostly recovered by rc-lookahead=50, the benefits for encoding speed is very large).

If you want to really cram the file size down as far as humanly possible, I'd suggest adding a lot of B frames and reference frames, but be advised that it does make decoding harder and at one point it will suddenly stop being possible to hardware accelerate the decoding on the GPU (not 100% sure if VLC uses hardware acceleration). If you want to be able to show this on a machine with less CPU this may be important.

Based on what I've seen and read I wouldn't recommend no-skip-p for CRF/single pass encodes, direct=auto is more debatable but I'd still not use it for CRF. Likewise, analysis=all is doubtful for both CRF and 2-pass encodes, though some say it can sometimes help at very low resolutions (much lower than DVD).

IIRC the reference frame limit for hardware decoding depends on resolution, and I think it's 4 for 1080p (though at least older x264 sometimes incorrectly used one more ref than specified when using with b-pyramid... Not sure if this is fixed or not). The max is higher at lower resolution but then you need to tweak it depending on resolution which most find undesirable. I also doubt it will reduce the size all that much! and storage media is very cheap nowadays.

[mduell]

You did mention the x264 build.

Note the post and edit times.

[Deleted User 11865]

You did mention the x264 build.

Note the post and edit times.

Hmm. I can't read today

[Deleted User 11865]

settings seems to give some minor size gains compared to the regular High Profile preset, but only 1-4% and it took 2-3 times as long to encode. However, starting with the High Profile preset and adding just 4R, 4B and B-Pyramid settings gives a similar or slightly smaller result than your encode with little or no noticeable speed hit on this source, casting doubt that your settings are that good.

You can't compare different x264 settings based on bitrate alone when using CRF rate control. Some options will increase file size but also improve quality.

[tlindgren]

You can't compare different x264 settings based on bitrate alone when using CRF rate control. Some options will increase file size but also improve quality.

AFAIK they try to make CRF always return the same quality, but nothing is ever perfect and it can't possibly tune it that finely anyway.

From everything I can find the SSIM Mean value is supposed to be the best sign of encode quality and as independent as possible of all but the psy option (which reduce all quality metrics but is intended to make it look better to the human eye). On the media I tried the settings on right now the SSIM's are very similar except for the standard High Profile encode. I did look at them earlier and there too they were "similar" but the devil is in the details.

Specifically, here the High Profile plus 4 ref/B-frames and pyramid was 0.9600810 while the "high-fidelity setting" was 0.9600768 and also 2.6% larger. Encode speed was 28.356161 fps vs 8.842588 fps so in this case it got beat on quality metric, size AND speed!

I think the regular High Profile preset did fairly well too, even if it was 1.7% larger than the "high fidelity" settings and 4.4% larger than the my custom one I cooked up earlier because it had the same high encode speed (28.215975) and a measurably better SSIM, 0.9603095, offsetting at least some of the larger file size. Theoretically I could try reducing the CRF in steps of 0.01 or so to see if I can get a close SSIM with the regular High Profile preset but I'm not going to bother, not when then 4R/4B/B-Pyramid tweak has already "beaten" the original suggestion.

That doesn't mean the "high-fidelity" settings may not work well on other media and/or resolutions but it's still enough to suggest he should run a significant number of tests on different sources that matter to him before he goes with these very slow settings, it may not yield any benefits.

I'm sure the result if someone actually did make an extensive study of the effect of different options (in combinations!) on both size and SSIM it would be fascinating and useful, but it would be a big project and don't have the slightest interest in doing that work.

Or do you have a better way than SSIM to gauge quality for this purpose?, by all accounts it's significantly superior to PSNR.

[L255J]

I'm a little cautious on using too many reference frames. Reference frames, right now, seem to be everything when it comes to decoding videos, and I don't want my videos to be difficult to play back. 4 is the limit for Blu-ray, so I'm going to try to keep it at 4 if I can help it. At the moment I'm experimenting with subme being 10 rather than 9, and I think allowing DCT Decimation proved to be a bad idea (too much detail I didn't want was retained). Dark Shikari said allowing P-Skip was okay in light of what "AQ" did, but I don't know what AQ is nor do I want to risk allowing it since the very thing I'm working with at the moment is a LOT of solid colors.

And while I take into consideration SSIM, I use my own eyes more than anything.

EDIT: Well, nevermind about the reference frame thing. I read that Blu-Ray doesn't restrict AVC past it's own levels, so all SD movies will have a reference frame limit of 6 (720×480@30.0) or 5 (720×576@25.0). The only thing to worry about with Blu-Ray is that only strict or no pyramidal B-framing is allowed. So...how do I use strict b-pyramid with Handbrake?

[tlindgren]

Well, nevermind about the reference frame thing. I read that Blu-Ray doesn't restrict AVC past it's own levels, so all SD movies will have a reference frame limit of 6 (720×480@30.0) or 5 (720×576@25.0). The only thing to worry about with Blu-Ray is that only strict or no pyramidal B-framing is allowed. So...how do I use strict b-pyramid with Handbrake?

I see that there's a bunch of check-in's in Oct. 2009 that fixed the regular b-pyramid and added a strict mode which should be fully Blu-ray compatible.

Looks like the standalone executable option would be "--b-pyramid strict", as mention in thread http://forum.handbrake.fr/viewtopic.php?f=6&t=14340 HB normally uses the normal mode (it also says how to use b-pyramid=strict).

This doesn't matter for the original query, but while there's no extra restrictions with Blu-Ray for Level 4.0 and lower other than the b-pyramid thingy and possibly that you can't use more than 3 B-frames (sources differ), most Blu-Ray sources is tagged (and produced) as 4.1 where it also requires a minimum of 4 "slices" (the h.264 standard has no such requirement).

Most (all?) hardware decoders has the same limitation in the specification even if they more often than not works on many non-BR 4.1 streams anyway. Hardware accelerated decoding on PC via GPU has a different set of restrictions, IIRC it usually handles at least up to real 4.1, full b-pyramid and also more ref-frames than the standard says at lower resolutions (but not 16 @ SD/DVD).. Also, for all hardware decoders (arguably always) you're supposed to set vbv-bufsize and vbv-maxrate for full Level x.y compatibility (avoid overrunning the hardware with bursts of data) because for some reason x264 doesn't provide defaults based on the level.