H265 in new HandBrake 1.3.0 gives 30-60% higher bitrate

HandBrake for Windows support
Forum rules
An Activity Log is required for support requests. Please read How-to get an activity log? for details on how and why this should be provided.
User avatar
s55
HandBrake Team
Posts: 10350
Joined: Sun Dec 24, 2006 1:05 pm

Re: H265 in new HandBrake 1.3.0 gives 30-60% higher bitrate

Post by s55 »

Would be there a way to get 1.2.2 with the 1.3.0 GUI?
Nope.
1.3.0 > AO 1 / visible worse quality / same size
Can you provide 1.2.2 and 1.3.0 logs?
Vaniii
Posts: 3
Joined: Wed Dec 18, 2019 4:12 pm

Re: H265 in new HandBrake 1.3.0 gives 30-60% higher bitrate

Post by Vaniii »

Hello.

Here are some Logs as Textfiles which i know there were good visible differences (using 2k Screen).
wetransfer.com (Online for 7 Days)

I don't know the exactly Bitrates and Sizes anymore, didn't thought to get asked about because todays support is usually like "we saw it and don't care about it anymore ".

[Leon] is with a lot Graining, so the v1.2.2 CRF 20 (9GB) was smaller than the v1.3.0 CRF 20 (11GB) and also v1.3.0 CRF 21 (10GB)
- No AQ Changes
- CRF 21 v1.3.0 is worse than CRF 20 v1.2.2 but still 1GB bigger
- CRF 20 v1.3.0 Quality is the same like v1.2.2 CRF 20 // You really need time to find somewhere tinest quality improvements for 2GB more space // But the point is, you can encode it in v1.2.2 as CRF 19 with a size like v1.3.0 CRF 20 but which much better quality.

[SB] is "normally", no Graining, almost same Bitrate and Filesize, good to compare the visible quality.
- Always CRF 18
- AQ Changes
- v1.2.2 similar to v1.3.0 AQ 2
- v1.2.2 better quality than v1.3.0 AQ 1


Overall
1.2.2 (Initial situation)
- reliable smaller or same file sizes with identical quality choosen by CRF

1.3.0
- AQ 1 worse quality by the same size as 1.2.2
- AQ 2 comparable quality but often much bigger files
- GUI and improved infos in Audio and Subtitle


Of course, there are always some little differences depend at the source and settings, but overall you usually will get bigger files which no quality improvements at all.
As soon as you take AQ 1 for the same size, you will get a worse quality.
User avatar
s55
HandBrake Team
Posts: 10350
Joined: Sun Dec 24, 2006 1:05 pm

Re: H265 in new HandBrake 1.3.0 gives 30-60% higher bitrate

Post by s55 »

- AQ 1 worse quality by the same size as 1.2.2
How are you determining that the quality is lower?


The logs show that the video stream outputs are identical between 1.3.0 AQ1 and 1.2.2


1.2

Code: Select all

x265 [info]: frame I:   1196, Avg QP:16.57  kb/s: 27821.44
x265 [info]: frame P:  13409, Avg QP:17.77  kb/s: 9970.44
x265 [info]: frame B:  51219, Avg QP:21.42  kb/s: 2582.55
x265 [info]: Weighted P-Frames: Y:0.9% UV:0.8%
x265 [info]: consecutive B-frames: 9.7% 1.7% 1.7% 1.7% 85.1%
[19:55:57] mux: track 0, 65824 frames, 1496480741 bytes, 4546.78 kbps, fifo 2048

1.3

Code: Select all

x265 [info]: frame I:   1196, Avg QP:16.57  kb/s: 27821.44
x265 [info]: frame P:  13409, Avg QP:17.77  kb/s: 9970.44
x265 [info]: frame B:  51219, Avg QP:21.42  kb/s: 2582.55
x265 [info]: Weighted P-Frames: Y:0.9% UV:0.8%
x265 [info]: consecutive B-frames: 9.7% 1.7% 1.7% 1.7% 85.1%
[19:57:04] mux: track 0, 65824 frames, 1496480741 bytes, 4546.78 kbps, fifo 2048
mduell
Veteran User
Posts: 8187
Joined: Sat Apr 21, 2007 8:54 pm

Re: H265 in new HandBrake 1.3.0 gives 30-60% higher bitrate

Post by mduell »

The differences in "quality" are in Vaniii's imagination.
delicious_storage
Posts: 7
Joined: Sat Jan 04, 2020 5:27 pm

Re: H265 in new HandBrake 1.3.0 gives 30-60% higher bitrate

Post by delicious_storage »

I did a quick test comparison between 1.2.2 and 1.3.0 with aq-mode=1, using a 240 sec preview encode.

Source file: Blu-Ray Remux
Video codec: H.265 10-bit
Preset: Slow
Quality: Constant quality RF18

Output file size was almost identical (128 bytes difference) and image quality appeared to be pixel for pixel identical. With aq-mode=1 removed from the advanced options in 1.3.0, the file size was 16.5% larger but appeared to retain slightly more detail.
User avatar
BradleyS
Moderator
Posts: 1860
Joined: Thu Aug 09, 2007 12:16 pm

Re: H265 in new HandBrake 1.3.0 gives 30-60% higher bitrate

Post by BradleyS »

Official presets in HandBrake 1.3.1 (just released) set aq-mode 1 via the additional encoder options.
mduell
Veteran User
Posts: 8187
Joined: Sat Apr 21, 2007 8:54 pm

Re: H265 in new HandBrake 1.3.0 gives 30-60% higher bitrate

Post by mduell »

Why is HB moving contra to the x265 developer's default recommendation?
User avatar
BradleyS
Moderator
Posts: 1860
Joined: Thu Aug 09, 2007 12:16 pm

Re: H265 in new HandBrake 1.3.0 gives 30-60% higher bitrate

Post by BradleyS »

We already do so in the case of strong-intra-smoothing for one.

In this case, the commit message is probably most helpful: https://github.com/HandBrake/HandBrake/ ... b9e678cc37

"We have received numerous reports about x265 in HandBrake 1.3.0 creating ~30% larger files with no perceived quality gain, and have confirmed this to be true in most cases. This appears to be the result of x265 recently switching to use aq-mode 2 by default. This change makes the behavior of the official presets more closely match that of HandBrake 1.2.2 and earlier. This seems to be the best course of action both technically and for the user experience until the default aq-mode improves."

Perhaps aq-mode 2 makes sense for some material (such as low bit rate, same as strong-intra-smoothing) but it seems to be worse—or at least not better enough to warrant the overhead—for high quality encodes.
User avatar
BradleyS
Moderator
Posts: 1860
Joined: Thu Aug 09, 2007 12:16 pm

Re: H265 in new HandBrake 1.3.0 gives 30-60% higher bitrate

Post by BradleyS »

It's important to note that HandBrake is not forcing an aq-mode globally. This only affects the official presets that use x265, all of which are geared toward relatively high quality encodes. Simply choosing the x265 encoder on the video tab will not set a specific aq-mode.
delicious_storage
Posts: 7
Joined: Sat Jan 04, 2020 5:27 pm

Re: H265 in new HandBrake 1.3.0 gives 30-60% higher bitrate

Post by delicious_storage »

BradleyS wrote: Mon Jan 06, 2020 4:32 pm We already do so in the case of strong-intra-smoothing for one.

In this case, the commit message is probably most helpful: https://github.com/HandBrake/HandBrake/ ... b9e678cc37

"We have received numerous reports about x265 in HandBrake 1.3.0 creating ~30% larger files with no perceived quality gain, and have confirmed this to be true in most cases. This appears to be the result of x265 recently switching to use aq-mode 2 by default. This change makes the behavior of the official presets more closely match that of HandBrake 1.2.2 and earlier. This seems to be the best course of action both technically and for the user experience until the default aq-mode improves."

Perhaps aq-mode 2 makes sense for some material (such as low bit rate, same as strong-intra-smoothing) but it seems to be worse—or at least not better enough to warrant the overhead—for high quality encodes.
Are strong-intra-smoothing and rect disabled in the presets because they reduce image quality, or is it because these options increase encoding time with little benefit? From my brief testing adding "strong-intra-smoothing=0:rect=0" increases the size of the encodes.
Deleted User 11865

Re: H265 in new HandBrake 1.3.0 gives 30-60% higher bitrate

Post by Deleted User 11865 »

rect definitely slows dow the encode significantly with generally little benefit (in my testing, file size increase was minimal). strong-intra-smoothing should help preserve a little fine detail, but TBH I cannot tell the difference, and AFAICT the file size difference is generally a few Kbps at most -- disabling sao has a much bigger effect on fine detail retention, sao is great for low-bitrate encodes but seems to blur things out quite a bit.
User avatar
BradleyS
Moderator
Posts: 1860
Joined: Thu Aug 09, 2007 12:16 pm

Re: H265 in new HandBrake 1.3.0 gives 30-60% higher bitrate

Post by BradleyS »

strong-intra-smoothing is what it sounds like, a bilinear smoothing filter on intra frames to limit bit rate spikes. Intra frames being of course, what other frames are based on. It's basically reducing the quality of your video before you even start encoding. You don't want this for high quality encodes, but it can lead to more consistent bit rates with lower quality encodes that would be smooth anyway. It's a bad default for HandBrake's presets, and I'd say not very wise in general.

rect on the other hand is disabled in the official presets because it doesn't provide enough benefit for the speed hit. When we created HandBrake's H.265 presets, x265's medium preset wasn't very impressive quality-/performance-wise and while the slow preset was better, it was really slow. Disabling rect gave a major speed increase while retaining the other benefits of the slow preset. This still seems to be the case years later; slow is about 2.4x slower than medium: https://handbrake.fr/docs/en/latest/tec ... mance.html
User avatar
BradleyS
Moderator
Posts: 1860
Joined: Thu Aug 09, 2007 12:16 pm

Re: H265 in new HandBrake 1.3.0 gives 30-60% higher bitrate

Post by BradleyS »

TL;DR While we generally try to avoid custom settings and stick to encoder presets, in the case of x265 there isn't a preset that creates the quality/speed/compression we thought would be most beneficial. And all x265 presets use strong-intra-smoothing which is not good for high quality encodes at all. So here we are.
Deleted User 11865

Re: H265 in new HandBrake 1.3.0 gives 30-60% higher bitrate

Post by Deleted User 11865 »

BradleyS wrote: Mon Jan 06, 2020 5:06 pm strong-intra-smoothing is what it sounds like, a bilinear smoothing filter on intra frames to limit bit rate spikes. Intra frames being of course, what other frames are based on. It's basically reducing the quality of your video before you even start encoding.
FWIW:
https://x265.readthedocs.io/en/default/cli.html#cmdoption-strong-intra-smoothing wrote:
Enable strong intra smoothing for 32x32 intra blocks. This flag performs bi-linear interpolation of the corner reference samples for a strong smoothing effect. The purpose is to prevent blocking or banding artifacts in regions with few/zero AC coefficients. Default enabled
(emphasis mine, it does look relevant, even though I have no clue what the corner reference samples are specifically)

For the record, I have it disabled in my own encodes as well, but in my testing, the difference between on vs. off was 3-4 Kbps at most (on 1080p encodes), with no discernible visual difference even looking at still frames.
delicious_storage
Posts: 7
Joined: Sat Jan 04, 2020 5:27 pm

Re: H265 in new HandBrake 1.3.0 gives 30-60% higher bitrate

Post by delicious_storage »

BradleyS wrote: Mon Jan 06, 2020 5:06 pm strong-intra-smoothing is what it sounds like, a bilinear smoothing filter on intra frames to limit bit rate spikes. Intra frames being of course, what other frames are based on. It's basically reducing the quality of your video before you even start encoding. You don't want this for high quality encodes, but it can lead to more consistent bit rates with lower quality encodes that would be smooth anyway. It's a bad default for HandBrake's presets, and I'd say not very wise in general.

rect on the other hand is disabled in the official presets because it doesn't provide enough benefit for the speed hit. When we created HandBrake's H.265 presets, x265's medium preset wasn't very impressive quality-/performance-wise and while the slow preset was better, it was really slow. Disabling rect gave a major speed increase while retaining the other benefits of the slow preset. This still seems to be the case years later; slow is about 2.4x slower than medium: https://handbrake.fr/docs/en/latest/tec ... mance.html
Thank you, this was very informative.
nhyone
Bright Spark User
Posts: 252
Joined: Fri Jul 24, 2015 4:13 am

Re: H265 in new HandBrake 1.3.0 gives 30-60% higher bitrate

Post by nhyone »

Since HandBrake is already adding its own options, how about turning SAO off by default? :D

(sao=0:selective-sao=0)

Also, set CTU and ME range to 32 for 1080p.

(ctu=32:merange=32)

I would set CTU/ME range = 16 for 480p, but I've not seen anyone suggest that.
User avatar
BradleyS
Moderator
Posts: 1860
Joined: Thu Aug 09, 2007 12:16 pm

Re: H265 in new HandBrake 1.3.0 gives 30-60% higher bitrate

Post by BradleyS »

You're welcome to do some analysis on those options. :)

I will say, if you're feeling the need to disable SAO, you probably need tune Grain anyway.
mduell
Veteran User
Posts: 8187
Joined: Sat Apr 21, 2007 8:54 pm

Re: H265 in new HandBrake 1.3.0 gives 30-60% higher bitrate

Post by mduell »

Someone on reddit had a case where the aq-mode + other changes in the new x265 made their output smaller.
User avatar
BradleyS
Moderator
Posts: 1860
Joined: Thu Aug 09, 2007 12:16 pm

Re: H265 in new HandBrake 1.3.0 gives 30-60% higher bitrate

Post by BradleyS »

Seems to be more than just changes between encoder versions. They aren't using exactly the same settings (typical). Note the presence of strong-intra-smoothing, which we disable in the official presets.
nhyone
Bright Spark User
Posts: 252
Joined: Fri Jul 24, 2015 4:13 am

Re: H265 in new HandBrake 1.3.0 gives 30-60% higher bitrate

Post by nhyone »

I did a small experiment with 1080p @ CRF 20 on a 20-core (+20 HT) Haswell machine:

Code: Select all

              fps      size    QP     options
x265 slow       8.107   0.00%  24.09
                8.585  +0.19%  24.08  merange=32
               10.594  +0.74%  24.09  rect=0
               15.550  +0.96%  24.15  ctu=32
               20.998  +1.49%  24.15  rect=0:ctu=32
               22.091  +1.69%  24.15  rect=0:ctu=32:merange=32

                8.613  -18.2%  24.47  aq-mode=1
               22.334  -16.9%  24.61  aq-mode=1:rect=0:ctu=32
               23.705  -16.7%  24.61  aq-mode=1:rect=0:ctu=32:merange=32
rect=0 and ctu=32 improve speed at expense of a little compression efficiency.

For this video, aq-mode 1 is about 18% smaller than aq-mode 2.

Code: Select all

              fps      size    QP     options
x264 veryfast 208.941  +4.22%  22.12
               88.937  +3.95%  22.14  lookahead-threads=1
               90.989  +3.72%  22.14  lookahead-threads=1:threads=6

     veryslow  26.157  +2.13%  24.31  ref=3:bframes=6
               20.124  +2.25%  24.31  lookahead-threads=1:ref=3:bframes=6
               10.229  +1.77%  24.31  lookahead-threads=1:threads=6:ref=3:bframes=6
x264 is pretty size-competitive with x265 slow, aq-mode 2. veryslow is not any faster than x265 slow, though -- to my surprise.

This experiment is not representative of encoding in general, but I still find the result interesting.

Updated: added QP.
Last edited by nhyone on Thu Jan 30, 2020 1:22 am, edited 4 times in total.
mduell
Veteran User
Posts: 8187
Joined: Sat Apr 21, 2007 8:54 pm

Re: H265 in new HandBrake 1.3.0 gives 30-60% higher bitrate

Post by mduell »

But same RF isn't the same quality across different settings. You'd have to look at PSNR or SSIM, with the appropriate tunes.
selen
Posts: 11
Joined: Fri Nov 22, 2019 8:32 pm

Re: H265 in new HandBrake 1.3.0 gives 30-60% higher bitrate

Post by selen »

BradleyS wrote: Tue Jan 21, 2020 3:39 am I will say, if you're feeling the need to disable SAO, you probably need tune Grain anyway.
Tune Grain will lead to much higher bitrate, can be a few times higher and much slower encoding.
NoSAO has almost none effect on bitrate, just a few kbps but picture will be without blurriness. I tend to use nosao on all H265 encodes.
nhyone
Bright Spark User
Posts: 252
Joined: Fri Jul 24, 2015 4:13 am

Re: H265 in new HandBrake 1.3.0 gives 30-60% higher bitrate

Post by nhyone »

mduell wrote: Wed Jan 22, 2020 2:07 am But same RF isn't the same quality across different settings. You'd have to look at PSNR or SSIM, with the appropriate tunes.
I'm not after identical quality. I assume they are close enough, so it is more about encoding speed and file size. I've added QP to the numbers if it means anything at all.

Throwing out another bunch of numbers:

Code: Select all

                fps      size     QP
x265 ultrafast  130.206  -34.48%  26.53
     superfast   76.380  -27.69%  25.90
     veryfast    29.539  -16.57%  24.56
     faster      29.273  -16.66%  24.56
     fast        26.671  -15.69%  24.50
     medium      18.766   -9.72%  24.15
     slow         8.034    0.00%  24.09
     slower       2.329   +1.71%  24.26
     veryslow     1.358   +1.23%  24.27
     placebo      0.833   +4.24%  24.30
The same clip, but trying out various x265 presets (note: not HandBrake presets).

ultrafast and superfast use CTU 32. CTU 32 almost half the time for 1080p encoding on high-core machines (33 WPP rows instead of 16). I'm in the camp that thinks CTU 32 is sufficient for 1080p.

The actual result is only meaningful for this video on this machine, but I think we can see some broad patterns. It may raise more questions than it answers -- like why file size goes up the slower/'better' the preset? It may even interest/puzzle you enough to do your own tests. :D
Post Reply