Same movie encoding, file 10% larger than before update

[eusi]

Description of problem or question:
I used Handbrake 1.0.7 for a long time.
After I updated to 1.1.1, my encodings feels larger than normal.
I tested it also:
via 1.0.7 Movie X got 983 MB
via 1.1.1 Movie X got 1100 MB
Why are my movies more like 10% bigger than before?
Is it possible to get the same "quality and size" than before? :-/

My Preset is x265, 25cfr, 30fps peak, slow with extra options: strong-intra-smoothing=1:me=1:lookahead-slices=5:rd=3:psy-rdoq=0.00:qpmax=51


HandBrake version:
Handbrake 1.0.7 vs Handbrake 1.1.1


Difference of media info data (all other data are the same):

Handbrake 1.0.7

Code: Select all

Writing library             : x265 2.1:[Windows][GCC 5.3.1][64 bit] 8bit
Encoding settings           : wpp / ctu=64 / min-cu-size=8 / max-tu-size=32 / tu-intra-depth=1 / tu-inter-depth=1 / me=1 / subme=3 / merange=57 / rect / no-amp / max-merge=3 / temporal-mvp / no-early-skip / rskip / rdpenalty=0 / no-tskip / no-tskip-fast / strong-intra-smoothing / no-lossless / no-cu-lossless / no-constrained-intra / no-fast-intra / open-gop / no-temporal-layers / interlace=0 / keyint=250 / min-keyint=25 / scenecut=40 / rc-lookahead=25 / lookahead-slices=5 / bframes=4 / bframe-bias=0 / b-adapt=2 / ref=4 / limit-refs=3 / limit-modes / weightp / no-weightb / aq-mode=1 / qg-size=32 / aq-strength=1.00 / cbqpoffs=0 / crqpoffs=0 / rd=3 / psy-rd=2.00 / rdoq-level=2 / psy-rdoq=0.00 / log2-max-poc-lsb=8 / no-rd-refine / signhide / deblock=0:0 / sao / no-sao-non-deblock / b-pyramid / cutree / no-intra-refresh / rc=crf / crf=25.0 / qcomp=0.60 / qpmin=0 / qpmax=51 / qpstep=4 / ipratio=1.40 / pbratio=1.30

Handbrake 1.1.1

Code: Select all

Writing library             : x265 2.6:[Windows][GCC 7.3.0][64 bit] 8bit+10bit+12bit
Encoding settings           : cpuid=1173503 / frame-threads=4 / numa-pools=16 / wpp / no-pmode / no-pme / no-psnr / no-ssim / log-level=2 / input-csp=1 / input-res=1918x816 / interlace=0 / total-frames=0 / level-idc=0 / high-tier=1 / uhd-bd=0 / ref=4 / no-allow-non-conformance / no-repeat-headers / annexb / no-aud / no-hrd / info / hash=0 / no-temporal-layers / open-gop / min-keyint=25 / keyint=250 / bframes=4 / b-adapt=2 / b-pyramid / bframe-bias=0 / rc-lookahead=25 / lookahead-slices=5 / scenecut=40 / no-intra-refresh / ctu=64 / min-cu-size=8 / rect / no-amp / max-tu-size=32 / tu-inter-depth=1 / tu-intra-depth=1 / limit-tu=0 / rdoq-level=2 / dynamic-rd=0.00 / no-ssim-rd / signhide / no-tskip / nr-intra=0 / nr-inter=0 / no-constrained-intra / strong-intra-smoothing / max-merge=3 / limit-refs=3 / limit-modes / me=1 / subme=3 / merange=57 / temporal-mvp / weightp / no-weightb / no-analyze-src-pics / deblock=0:0 / sao / no-sao-non-deblock / rd=3 / no-early-skip / rskip / no-fast-intra / no-tskip-fast / no-cu-lossless / no-b-intra / no-splitrd-skip / rdpenalty=0 / psy-rd=2.00 / psy-rdoq=0.00 / no-rd-refine / analysis-reuse-mode=0 / no-lossless / cbqpoffs=0 / crqpoffs=0 / rc=crf / crf=25.0 / qcomp=0.60 / qpstep=4 / stats-write=0 / stats-read=0 / ipratio=1.40 / pbratio=1.30 / aq-mode=1 / aq-strength=1.00 / cutree / zone-count=0 / no-strict-cbr / qg-size=32 / no-rc-grain / qpmax=51 / qpmin=0 / no-const-vbv / sar=1 / overscan=0 / videoformat=5 / range=0 / colorprim=1 / transfer=1 / colormatrix=1 / chromaloc=0 / display-window=0 / max-cll=0,0 / min-luma=0 / max-luma=255 / log2-max-poc-lsb=8 / vui-timing-info / vui-hrd-info / slices=1 / no-opt-qp-pps / no-opt-ref-list-length-pps / no-multi-pass-opt-rps / scenecut-bias=0.05 / no-opt-cu-delta-qp / no-aq-motion / no-hdr / no-hdr-opt / no-dhdr10-opt / analysis-reuse-level=5 / scale-factor=0 / refine-intra=0 / refine-inter=0 / refine-mv=0 / no-limit-sao / ctu-info=0 / no-lowpass-dct / refine-mv-type=0 / copy-pic=1

Operating system and version (e.g., Ubuntu 16.04 LTS, macOS 10.13 High Sierra, Windows 10 Creators Update):
Win10

[mduell]

You used different settings. This is why an Activity Log is required for support requests. Please read How-to get an activity log? for details on how and why this should be provided.

[eusi]

You used different settings. This is why an Activity Log is required for support requests.

Hi mduell ,
thanks for your answer.

No I did not. I just updated to 1.1.1 and used the same custom present. Never touched this for years.

(Sadly I have no log for 1.0.7 and 1.1.1 anymore :-/ )

*EDIT*
I found the log in Roaming... I will post it soon

[rollin_eng]

Could you please post your HB logs, instructions can be found here:

https://handbrake.fr/docs/en/latest/hel ... y-log.html

[eusi]

Thanks god to the logs in Roaming!

Log for 1.0.7
https://pastebin.com/XmCKsWE7

Log for 1.1.1
https://pastebin.com/FYETnKhZ


Updated 1.0.7 link.

[mduell]

Frame threads changed.

[eusi]

Frame threads changed.

I did not change the frames, probably it was changed by Handbrake update?

Does the frame-threads really cause this larger file?

I will add it via extra options "frame-threads=5" and transcode it again...


P.S. I went through the entire logs of 1.0.7 and 1.1.1, differences:
* some lib versions were different of course, like dvdnav 5.0.1 vs 5.0.3
* HEVC encoder version 2.1 vs. 2.6
* [Windows][GCC 5.3.1][64 bit] 8bit vs. [Windows][GCC 7.3.0][64 bit] 8bit+10bit+12bit
* 1.0.7 used "OpenCL": false, while this flag was missing for 1.1.1
* frame threads 5 vs 4
* bias was missing vs. 5.00
Further potential setting changes for that impact?

[eusi]

So, I transcoded the video anew with frame-threads=5, to get the same frame-threads like in the 1.0.7.
Result: Still same size, 1100 MB = larger than with 1.0.7... => Changing Frame Threads has no impact to the file size.

Log for 1.1.1 (new):
https://pastebin.com/WmNNwVEY

Log for 1.0.7:
https://pastebin.com/XmCKsWE7

What now? Why are the files (transcoded with HB 1.1.1) like 10% larger than with 1.0.7 and how to fix it?

[Ritsuka]

If you think the file is too big, use a higher RF. x265 2.1 to 2.6 is a bug jump, you can expect it to produce the same file. If you want to compare the quality, you should check the ssim or encode two clips at the same bitrate and use your eyes.

[mduell]

If you want to compare the quality, you should check the ssim or encode two clips at the same bitrate and use your eyes.

Be sure to tune for SSIM, so the psychovisual model is nerfed.

[eusi]

Thanks @Ritsuka and @mduell for your answers!

If you want to compare the quality, you should check the ssim or encode two clips at the same bitrate and use your eyes.

Be sure to tune for SSIM, so the psychovisual model is nerfed.

I don't understand "nerfed" in this context. Do you mean, if I tune for "none", than the psycho-visual optimizations are active. And in your opinion the psycho-visual optimizations are not that good any more?


I did some tests (same custom preset for all tests [CRF25 + no tune], with noted adjustments):
Movie via HB 1.0.7 has 980 MB
Movie via HB 1.1.1: 1100 MB
Movie via HB 1.1.1 + CRF26: 970 MB
Movie via HB 1.1.1 + CRF25 SSIM: 1160 MB
Movie via HB 1.1.1 + CRF26 SSIM: 1020 MB
My eyes were not able to nominate a winner regarding best looking (all looked different but none of them way better than another one). The movie is probably also not the best reference, because its quality is not great.

Another test with another movie (better quality) brought different results in size. The CRF26 SSIM was a little bit smaller (3%) here than the CRF26 none, but without SSIM it looked a bit better.


Now I am confused...
Do you think SSIM brings better quality or better size? Or is it just for metrics and use it as a rating (as a measurement tool for checking similarity to the original video)?

[Ritsuka]

No, SSIM is a way to measure quality, you will find it in the activity log when he encode is done (if you enabled it); vmaf is another (possibly better) algorithm to measure quality, but HandBrake doesn't support it.

[eusi]

No, SSIM is a way to measure quality, you will find it in the activity log when he encode is done (if you enabled it); vmaf is another (possibly better) algorithm to measure quality, but HandBrake doesn't support it.

Yes, that is what I have thought when i defined it as a "rating ([...] for checking similarity to the original video)". Thanks for your confirmation!


So, right now, my conclusion is as follow:

My config/preset:
h265, 30fps peak, CRF25, 2pass, Slow
Extra options: strong-intra-smoothing=1:me=1:lookahead-slices=5:rd=3:psy-rdoq=0.00:qpmax=51

To get the same size and quality like I did with HB 1.0.7,
I needed to adjust the CRF to 26 for HB 1.1.1 (due to HEVC updates).

Enabling SSIM allowed me to get a nice rating via metrics, but it made the video also smoother (which is not good in this case, more like smudgy), so I prefer to tune to "none".


@mduell
I would be happy, if you could answer my last question to you anyway. I am really interested in.

[mduell]

Thanks @Ritsuka and @mduell for your answers!

If you want to compare the quality, you should check the ssim or encode two clips at the same bitrate and use your eyes.

Be sure to tune for SSIM, so the psychovisual model is nerfed.

I don't understand "nerfed" in this context. Do you mean, if I tune for "none", than the psycho-visual optimizations are active. And in your opinion the psycho-visual optimizations are not that good any more?

Nerfed here means disabled. The psy-vis optimizations are great, they make the picture look better to actual people, but they cause a reduction in objective metrics like SSIM.

If you want to compare by watching the movie, choose whatever tune is appropriate for the content, if any, or none.
If you want to compare by an objective quality metric like SSIM, tune SSIM so the encoder can do the best job at that metric rather than the best job at making the movie look good.

Extra options: strong-intra-smoothing=1:me=1:lookahead-slices=5:rd=3:psy-rdoq=0.00:qpmax=51

Where did you find these extra settings? Some blogger on the internet? I'd disregard them and stick to the preset/tune system, developed by actual encoder experts.

[eusi]

Thanks mduell for your answer!

Extra options: strong-intra-smoothing=1:me=1:lookahead-slices=5:rd=3:psy-rdoq=0.00:qpmax=51

Where did you find these extra settings? Some blogger on the internet? I'd disregard them and stick to the preset/tune system, developed by actual encoder experts.

Actually that are my settings which I ve been using for some years. I like the trade-off between good quality and the small size.

[mduell]

Given the changes in x265 over the last "years", they're almost certainly a bad idea if you've been using them that long. Much better off with the preset/tune system which are set up for the current version of x265.

[eusi]

Given the changes in x265 over the last "years", they're almost certainly a bad idea if you've been using them that long. Much better off with the preset/tune system which are set up for the current version of x265.

Yea you are right. I played a lot with the settings in the last days.
My new favorite setting is just the "H.265 MKV 1080p30"-preset, but changed the CRF to 26. This looks like a fine trade of size/quality.