Help with Optimal H.265 1080p settings

[cutter]

Hello, first time poster, with very specific questions, so let me please give you some background info that will hopefully allow you to give me the best advice based on my particular needs.

My movie/TV series library is currently spanned across two Synology NAS's with a total usable capacity of approx 80 TB. I am really strict about video quality and my library is kept as Remuxed mkv files (in the case of Bluray movies) or untouched Video_TS folders (in the case of the relatively few remaining DVDs that I still haven't upgraded to Bluray). Given the size of my collection and the fact that I just can't afford the purchase of yet another NAS, I unfortunately came to the decision to convert my library to H.265 in order to reclaim much needed disk space capacity.

I started experimenting with Handbrake 1.0.7 on a random movie (As Above, So Below) Remux mkv file size 23.5GB, which when passed through Handbrake with the H.265 1080p default preset, changing the Video Section Quality RF slider from 22 to RF16 and the Audio Section to Auto Passthrough for all streams, produced an acceptable quality on my 55" TV without noticeable artefacts. The output file size was 7.5 GB, so it seems that I can get acceptable picture quality for roughly 1/3 of the original file size, which is pretty enticing.

On to my questions:

1. Based on my needs (output file size approx 1/3 of original file size), can you recommend a guide someone has posted or the settings that can help my fine tune even more the picture of the resulted file? Yes, I know one set of settings will not fit all and that's why I provided the above background info to provide you an idea of my viewing preferences and expectations when it comes to quality. By the way, I don't mind about the time each encode takes, as long as I can get good picture quality for approximately 1/3 of the original remux mkv file.

2. Strictly speaking, I don't have to start converting my library right away, since my free space is enough to accommodate me for a few more months. Do you believe that any significant changes in either H.265 or Handbrake itself are forthcoming, so that it would be wise on my part to wait until, say the end of the year, for updated versions?

3. Probably a dumb question, but I have the option to do the encodes on a Mac or a Windows PC. I don't suppose there are fundamental differences in Handbrake that would make me prefer one platform over the other, right?

I'm sorry for the long post and I thank you in advance for your replies and recommendations.

[Woodstock]

Optimal settings vary on a per-source basis. There are sources where your target of 66% size reduction is incompatible with your RF quality selection. Then there sources where you can get 90% reduction with that RF. And does your playback hardware all support h.265?

Optimization of x264 and x265 encoders may come in how quick they are, but there is not that much chance that they'll get significantly better compression for the same settings. It may be that a hardware implementation comes around that gets the compression efficiency of x264/x265 while giving you the speed (I don't see that right now).

Choice of Windows/Mac/Linux, beyond your personal preferences, comes down to what the processor is capable. Apple does not keep up with the absolute latest processors in their machines, only now introducing processors that were available in Windows machines a year ago. The operating system makes a difference in the graphic user interface, but the "working parts" of the encoders depend more on the processor capabilities.

[cutter]

Thank you for your answer Woodstock.

Let me rephrase then my original question; given that my goal is to remain true to keeping a good picture quality (not being able to discern differences from a normal viewing distance between the original source and the encoded H.265 1080p file), are there glaring omissions on my part or any other changes I should experiment with other than changing the Video Section Quality RF slider from 22 to RF16 and the Audio Section to Auto Passthrough? This preset is acceptable in my eyes, are there any other settings I can change from the default Handbrake H.265 MKV 1080p preset that would make the picture quality even better? I know there are no guarantees, I'm just looking for something that screams that should be changed based on my needs that I haven't mentioned.

As far as compatibility with playback devices is concerned, right now only one of my media players can handle a H.265 file with these settings, which is a Vero 4K; my Raspberry Pi 3 stutters, but the cost of replacing a Pi with another Vero 4K is insignificant compared to the savings or the possible expense of buying another NAS.

Thank you again for your time and answers.

[BradleyS]

You're on the right path. Nothing in particular stands out as needing tweaked beyond what you mention.

[mduell]

it seems that I can get acceptable picture quality for roughly 1/3 of the original file size, which is pretty enticing.

This will be highly source dependent!

[nhyone]

My personal opinion, if you want to use HEVC, you might want to wait for x265 2.4 with its tweaked lamda tables for increased details, and use 10-bit encoding for that edge over H.264.

Your 2/3 size saving is not due to HEVC alone. If you use x264 veryslow preset (but set ref=3, bframes=5; something sane), CRF 20, you should get roughly 50% smaller too.

[Dysthymia]

I came across someone who had done some testing to find optimal H.265 settings, and they ended up with a 1.5 Mbit 1080p result that was stunning to me compared to the RF/Very Slow settings I'd been tweaking.

https://forum.doom9.org/showthread.php? ... ost1807501
Here is the video result:
http://www.msystem.waw.pl/x265/bb1500.mkv

Using these settings I have had great results with 10+ Mbit 1080p x264 source files.


They mention bumping the bitrate up to 2000 kbps for additional quality but I've been so happy with 1500 I haven't bothered. 48ish minute source files have been taking my i5-4430 ~26 hours, and an i5-4670 and i7-6700T both about ~19ish hours.

Edit: I just realized from the linked video that multiple versions of the 10 and 12-bit x264 and x265 dlls are linked here: http://www.msystem.waw.pl/x265/

[shadejinx]

I don't think your extra options are working. I'll bet if you check your encode logs, you'll find an error in the second pass. I think you'll need to convert them to Handbrake format:

Code: Select all

rc-lookahead=120:bframes=12:ref=6:subme=7

I removed two of the options:

-p9 sets the Placebo preset, which can be done on the slider
-F1 makes part of the encode single-threaded for a slight compression benefit. This is a HUGE performance penalty and not related to video quality. More info here: https://x265.readthedocs.io/en/default/ ... me-threads

Personally, I think these settings are insane. I know this is all very subjective, but realistically, you will spend a week encoding one video what will stutter on fast actions scenes or camera pans. I have an i7-6770K and it took 22 minutes to encode a 30-second preview clip.

Interesting side note: It appears that 2-pass encoding throws away my AC3 passthrough audio track. I generally use CQ encoding, so I never knew that.

[mduell]

1) It's unlikely a fixed bitrate target is going to be optimal across a range of content; particularly where an RF target hits the same average bitrate.

2) You're not specifying the options in an HB compatible way.

[rollin_eng]

One thing to think about is the cost and time associated with all this, buying more HD space may actually be cheaper than encoding your whole library.

[Dysthymia]

I don't think your extra options are working. I'll bet if you check your encode logs, you'll find an error in the second pass. I think you'll need to convert them to Handbrake format:

Code: Select all

rc-lookahead=120:bframes=12:ref=6:subme=7

I removed two of the options:

-p9 sets the Placebo preset, which can be done on the slider
-F1 makes part of the encode single-threaded for a slight compression benefit. This is a HUGE performance penalty and not related to video quality. More info here: https://x265.readthedocs.io/en/default/ ... me-threads

Personally, I think these settings are insane. I know this is all very subjective, but realistically, you will spend a week encoding one video what will stutter on fast actions scenes or camera pans. I have an i7-6770K and it took 22 minutes to encode a 30-second preview clip.

Interesting side note: It appears that 2-pass encoding throws away my AC3 passthrough audio track. I generally use CQ encoding, so I never knew that.

Yep, you're right. It takes that entire string and quotes it as an unknown option.

If I'm as happy as I am with a simple VerySlow preset and this low fixed bitrate, I'm curious to see what the result will be with these options actually working (placebo indeed!). Thank you for bringing that to my attention.

Why is there a different syntax for the command line and the Advanced options in the GUI? Can you point me to a reference fully explaining the GUI syntax?

I sympathize with your view on the viability of the settings. Obviously I didn't fully understand them and I agree with your removal of the single-threaded and placebo options. I'm not sure I agree with your statement regarding fast action/camera pans -- I feel like those always look bad due to the nature of ~24 FPS video. But I'm listening if you want to enlighten me.

[rollin_eng]

The problem with recommending these settings to the OP is that he/she has 80 TB of Blu Ray rips to encode.

At 30 GB per movie that's 2500+ movies (or TV show equivalent).

If you could get the above settings to encode 1 movie per day it would take over 6 years of non stop encoding to finish the project

[Dysthymia]

You're right, rollin_eng. I think x264 is the only option to get through that amount of video in a reasonable amount of time. I was overly excited about the H.265 settings post I'd found.

[nhyone]

Dysthymia, with your suggested settings, I would keep the source! (Because you will re-encode again in the future.)

[Dysthymia]

Dysthymia, with your suggested settings, I would keep the source! (Because you will re-encode again in the future.)

Actually, nhyone, despite the fact that my improper syntax failed to enable the extra options I was trying to use, I'm quite happy with the results. As one user in the above linked thread stated, H.265 shines in bitrate constrained scenarios, and using 10-bit (perhaps to a lesser extent 12-bit) helps to eliminate any banding that would normally occur.

In further testing with a different source (> 13 Mbit x264), I tried 10-bit H.264 with an RF of 20, Preset Placebo, tuned to Film, Profile High10, Level 5.2 -- I still prefer the above settings (with the bitrate bumped to 2000 kbps).

[nhyone]

Actually, nhyone, despite the fact that my improper syntax failed to enable the extra options I was trying to use, I'm quite happy with the results.

It's okay to use whatever settings you like. It is always amazing when you start, it is only later that you realize the flaws. That's why I suggest to keep the source until you have a better understanding.

As mduell has pointed out, using a fixed average bitrate is not the right foot forward.

In your linked article, that user seems to be using PSNR/SSIM to judge the video quality? His log shows this:

Code: Select all

x265 [warning]: --psnr used with psy on: results will be invalid!
x265 [warning]: --tune psnr should be used if attempting to benchmark psnr!

One way to start is to look at the x265 presets, what they set and compare each changed settings. You might get an idea what matters and what not, and if the quality is worth the time.

[rollin_eng]

Why constrain your video bitrate if you don't have to?

[Dysthymia]

Why constrain your video bitrate if you don't have to?

Because my first priority is getting file size reduced. I have become frustrated with trying to find the optimal balance with the RF slider. I don't have a good test video (I wasn't able to download any of the Big Buck Bunny videos someone linked me to elsewhere on this forum) and the 1.5 Mbit option I've found keeps quality more than acceptable in the vast majority of cases and filesizes in strict control.

[rollin_eng]

Because my first priority is getting file size reduced. I have become frustrated with trying to find the optimal balance with the RF slider. I don't have a good test video (I wasn't able to download any of the Big Buck Bunny videos someone linked me to elsewhere on this forum) and the 1.5 Mbit option I've found keeps quality more than acceptable in the vast majority of cases and filesizes in strict control.

Just chose a RF that will give you a similar bitrate in your test video then use that.

You may even save more space as you may need a lower bitrate than what you are selecting.

[Dysthymia]

Just chose a RF that will give you a similar bitrate in your test video then use that.

You may even save more space as you may need a lower bitrate than what you are selecting.

It's been my experience that the RF degrades quality unacceptably before the filesize becomes acceptably small. Maybe the particular H.265 settings I stumbled on hit a sweet spot for me and the source files I'm using that isn't perceived as well by others. I feel like we'd have to run encodes on the same source file to get on the same page here, comparing the results.

[rollin_eng]

If you run an RF encode and it averages 1.5Mbps it will be the same as running a 2 pass 1.5Mbps encode.

[Dysthymia]

Any idea what range of RF settings I should try in both x264 and x265 to try'n end up with that bitrate? I know it is dependent on the source, but... average action movie, say?

[rollin_eng]

Unless you NEED 1.5Mbps don't worry about it so much, if you pick an average action movie encode it with an RF that gets you your desired quality.

[WhatZit]

Any idea what range of RF settings I should try in both x264 and x265 to try'n end up with that bitrate? I know it is dependent on the source, but... average action movie, say?

You could always create 30-second previews of carefully-chosen representative scenes to see the correlation between bitrate and Constant Quality RF's.

In fact, if you were REALLY desperate to hit an "exact" bitrate target without multi-pass, you could set HandBrake to 60 previews, make a 5-second sample of every 5th preview, then average the bitrates.

Either way, eventually, you'll figure out a median RF for each type of content.

[Dysthymia]

Any idea what range of RF settings I should try in both x264 and x265 to try'n end up with that bitrate? I know it is dependent on the source, but... average action movie, say?

You could always create 30-second previews of carefully-chosen representative scenes to see the correlation between bitrate and Constant Quality RF's.

In fact, if you were REALLY desperate to hit an "exact" bitrate target without multi-pass, you could set HandBrake to 60 previews, make a 5-second sample of every 5th preview, then average the bitrates.

Either way, eventually, you'll figure out a median RF for each type of content.

I did use Previews when I started trying to use RF instead of bitrate to test rollin_eng's suggestion, but I'm not aware of how to set up previews in the way you suggest.

One thing I've learned is that (using 12-bit H.265) with a grainy source, when I go 2-pass/bitrate, I need to leave Encoder Tune set to None to retain that noisy detail. I tuned Grain and it looked worse. But when going constant quality (trying 30 and 32) if I leave the Encoder Tune at None it is blocky and horrible looking, so I need to set Encoder Tune to Grain to retain noisy detail. Why is that?