Videotoolbox quality

HandBrake for Mac support
Forum rules
An Activity Log is required for support requests. Please read How-to get an activity log? for details on how and why this should be provided.
Post Reply
allbundy
Posts: 9
Joined: Sun Oct 30, 2016 6:12 am

Videotoolbox quality

Post by allbundy »

Hello everyone,

how is your general impression with the videotoolbox?

I must say that I am really impressed by my latest test with my Macbook Air M1.

I did try the Videotoolbox option now and then, but quality wise, it was no option to consider for me.

But today I decided to give it a go again and - Wow i am really impressed.

My current 1080p Bluray conversions, I did with x265 on quality medium and an RF of 21.

That also lines my basemark for quality lookups with different encodes.

I then did some runs with Blurays, trailers and demos with Videotoolbox to get about the same filesize / videobitrate that I am getting with my settings above.

I stopped at CQ60 for the 8bit VT (and CQ30 for 10bit HDR VT)

When i compared screenshots from the same timestamps of the movie/trailers/demos I could not see much of a difference. I f you look close enough one could spot that VT might be slightly blurrier, but only by a little margin in still photos/screenshots.

conversion is done with about 160frames per seconds.

What is your experience?
p_alex_r
Posts: 10
Joined: Mon Aug 16, 2021 11:28 am

Re: Videotoolbox quality

Post by p_alex_r »

Hi

I know there are plenty of people on this forum that don't like the output of VT and prefer only the software encoders, but with my M1 Mini, I've found the VT H.265 10bit output to be more than good enough for my viewing needs. I did quite a bit of testing and found my sweet spot to be CQ45 for most (pretty much indistinguishable from the BluRay H.264 original), sometimes CQ40 depending on original quality/file size. CQ35 was producing noticeably worse results (even to my untrained eye) in most circumstances. I was pleasantly surprised by this, I always found VT H.264 encoding on my old Mac Pro not worth the effort, only software produced good results.

The killer reason for using VT was obviously the speed. The software H.265 10bit encoder yields around 20fps for me on the M1 Mini, the VT around 180-200fps. I've quite a BluRay collection and literally, life is too short to covert all the H.264 content to H.265 without using the VT. Back of a [Censored] packet calculations suggested conversion time using the software encoder would be around 20 years on the Mini.

The driving force for the conversion is disk space usage. I know adding additional disks to a setup is cheap (around £15/Tb) but due to house-space/noise/heat/power consumption/cabling concerns, I'm only prepared to replace existing disks in my RAID6 storage array with larger disks and not adding additional ones. This effectively means that cost is in excess of £100/Tb for me for the additional capacity. Halving the space requirements of the collection is worth the effort, but only if VT is used.
p_alex_r
Posts: 10
Joined: Mon Aug 16, 2021 11:28 am

Re: Videotoolbox quality

Post by p_alex_r »

Lol, sorry forum, I wasn't being rude, the [Censored] was not meant as recognised by a filter ... maybe "back of an envelope" or "back of a napkin" would mean more to more of the world ...https://en.wikipedia.org/wiki/Back-of-t ... alculation
mduell
Veteran User
Posts: 8187
Joined: Sat Apr 21, 2007 8:54 pm

Re: Videotoolbox quality

Post by mduell »

If you can't see the difference, you could probably increase the RF or speed quite a bit with x265 and also not see a difference.
rachel
Novice
Posts: 71
Joined: Thu Mar 15, 2007 7:34 pm

Re: Videotoolbox quality

Post by rachel »

I never had any joy out of videotoolbox before; with only a bitrate control you could basically choose between OK-looking and gigantic, or ok-sized and appalling to look at.

I tried again with my new M1Pro MacBook Pro. It lets you select a Constant Quality setting now.

I usually use the AppleTV 4K preset. Switching to videotoolbox it automatically defaulted to CQ 46. I found the resulting encodes slightly smaller and slightly worse-looking than using x265 at that preset's default RF 24.

I bumped CQ up to 50, and the results are slightly larger, and to my eyes better, than x265 RF 24. x265 has been introducing - at least to my eyes - an almost subliminal and tolerable fuzziness, and hevc_videotoolbox is not. I'm now just using that.

I did try splitting the difference and using 48, but the results seemed identical to 50 in both size and quality, so I'm guessing there are fewer actual distinct quality steps than the 0-100 scale implies.

Not entirely coincidentally I also moved to using FFmpeg deinterlacer filter 'estdif' instead of letting Handbrake do its Yadif-Bob. The results look quite a lot better (going via huge utvideo lossless intermediate format) but this cpu-bound deinterlace step on its own is slower on this MBP than the subsequent actual encode, meaning the total encode time is remaining more or less constant, at least when it's interlaced content. This... seems to be the law for me. Or possibly a curse... 😀
p_alex_r
Posts: 10
Joined: Mon Aug 16, 2021 11:28 am

Re: Videotoolbox quality

Post by p_alex_r »

rachel wrote: Sun Nov 07, 2021 7:02 pm I did try splitting the difference and using 48, but the results seemed identical to 50 in both size and quality, so I'm guessing there are fewer actual distinct quality steps than the 0-100 scale implies.
totally agree Rachel, while the CQ scale is 0-100, the end results look like 5 is the quanti and 0-20 would be just as effective for the range available ie. anything 45-49 yields the same results.
bekloptopus
Posts: 1
Joined: Thu Dec 16, 2021 12:33 pm

Re: Videotoolbox quality

Post by bekloptopus »

Hi,

is there a post around here that illustrates how CQ is supposed to scale with RF?

Thanks ...
majortom67
Posts: 18
Joined: Thu Nov 20, 2014 8:59 am

Re: Videotoolbox quality

Post by majortom67 »

bekloptopus wrote: Thu Dec 16, 2021 12:35 pm Hi,
is there a post around here that illustrates how CQ is supposed to scale with RF?
Thanks ...
This is my historycal question. If CPU encoding way differs from VT encoding way (as I believe), there's no way to have a table of conversion. Basically HB turn RF 22 into CQ (or QC, never remember) into 44 but the first gives good results while the second, regardless positive opinions around here, IMHO, NO! I've done lots of "movie-stops" (high and low action scenes) and checked details in both situations and CQ at 44 gives poor quality results if compared to RF22. Then I encoded a whole episode of Yellostone S04 (E01) via CPU/RF44/medium speed and got 1,78 gbyte filesize (h264!!!). In order to get the same file size supposing to get the same quality, I got CQ to be 59 through encodings from CQ 40 to CQ 60 (see my latest post). But this could be an interesting thought if the two encoders work in the same way and as I already said, I don't believe it is.
In my almost 20-ennial experience in converting movies you can only choose after some trial and error. My final opinion is: not less than 50, and if you do not have size problems, better 55-59.
Falcon81
Posts: 6
Joined: Thu Mar 10, 2022 9:57 am

Re: Videotoolbox quality

Post by Falcon81 »

majortom67 wrote: Wed Jan 26, 2022 6:38 pm
bekloptopus wrote: Thu Dec 16, 2021 12:35 pm Hi,
is there a post around here that illustrates how CQ is supposed to scale with RF?
Thanks ...
This is my historycal question. If CPU encoding way differs from VT encoding way (as I believe), there's no way to have a table of conversion. Basically HB turn RF 22 into CQ (or QC, never remember) into 44 but the first gives good results while the second, regardless positive opinions around here, IMHO, NO! I've done lots of "movie-stops" (high and low action scenes) and checked details in both situations and CQ at 44 gives poor quality results if compared to RF22. Then I encoded a whole episode of Yellostone S04 (E01) via CPU/RF44/medium speed and got 1,78 gbyte filesize (h264!!!). In order to get the same file size supposing to get the same quality, I got CQ to be 59 through encodings from CQ 40 to CQ 60 (see my latest post). But this could be an interesting thought if the two encoders work in the same way and as I already said, I don't believe it is.
In my almost 20-ennial experience in converting movies you can only choose after some trial and error. My final opinion is: not less than 50, and if you do not have size problems, better 55-59.

Is your recommendation based on 4K content only? If I was working with 1080p, are you suggesting I use CQ 50 or should I consider rolling it back to 46?
majortom67
Posts: 18
Joined: Thu Nov 20, 2014 8:59 am

Re: Videotoolbox quality

Post by majortom67 »

Nope, it's based on 2K and I'm now stuck on 56. For 4K I would choose 52/54.
Post Reply