H.265 (x265) vs H.265 (AMD VCE) Encoding sizes

[Cougar281]

I've been using Handbrake for some time and just upgraded from v1.0.3 to 1.2.1, so I've been out of date for quite some time. Before the upgrade, I renamed my original profiles file to preserve it and create a new one, and upon launching the new version, proceeded to customize a stock one for DVDs and test. In doing so, I discovered the new 'H.265 (AMD VCE)' Codec and decided to see what it was about. After reading about it, I decided to give it a shot (I have a RX 480 GPU) and proceeded to fall out of my chair with how fast it encoded - it took ~45 minute TV Episode DVD rips from about 35 minutes to about 4! I had encoded a few on 1.0.3 earlier today, which came out around 350MB or so from a 1.9GB or so source, but on 1.2.1 with the AMD VCE codec, they came out around 600MB and 900MB - sill nice improvement over the source files, but quite a difference from the ones I had done earlier today on the old version, so I decided to do a bunch of testing.

I took a BR Rip of Jurassic World Fallen Kingdom and used that for some baseline testing. I created a SHQ profile that was as close to the same settings as I had before as I could get (I didn't really have THAT much customization - under 'video', I set it to 18RF and the Encoder set to slow, using AMD VCE the encoder preset it set to balanced) and re-encoded it several times with the only change between them being the audio - passthrough or encoded. Using 'Auto Passthrough', as I have always done, the re-encoded file came out to 10.2GB using the AMD VCE codec. The smallest was AAC encoded audio at 6.9GB and then FLAC 24-bit took up the middle at 8.81GB. The encode I did some time back on 1.0.3 was 6.1GB. So that's a pretty huge difference.

Everything I tried to encode, regardless of what audio I used or other settings I tweaked was much larger than precious encodes, so I shut down Handbrake, renamed my new profiles file and reverted to the original, which had to be updated on launch, but that went fine and my old profiles were there.

To to test and see if I missed some setting hidden somewhere in creating my new profile, I selected my original custom 'SHQ encode' profile and changed ONLY the Video Codec from 'H.265 (x265)' to 'H.265 (AMD VCE)' and did another encode. This one came out within about 100MB of the one generated using the same (AMD VCE) codec as before, which for these purposes, is close enough to be considered the same.

I then re-selected my old SHQ profile and didn't change anything, which used the 'H.265 (x265)' codec, which of course took a LOT longer, but upon finishing, the finished file was within about 200MB of the original encoded one, which again, in my opinion, is close enough to be considered 'the same size' when coming from a 27GB file.

So it would seem there is a HUGE difference between the two codecs. All other settings being equal, should there be that substantial of a difference? Are there any settings or extra options for the AMD VCE codec that would maintain quality, hopefully at least some of the speed and get the file size back down to near the 'x265' codec size, or is that just the cost of using GPU encoding (It'll be a lot faster but no where near as small)?

[mduell]

The quality scales aren't remotely the same, so this is not meaningful. The quality scales aren't even the same for a given encoder across different settings.

To compare you'd need to use some objective quality metric (PSNR or SSIM, and use the tune for each) that you equalize and compare the bitrate, or do the same bitrate (and keep lowering it until you can see a difference) and watch the result to see where you're happy.