Another Handbrake test

[Nobozmeg]

I know I’m not the first one to test the advenced h264 functions but I think my test is somwhat different than the others becouse my concerns are different. When encodeing, my no.1 concern is the size of the output file. I want to achive the smallest possible file size with still a decent quality. And I’d like to have it in an acceptable time.
In my trials I encoded a 6:10 min. long DVD chapter. I took note not only of the results, but also the time it took and the average fps. My system configuration: 2 Ghz dual core CPU, Win 7 64 bit.

1.: bitrate: 1024; 2 pass
ref: 3; mixed, b-frames: 3; adaptive b: optimal; direct pred.: auto; wighted b; pyramidal b; motion esti.: umh; subp.: 7; range: 24; analysis: some; 8x8 DCT; CABAC; Trellis: 1; Psy-rd: 1,0; 0,1
1st pass: avg fps: ~19; 2nd pass: 9,85 Time: 22 min Size: 54,1 MB Quality: ok (a few minor artifacts)
SSIM Mean Y:0.9449298
PSNR Mean Y:39.233 U:45.455 V:46.645 Avg:40.493 Global:39.846 kb/s:1024.48
These are the standard, these are the settings (except for 0.94-s newly added ones) I use mostly, it results a very acceptable quality and size. Below I write only what I modified.

2.: CQ 52,94%; RF: 24
avg fps: 8,53; Time: 18 min; size: 58 MB Quality: ok, same as above
SSIM Mean Y:0.9466916
PSNR Mean Y:39.399 U:45.566 V:46.747 Avg:40.649 Global:40.070 kb/s:1112.65
I tried to guess a constant quality value that produces a similar output as my standard. It seems my 1st guess was right.

3. Psy-rd: 1,0; 0,0
avg fps: ~9; Time: 17 min; size: 58 MB; Quality: little less good than above
SSIM Mean Y:0.9466916
PSNR Mean Y:39.399 U:45.566 V:46.747 Avg:40.649 Global:40.070 kb/s:1112.65
I was curious of the effect of psy-trellis. It seems it’s good if it’s switched on, even if it slows down the encode a bit.

4. Psy-rd: 1,0; 0,2; no fast P-skip
avg fps: ~9; Time: 18 min; size: 58,5 MB; Quality: better
SSIM Mean Y:0.9469646
PSNR Mean Y:39.426 U:45.584 V:46.766 Avg:40.676 Global:40.090 kb/s:1123.39
Some psy-trellis again and a trial of ’no fast P-skip’, becouse there was blocking on the skies. It seems it doesn’t have that much impact on the encode time.

5.: Psy-rd: 1,0; 0,1; no fast P-skip; analysis: all
avg fps: 8,65; Time: 18 min; size: 58,4 MB; Quality: good
SSIM Mean Y:0.9469504
PSNR Mean Y:39.422 U:45.587 V:46.768 Avg:40.673 Global:40.085 kb/s:1122.43
Some more play with psy-trellis and changeing the analysis to ’all’. Not much visible change.

6.: bitrate: 1280; 2 pass; Psy-rd: 1,0; 0,2
avg fps: 1st pass: ~22; 2nd pass: 9,93; Time: 23 min; size: 65,4 MB; Q: not much better
SSIM Mean Y:0.9492258
PSNR Mean Y:39.669 U:45.730 V:46.895 Avg:40.910 Global:40.285 kb/s:1280.49
I started to search for higher quality CQ values and needed reference.

7.: bitrate: 1536; 2 pass;
avg fps: 1st pass: 17,63; 2nd pass: 8,41; Time: 27 min; size: 76,7 MB Q: good
SSIM Mean Y:0.9524848
PSNR Mean Y:40.026 U:45.951 V:47.093 Avg:41.250 Global:40.644 kb/s:1536.48
It seems high profile settings have greater impact on 2 pass encodings duration than on CQ.

8.: Trellis: 2;
avg fps: 1st pass: 22; 2nd pass: 6,55; Time: 30 min, size: 76,7 MB; Q: good
SSIM Mean Y:0.9527127
PSNR Mean Y:40.039 U:45.944 V:47.083 Avg:41.261 Global:40.655 kb/s:1536.38
An experiment with raising the Trellis. Not much change, except for the time.

9. . CQ 52,94%; RF: 24; audio: AC3
avg fps, time: ~same as in 5; Size: 69,6 MB
SSIM Mean Y:0.9469504
PSNR Mean Y:39.422 U:45.587 V:46.768 Avg:40.673 Global:40.085 kb/s:1122.43
An experiment on how much AC3 audio raises the file size – everywhere else I used Vorbis on 192 kbps. Well, it enlarges the file a lot!

10. . CQ 56,86%; RF: 22;
avg fps: ~8,7; Time: ~18 min; Size: 85,3 MB; Quality: fine
SSIM Mean Y:0.9547446
PSNR Mean Y:40.264 U:46.108 V:47.222 Avg:41.476 Global:40.941 kb/s:1730.37
Searching for another correct CQ setting. This one makes too large file.

11. CQ 54,90%; RF: 23
avg fps: ~8,7; Time: ~18 min; Size: 70,2; Q: good
SSIM Mean Y:0.9508584
PSNR Mean Y:39.833 U:45.836 V:46.990 Avg:41.064 Global:40.505 kb/s:1389.36
This file size is much more friendly.

12. adaptive b: fast; Psy-rd: 1,0; 0,0
avg fps: 8,53; Time: 19 min; Size: 70,8 MB; Q: ok
SSIM Mean Y:0.9505824
PSNR Mean Y:39.840 U:45.742 V:46.927 Avg:41.051 Global:40.503 kb/s:1401.92
I started to turn of the high profile settings to see their impact on time and quality. It seems ’no fast P-skip, analysis: all and psy trellis have more impact on quality than on time, and with 3 B-frames the ’adaptive B: fast’ also doen’t make the encode much faster than the ’optimal’.

13. motion esti.: hex
avg fps: 11; Time: 14 min; Size: 70,7 MB; Q: ok
SSIM Mean Y:0.9504742
PSNR Mean Y:39.832 U:45.730 V:46.919 Avg:41.042 Global:40.498 kb/s:1400.41
This experiment taught me a great lesson: NEVER USE UMH MOTION ESTIMATION!!! It seems umh has a HUGE impact on encode time, but a negligable on quality. This movie looked exactly like the previous one!

14. . CQ 52,94%; RF: 24;
avg fps: 11,5; Time: 14 min; Size: 58,3 MB; Q: still ok
SSIM Mean Y:0.9463469
PSNR Mean Y:39.401 U:45.468 V:46.689 Avg:40.631 Global:40.063 kb/s:1120.32
Just switched back to RF: 24.

15. Psy-rd: 0,0; 0,0;
avg fps: 13,3; Time: 12,5 min; Size: 52,3 MB; Q: surpisingly good
SSIM Mean Y:0.9465178
PSNR Mean Y:39.439 U:45.058 V:46.330 Avg:40.623 Global:40.050 kb/s:983.66
The other lesson: DON’T USE THE ’Psychovisual Rate Distortion’!!!
With it turned off not only the encode time and output size decresed significantly, but the movie also looked better than with that option!

16. ref. frame: 4
subp.: 7; analysis: some; 8x8 DCT; CABAC; Trellis: 1; Psy-rd: 0,0; 0,0
avg fps: 12,7; Time: 13 min; Size: 52,3 MB; Q: ok
SSIM Mean Y:0.9466027
PSNR Mean Y:39.446 U:45.069 V:46.336 Avg:40.631 Global:40.056 kb/s:982.12
It seems raising the ref. frames with one didn’t have that much impact on time. Too bad it didn’t make the movie look much better either.

17. B-frames: 6
avg fps: 12,4; Time: 13 min; Size: 52,2 MB; Q: better
SSIM Mean Y:0.9466065
PSNR Mean Y:39.446 U:45.073 V:46.337 Avg:40.635 Global:40.056 kb/s:981.92
More b-frames made it look a little better.

18. subp.: 9
avg fps: 11,4; Time: 14 min; Size: 51,6 MB; Q: better
SSIM Mean Y:0.9465940
PSNR Mean Y:39.454 U:45.069 V:46.330 Avg:40.642 Global:40.066 kb/s:966.36
Not a huge impact on time, but made the movie look much better.

19. subp.: 7; adaptive b: optimal
avg fps: 9,8; Time: 16 min; Size: 51,7 MB; Q: good
SSIM Mean Y:0.9471569
PSNR Mean Y:39.450 U:45.187 V:46.417 Avg:40.663 Global:40.073 kb/s:969.45
With 6 b-frames optimal adaptive b slows down the encode quite a bit, but the result also looks good.

20. Psy-rd: 0,4; 0,2; adaptive b: fast
avg fps: 10,75; Time: 14 min; Size: 85,9 MB; Q: ok
SSIM Mean Y:0.9461238
PSNR Mean Y:38.979 U:45.606 V:46.837 Avg:40.270 Global:39.662 kb/s:1744.25
My last experiment with psy-rd. Dispite the huge size it didn’t even look really good.

21. Psy-rd: 0,0; 0,2
avg fps: 12; Time: 13 min; Size: 52,2 MB; Q: good
SSIM Mean Y:0.9466065
PSNR Mean Y:39.446 U:45.073 V:46.337 Avg:40.635 Global:40.056 kb/s:981.92
An experiment with psy-trellis alone. It worth to be used, it improved the quality.

22. :CQ 54,90%; RF: 23
ref: 4; mixed, b-frames: 5; adaptive b: optimal; direct pred.: auto; wighted b; pyramidal b; motion esti.: hex; subp.: 9; analysis: all; 8x8 DCT; CABAC; Trellis: 1; Psy-rd: 0,0; 0,2
avg fps: 8,3; Time: 19 min; size: 61,4 MB Quality: nice!
SSIM Mean Y:0.9513193
PSNR Mean Y:39.893 U:45.457 V:46.665 Avg:41.082 Global:40.516 kb/s:1189.64
The datas speak for themselves. This had the best quality of all, even the one with RF: 22 doesn’t look better.

23. motion esti.: umh
avg fps: 6,32; Time: 25 min; size: 61,4 MB Quality: nice (but no noticable improvement)
SSIM Mean Y:0.9513377
PSNR Mean Y:39.895 U:45.460 V:46.673 Avg:41.085 Global:40.517 kb/s:1188.99


Before I draw the clonclusion I need to add a rider: after these tests I thought that the output size with constant quality encodeing is at least somewhat predictable (dispite what is written in the manual). Later when I wanted to encode another movie I tried it with CQ and the result was shocking: with the same settings as in test 22. not only the output files size was huge (the avg bitrate was more than 2000) but also the encode time (the avg fps was around 4,5)!


Conclusion (for thos who don’t want to wade trough the text above):
- I found psychovisual rate distortion bad, it increases file size (a lot) and encode time, but decreses quality.
- However, it worths trieing to use the psychovisual trellis, it can improve the quality.
- The uneven multihexagonal motion estimation is also bad, it increses the encode time a lot, but gives very little quality improvement over the hexagonal me..
- Settings described as high profile, like ’pyramidal b frames’, ’no fast p-skip’, ’analysis: all’ have small impact on encode time, so it worths to leave them checked.
- Constant quality encodeing is totally unpredictable in every respect! (Even the quality varies somewhat as the same RF values can result totally different avg bitrates, there can be more than 600 kbps difference!) Use it only if you don’t give a s**t about nor the output size nor the encode time, nor the quality! (But in that case why would you encode at all?)

[jbrjake]

- Constant quality encodeing is totally unpredictable in every respect! (Even the quality varies somewhat as the same RF values can result totally different avg bitrates, there can be more than 600 kbps difference!) Use it only if you don’t give a s**t about nor the output size nor the encode time, nor the quality! (But in that case why would you encode at all?)

a) You clearly have no idea what you're talking about if you think "the quality varies" because "the same RF values can result [sic] totally different avg bitrates"
b) So essentially you're saying you know better than the developers of x264, who have CRF as the default rate control method for their presets?

[Nobozmeg]

I know the bitrate not equals quality (if I didn't know that, I wouldn't have bothered with the above experiments). And no, I don't know the encoding better the developers of x264, but I do think that their choice of presets aren't really good. And I did quite a few encodins (and no, I'm not telling that I did more than you or the codec developers), and my experience is that while a movie with relatively low bitrate can be free of blatant artefacts and look smooth, higher bitrates almost always have nicer, louder colors. I noticed it in the experiment too, for example when a CRF encode resulted a file with about 911 kbps bitrate, its colors looked less nice than the others. (And anyway if the same settings result a 60 MB output file from a 6:10 min long movie chunk, and about 100 MB from a a 6:30 chunk of another movie it seems too big difference to me.)

[k209]

Hi. We were in the same boat a couple of months ago and after several test I decided no more tests for me .
Every time I got more confused then investigated the matter, but never found a "real" difference or a winner preset.

My conclusion at that time were:

• More references frames help you pack more quality in same file size. But do not get too enthusiastic, I'm talking about going from 2 ref frames to 6 ref frames.
  
  B-pyramid option isn't working because is not yet compatible con mbtree (which is more useful). You'll never notice the difference
  
  I'm using 3 b-frames cause in my test it decrease (slightly) the SSIM value and I'm also under the impression that it is only for animated content. But I might be wrong about. I'd like someone to clarify the matter.
  
  Multi-hexagon give me more quality (slightly) in SSIM and PSNR values at a reduced file size. But again, I might be wrong since I've never found a "visual" difference.
  
  Subq give me more quality (slightly) and increased (slightly) the file size. Never understood how subq worked. If you're getting good results with same enconding speed at subq=9 please share them.(But in a excel table in order to be easy to read).
  
  I'm always under the impression that the encoder will do a better job if it exactly know how motion is moving along the frames. That's why I always choose me=umh me_range=24 subq=8 to let the encoder analyse freely the picture and then use the restricted kpbs in a better way.
  
  Never used Trellis 2, but it seems to provide with good results. Anyone?

Now I'm using the following setting for all my DVD-backups and I'm getting different file sizes depending on the material. My SSIM is always more than 0.97 for what is worth and I'm getting from 2GB to 4GB per 2 hour DVD full 720x480.

CRF = 20 / Reference Frames=6 / Mixed Reference=CHECK / B-FramesS=3 / Adaptive B-Frame:OPTIMAL / Direct Prediction:AUTO / Weighted B-Frames: CHECK / B-Pyramid: UNCHECKED / Motion Estimation Method=UNEVEN MULTI-HEXAGON / Subpixel Motion Estimation=8 / Motion Estimation Range=24 / Analysis=ALL / 8x8 DCT=Check / CABAC=Check / Trellis=1 / No Fast-P-Skip=UNCHECKED / No DCT-Decimate=UNCHECKED / Deblocking=DEFAULT

Check my threat before you reply to know what I'm talking about:
http://forum.handbrake.fr/viewtopic.php?f=6&t=13877

Happy encodings,