Page 1 of 1

Athlon x2 3800+ - 186 benchmarks @ diff settings...

Posted: Mon Sep 13, 2010 12:39 am
by Arlki
PC: Athlon x2 3800+; 3GB DDR400 CL3 ram.

VIDEOS: 3 80 second crops from 3 live action videos; 2 @ 640x480, 1 @ 512*384;
all less than dvd quality with some blocking.

S/W: HB 3496.

Constant Settings: bframes=16trellis=0:ref=2; Constant quality rf=19.
(NOTE! Trellis=2 was used for the subq=10 tests as per handbrake tooltip)

Variable Settings: 8x8dct=0/1; me=dia/hex/umh; subq=0, 2, 3, 4, 5, 6, 10
(NOTE! UMH only used for the subq=10 tests)

I want to convert some videos to perfect copies of size 0kb in 0 seconds of cpu time. Since that's impossible, I've been trying to work out which settings to use that will give me a compromise that I'm happy with. To do this I've generated one base encode and one top encode for each file, then normalized the data so that I can say "These settings will give me 85% max compression and 75% highest quality in 1.4x base time".


NOTE!
=====
I KNOW these results won't hold for every video or even one particular video - the idea is to find some settings that I can be reasonably confident are a good compromise. The only alternative would be to do multiple encodes then check each one - which would defeat the point of minimising cpu time.

I also KNOW that SSIM values are not exact and may not tell the whole story - but unless someone (and his / her friends) wants to rate each (RF=19) video by hand on his/her own, and then justify why their score should be chosen as the final score for each of the 186 encodes... If you can suggest a different metric, I'll look at that.

I'm hoping that some people might find that these results hold for them; but the source material obviously plays a major part. Those who are interested
might want to try running some of the low-cpu suggestions with a short clip from some of their own sources just to see how well - or not! - the results
hold up for them.

Anyway...
=====================================
BASE ENCODE SETTINGS: bframes=16:subq=0:trellis=0:me=dia:8x8dct=0:ref=1
SOURCE : FPS > SSIM > SIZE /bytes
FILE 1 : 91.7 >17.717 >22259606
FILE 2 : 94.3 >18.298 >12676318
FILE 3 : 101.3 >15.632 >22547999
<- Note the variance in SSIM despite the "constant quality" setting.
<Yes, I know it's a rate factor; I'm just pointing it out.>
=====================================

TOP ENCODE SETTINGS: bframes=16:subq=10:trellis=2:me=umh:ref=3
SOURCE : FPS > SSIM > SIZE
FILE 1 : 5.6 > 20.137 > 20124749
-> +2.42db, -9.59%
FILE 2 : 6.91 > 20.218 > 11442790
-> +1.92db, -9.73%
FILE 3 : 5.87 > 17.751 > 18278853
-> +2.12db, -18.93%
=====================================


Anyway...
The encode name follows the format <Reference Frames> <8x8DCT> <Motion Estimation> <SubQ>.
I think that the processed version is more useful, but here's the raw data first.

PLEASE FEEL FREE TO SKIP THE RAW STUFF!! m(_ _)m
-------------------------------------------------------------------------

Code: Select all

Encode Name : <fps1, ssim1, size1> <fps2, ssim2, size2> <fps3, ssim3, size3>
------------------------------------------------------------------------------------------------------------------
1 0 Dia 2: <64.406, 19.210, 22876346> <65.230, 19.468, 12502104> <71.306, 16.380, 19624507>
1 0 Dia 3: <57.756, 19.393, 22634694> <57.424, 19.677, 12389068> <60.229, 16.906, 19399121>
1 0 Dia 4: <48.806, 19.445, 22560083> <46.787, 19.729, 12335966> <48.810, 16.921, 19306440>
1 0 Dia 5: <38.976, 19.464, 22502994> <37.030, 19.757, 12293160> <39.547, 16.958, 19140617>
1 0 Dia 6: <32.543, 19.415, 22625820> <32.101, 19.641, 12343081> <33.445, 16.923, 19413161>
1 0 Hex 2: <60.262, 19.196, 22804456> <59.991, 19.483, 12464862> <65.235, 16.378, 19571381>
1 0 Hex 3: <55.633, 19.383, 22581828> <54.704, 19.688, 12362564> <57.333, 16.901, 19337468>
1 0 Hex 4: <47.346, 19.433, 22498088> <46.723, 19.739, 12313936> <48.474, 16.915, 19249554>
1 0 Hex 5: <38.535, 19.452, 22445090> <36.956, 19.767, 12270201> <38.310, 16.952, 19084031>
1 0 Hex 6: <31.581, 19.403, 22564391> <30.941, 19.651, 12326597> <32.586, 16.921, 19379998>
1 1 Dia 2: <62.224, 19.486, 20997862> <61.828, 19.538, 12215393> <67.662, 16.647, 19251541>
1 1 Dia 3: <57.547, 19.721, 20920285> <55.267, 19.858, 12289299> <57.208, 17.381, 19259035>
1 1 Dia 4: <48.867, 19.767, 20989503> <47.369, 19.902, 12260065> <48.728, 17.390, 19182993>
1 1 Dia 5: <39.699, 19.779, 20921267> <37.513, 19.919, 12177313> <38.455, 17.411, 18974965>
1 1 Dia 6: <30.240, 19.785, 20442999> <29.639, 19.797, 11844621> <31.247, 17.508, 19028174>
1 1 Hex 2: <59.356, 19.469, 20967821> <58.599, 19.552, 12201286> <64.054, 16.646, 19237745>
1 1 Hex 3: <53.877, 19.706, 20890235> <48.870, 19.867, 12275203> <54.190, 17.377, 19217441>
1 1 Hex 4: <44.964, 19.753, 20956368> <43.703, 19.910, 12241584> <44.064, 17.385, 19143214>
1 1 Hex 5: <36.407, 19.765, 20887486> <34.610, 19.928, 12154272> <35.331, 17.407, 18945934>
1 1 Hex 6: <29.106, 19.769, 20416264> <28.571, 19.807, 11838701> <30.216, 17.506, 19011139>
2 0 Dia 2: <54.477, 19.205, 22783068> <54.809, 19.496, 12370391> <58.404, 16.354, 19517494>
2 0 Dia 3: <50.806, 19.382, 22505275> <50.244, 19.689, 12228407> <50.875, 16.863, 19193329>
2 0 Dia 4: <39.757, 19.446, 22322949> <39.612, 19.757, 12053883> <40.326, 16.875, 19055962>
2 0 Dia 5: <29.569, 19.463, 22269957> <28.499, 19.787, 11996168> <29.846, 16.918, 18896959>
2 0 Dia 6: <25.275, 19.435, 22435708> <24.870, 19.686, 12125022> <26.075, 16.902, 19234436>
2 0 Hex 2: <51.678, 19.193, 22714871> <51.909, 19.515, 12348934> <55.309, 16.354, 19466699>
2 0 Hex 3: <48.126, 19.370, 22444988> <47.493, 19.700, 12212723> <48.771, 16.855, 19134218>
2 0 Hex 4: <37.865, 19.433, 22266387> <37.874, 19.770, 12047059> <38.554, 16.869, 19009046>
2 0 Hex 5: <28.446, 19.452, 22204730> <27.513, 19.795, 11982801> <28.890, 16.914, 18853927>
2 0 Hex 6: <24.463, 19.421, 22372976> <24.011, 19.696, 12113783> <25.331, 16.899, 19198916>
2 1 Dia 2: <53.847, 19.484, 21074702> <53.971, 19.574, 12153390> <57.212, 16.616, 19260225>
2 1 Dia 3: <49.707, 19.710, 20917972> <48.365, 19.868, 12166314> <48.792, 17.337, 19111247>
2 1 Dia 4: <38.960, 19.769, 20905979> <38.475, 19.930, 12017656> <38.622, 17.348, 19001757>
2 1 Dia 5: <29.110, 19.781, 20843266> <27.803, 19.947, 11928870> <28.737, 17.371, 18812497>
2 1 Dia 6: <23.769, 19.801, 20419330> <22.412, 19.833, 11697081> <24.287, 17.490, 18933358>
2 1 Hex 2: <51.906, 19.469, 21039567> <51.602, 19.593, 12134206> <55.216, 16.619, 19242146>
2 1 Hex 3: <48.010, 19.697, 20884398> <46.790, 19.877, 12153513> <47.246, 17.334, 19081345>
2 1 Hex 4: <37.900, 19.755, 20869852> <37.433, 19.938, 12005262> <37.633, 17.342, 18971773>
2 1 Hex 5: <28.558, 19.766, 20795445> <27.262, 19.957, 11914361> <28.266, 17.367, 18781098>
2 1 Hex 6: <23.499, 19.788, 20385155> <22.950, 19.844, 11693731> <24.243, 17.488, 18916783>
3 0 Dia 2: <49.993, 19.207, 22806084> <50.479, 19.507, 12382058> <53.662, 16.354, 19520438>
3 0 Dia 3: <46.624, 19.378, 22513996> <46.444, 19.692, 12230349> <46.769, 16.851, 19167928>
3 0 Dia 4: <36.477, 19.444, 22329089> <36.506, 19.760, 12044834> <36.541, 16.863, 19026570>
3 0 Dia 5: <26.815, 19.461, 22271262> <25.966, 19.786, 11996826> <27.013, 16.909, 18865805>
3 0 Dia 6: <23.279, 19.435, 22435860> <22.883, 19.689, 12118733> <23.785, 16.898, 19201942>
3 0 Hex 2: <46.910, 19.196, 22740488> <47.116, 19.524, 12355995> <50.450, 16.357, 19463453>
3 0 Hex 3: <43.735, 19.367, 22456520> <43.427, 19.706, 12213989> <44.305, 16.845, 19110861>
3 0 Hex 4: <34.618, 19.432, 22275751> <34.552, 19.772, 12033644> <35.052, 16.858, 18980269>
3 0 Hex 5: <25.845, 19.453, 22209968> <24.958, 19.798, 11980151> <26.217, 16.906, 18822369>
3 0 Hex 6: <22.481, 19.427, 22375701> <22.091, 19.701, 12105611> <23.219, 16.894, 19165156>
3 1 Dia 2: <48.725, 19.485, 21127961> <48.779, 19.585, 12168263> <51.820, 16.617, 19301050>
3 1 Dia 3: <45.069, 19.708, 20947651> <44.265, 19.865, 12165576> <44.066, 17.328, 19101446>
3 1 Dia 4: <35.336, 19.767, 20924746> <35.095, 19.933, 12014943> <34.761, 17.338, 18994694>
3 1 Dia 5: <26.420, 19.778, 20854203> <25.260, 19.949, 11924680> <25.987, 17.362, 18788740>
3 1 Dia 6: <21.919, 19.803, 20432216> <21.496, 19.836, 11699297> <22.326, 17.487, 18917156>
3 1 Hex 2: <47.231, 19.470, 21089000> <47.286, 19.603, 12144387> <50.556, 16.619, 19278794>
3 1 Hex 3: <43.522, 19.695, 20914312> <42.774, 19.879, 12154516> <42.889, 17.323, 19060875>
3 1 Hex 4: <34.579, 19.752, 20887765> <34.126, 19.942, 12004905> <34.298, 17.334, 18957336>
3 1 Hex 5: <25.939, 19.766, 20819529> <24.742, 19.958, 11913841> <25.732, 17.359, 18758440>
3 1 Hex 6: <21.773, 19.787, 20395827> <21.145, 19.850, 11704505> <22.230, 17.482, 18898652>
------------------------------------------------------------------------------------------------------------------

Semi-Processed Data
====================

Please carry on skipping these unless you want to check my maths or something! Provided for completeness only.
<The processed FPS is easiest to read - if an encode gets a score of 2 then it takes twice as long as the base encode.>

Code: Select all

Encode Name :- Normalized FPS, SSIM db Increase, Percentage File Decrease
=====================================================================================
1 0 Dia 2: <1.42, 1.493, -2.8> <1.45, 1.170, 1.4> <1.42, 0.7, 13.0>
1 0 Dia 3: <1.59, 1.676, -1.7> <1.64, 1.379, 2.3> <1.68, 1.3, 14.0>
1 0 Dia 4: <1.88, 1.728, -1.3> <2.01, 1.431, 2.7> <2.07, 1.3, 14.4>
1 0 Dia 5: <2.35, 1.747, -1.1> <2.55, 1.459, 3.0> <2.56, 1.3, 15.1>
1 0 Dia 6: <2.82, 1.698, -1.6> <2.94, 1.343, 2.6> <3.03, 1.3, 13.9>
1 0 Hex 2: <1.52, 1.479, -2.4> <1.57, 1.185, 1.7> <1.55, 0.7, 13.2>
1 0 Hex 3: <1.65, 1.666, -1.4> <1.72, 1.390, 2.5> <1.77, 1.3, 14.2>
1 0 Hex 4: <1.94, 1.716, -1.1> <2.02, 1.441, 2.9> <2.09, 1.3, 14.6>
1 0 Hex 5: <2.38, 1.735, -0.8> <2.55, 1.469, 3.2> <2.64, 1.3, 15.4>
1 0 Hex 6: <2.90, 1.686, -1.4> <3.05, 1.353, 2.8> <3.11, 1.3, 14.1>
1 1 Dia 2: <1.47, 1.769, 5.7> <1.52, 1.240, 3.6> <1.50, 1.0, 14.6>
1 1 Dia 3: <1.59, 2.004, 6.0> <1.71, 1.560, 3.1> <1.77, 1.7, 14.6>
1 1 Dia 4: <1.88, 2.050, 5.7> <1.99, 1.604, 3.3> <2.08, 1.8, 14.9>
1 1 Dia 5: <2.31, 2.062, 6.0> <2.51, 1.621, 3.9> <2.63, 1.8, 15.8>
1 1 Dia 6: <3.03, 2.068, 8.2> <3.18, 1.499, 6.6> <3.24, 1.9, 15.6>
1 1 Hex 2: <1.54, 1.752, 5.8> <1.61, 1.254, 3.7> <1.58, 1.0, 14.7>
1 1 Hex 3: <1.70, 1.989, 6.2> <1.93, 1.569, 3.2> <1.87, 1.7, 14.8>
1 1 Hex 4: <2.04, 2.036, 5.9> <2.16, 1.612, 3.4> <2.30, 1.8, 15.1>
1 1 Hex 5: <2.52, 2.048, 6.2> <2.72, 1.630, 4.1> <2.87, 1.8, 16.0>
1 1 Hex 6: <3.15, 2.052, 8.3> <3.30, 1.509, 6.6> <3.35, 1.9, 15.7>
2 0 Dia 2: <1.68, 1.488, -2.4> <1.72, 1.198, 2.4> <1.73, 0.7, 13.4>
2 0 Dia 3: <1.80, 1.665, -1.1> <1.88, 1.391, 3.5> <1.99, 1.2, 14.9>
2 0 Dia 4: <2.31, 1.729, -0.3> <2.38, 1.459, 4.9> <2.51, 1.2, 15.5>
2 0 Dia 5: <3.10, 1.746, 0.0> <3.31, 1.489, 5.4> <3.39, 1.3, 16.2>
2 0 Dia 6: <3.63, 1.718, -0.8> <3.79, 1.388, 4.3> <3.88, 1.3, 14.7>
2 0 Hex 2: <1.77, 1.476, -2.0> <1.82, 1.217, 2.6> <1.83, 0.7, 13.7>
2 0 Hex 3: <1.90, 1.653, -0.8> <1.98, 1.402, 3.7> <2.08, 1.2, 15.1>
2 0 Hex 4: <2.42, 1.716, 0.0> <2.49, 1.472, 5.0> <2.63, 1.2, 15.7>
2 0 Hex 5: <3.22, 1.735, 0.2> <3.43, 1.497, 5.5> <3.51, 1.3, 16.4>
2 0 Hex 6: <3.75, 1.704, -0.5> <3.93, 1.398, 4.4> <4.00, 1.3, 14.9>
2 1 Dia 2: <1.70, 1.767, 5.3> <1.75, 1.276, 4.1> <1.77, 1.0, 14.6>
2 1 Dia 3: <1.84, 1.993, 6.0> <1.95, 1.570, 4.0> <2.08, 1.7, 15.2>
2 1 Dia 4: <2.35, 2.052, 6.1> <2.45, 1.632, 5.2> <2.62, 1.7, 15.7>
2 1 Dia 5: <3.15, 2.064, 6.4> <3.39, 1.649, 5.9> <3.52, 1.7, 16.6>
2 1 Dia 6: <3.86, 2.084, 8.3> <4.21, 1.535, 7.7> <4.17, 1.9, 16.0>
2 1 Hex 2: <1.77, 1.752, 5.5> <1.83, 1.295, 4.3> <1.83, 1.0, 14.7>
2 1 Hex 3: <1.91, 1.980, 6.2> <2.01, 1.579, 4.1> <2.14, 1.7, 15.4>
2 1 Hex 4: <2.42, 2.038, 6.2> <2.52, 1.640, 5.3> <2.69, 1.7, 15.9>
2 1 Hex 5: <3.21, 2.049, 6.6> <3.46, 1.659, 6.0> <3.58, 1.7, 16.7>
2 1 Hex 6: <3.90, 2.071, 8.4> <4.11, 1.546, 7.8> <4.18, 1.9, 16.1>
3 0 Dia 2: <1.83, 1.490, -2.5> <1.87, 1.209, 2.3> <1.89, 0.7, 13.4>
3 0 Dia 3: <1.97, 1.661, -1.1> <2.03, 1.394, 3.5> <2.17, 1.2, 15.0>
3 0 Dia 4: <2.51, 1.727, -0.3> <2.58, 1.462, 5.0> <2.77, 1.2, 15.6>
3 0 Dia 5: <3.42, 1.744, -0.1> <3.63, 1.488, 5.4> <3.75, 1.3, 16.3>
3 0 Dia 6: <3.94, 1.718, -0.8> <4.12, 1.391, 4.4> <4.26, 1.3, 14.8>
3 0 Hex 2: <1.95, 1.479, -2.2> <2.00, 1.226, 2.5> <2.01, 0.7, 13.7>
3 0 Hex 3: <2.10, 1.650, -0.9> <2.17, 1.408, 3.6> <2.29, 1.2, 15.2>
3 0 Hex 4: <2.65, 1.715, -0.1> <2.73, 1.474, 5.1> <2.89, 1.2, 15.8>
3 0 Hex 5: <3.55, 1.736, 0.2> <3.78, 1.500, 5.5> <3.86, 1.3, 16.5>
3 0 Hex 6: <4.08, 1.710, -0.5> <4.27, 1.403, 4.5> <4.36, 1.3, 15.0>
3 1 Dia 2: <1.88, 1.768, 5.1> <1.93, 1.287, 4.0> <1.95, 1.0, 14.4>
3 1 Dia 3: <2.03, 1.991, 5.9> <2.13, 1.567, 4.0> <2.30, 1.7, 15.3>
3 1 Dia 4: <2.59, 2.050, 6.0> <2.69, 1.635, 5.2> <2.91, 1.7, 15.8>
3 1 Dia 5: <3.47, 2.061, 6.3> <3.73, 1.651, 5.9> <3.90, 1.7, 16.7>
3 1 Dia 6: <4.18, 2.086, 8.2> <4.39, 1.538, 7.7> <4.54, 1.9, 16.1>
3 1 Hex 2: <1.94, 1.753, 5.3> <1.99, 1.305, 4.2> <2.00, 1.0, 14.5>
3 1 Hex 3: <2.11, 1.978, 6.0> <2.20, 1.581, 4.1> <2.36, 1.7, 15.5>
3 1 Hex 4: <2.65, 2.035, 6.2> <2.76, 1.644, 5.3> <2.95, 1.7, 15.9>
3 1 Hex 5: <3.53, 2.049, 6.5> <3.81, 1.660, 6.0> <3.94, 1.7, 16.8>
3 1 Hex 6: <4.21, 2.070, 8.4> <4.46, 1.552, 7.7> <4.56, 1.9, 16.2>
------------------------------------------------------------------------------------------------------------------

Results Coming Up...
====================

The processed FPS is easiest to read - if an encode gets a score of 2 then it takes twice as long as the base encode.
For the SSIM and SIZE data I worked out the range between the base and top encodes; and then I worked out what percentage of that range each encode gave.
<Obviously, then, the base encode would score 0, and though not as obviously, the top encode did get 100%.> This was done so that files which were easy to compress / easy to keep at high quality wouldn't overwhelm the others.

Processed Data / Results...
=========================

I've :
(a) left the FPS in their normalized form as it makes it clear roughly how long an encode will take compared to the base encode.
(b) sorted the results by FPS as well.
(c) added a score that tells us how well the encode combines size reduction with ssim quality.
<Basically the score is equal to normalized ssim * normalized file reduction / 100>.

Code: Select all

Encode Name: <FPS, SSIM, SIZE, SCORE>
=======================================
[b]1 0 Dia 2: <1.43, 52.64, 17.90, 9.426>[/b]
1 1 Dia 2: <1.50, 61.86, 57.89, 35.814>
1 0 Hex 2: <1.55, 52.68, 20.45, 10.772>
1 1 Hex 2: <1.58, 61.85, 58.85, 36.403>
1 0 Dia 3: <1.64, 67.07, 26.49, 17.767>
[b]1 1 Dia 3: <1.69, 82.20, 57.05, 46.895>[/b]
2 0 Dia 2: <1.71, 52.65, 23.76, 12.508>
1 0 Hex 3: <1.71, 67.04, 28.52, 19.117>
2 1 Dia 2: <1.74, 61.97, 58.30, 36.131>
2 0 Hex 2: <1.81, 52.82, 25.80, 13.625>
2 1 Hex 2: <1.81, 62.14, 59.51, 36.981>
1 1 Hex 3: <1.83, 82.09, 58.23, 47.795>
3 0 Dia 2: <1.86, 52.87, 23.06, 12.191>
2 0 Dia 3: <1.89, 66.45, 34.46, 22.899>
3 1 Dia 2: <1.92, 62.19, 56.75, 35.294>
2 1 Dia 3: <1.96, 81.53, 61.56, 50.193>
3 1 Hex 2: <1.98, 62.33, 58.18, 36.261>
1 1 Dia 4: <1.98, 83.74, 57.35, 48.027>
3 0 Hex 2: <1.99, 53.06, 25.23, 13.388>
2 0 Hex 3: <1.99, 66.35, 36.29, 24.076>
1 0 Dia 4: <1.99, 68.92, 29.82, 20.550>
1 0 Hex 4: <2.01, 68.84, 31.82, 21.906>
2 1 Hex 3: <2.02, 81.46, 62.67, 51.049>
3 0 Dia 3: <2.05, 66.26, 34.47, 22.839>
3 1 Dia 3: <2.15, 81.31, 61.20, 49.758>
1 1 Hex 4: <2.16, 83.61, 58.68, 49.061>
3 0 Hex 3: <2.18, 66.25, 36.26, 24.021>
3 1 Hex 3: <2.22, 81.29, 62.33, 50.673>
2 0 Dia 4: <2.40, 68.70, 43.10, 29.607>
2 1 Dia 4: <2.48, 83.59, 66.62, 55.691>
1 1 Dia 5: <2.49, 84.53, 62.28, 52.644>
1 0 Dia 5: <2.49, 70.25, 33.16, 23.295>
2 0 Hex 4: <2.51, 68.65, 44.53, 30.571>
1 0 Hex 5: <2.52, 70.17, 35.12, 24.646>
2 1 Hex 4: <2.54, 83.44, 67.76, 56.538>
3 0 Dia 4: <2.62, 68.53, 43.47, 29.795>
1 1 Hex 5: <2.70, 84.43, 63.66, 53.745>
3 1 Dia 4: <2.73, 83.46, 66.46, 55.466>
3 0 Hex 4: <2.76, 68.50, 44.97, 30.805>
3 1 Hex 4: <2.79, 83.35, 67.60, 56.341>
1 0 Dia 6: <2.93, 67.01, 27.76, 18.605>
1 0 Hex 6: <3.02, 66.99, 29.43, 19.713>
[b]1 1 Dia 6: <3.15, 84.02, 78.32, 65.806>
1 1 Hex 6: <3.27, 83.94, 79.03, 66.341>[/b]
2 0 Dia 5: <3.27, 70.13, 46.73, 32.768>
2 1 Dia 5: <3.35, 84.41, 71.48, 60.338>
2 0 Hex 5: <3.38, 70.05, 48.44, 33.935>
2 1 Hex 5: <3.42, 84.32, 72.86, 61.437>
3 0 Dia 5: <3.60, 69.94, 46.93, 32.825>
3 1 Dia 5: <3.70, 84.27, 71.61, 60.340>
3 0 Hex 5: <3.73, 69.99, 48.68, 34.071>
3 1 Hex 5: <3.76, 84.21, 72.68, 61.202>
2 0 Dia 6: <3.77, 67.74, 38.02, 25.754>
2 0 Hex 6: <3.89, 67.67, 39.58, 26.785>
2 1 Hex 6: <4.06, 84.56, 84.17, 71.178>
2 1 Dia 6: <4.08, 84.58, 83.42, 70.557>
[b]3 0 Dia 6: <4.10, 67.73, 38.44, 26.036>
3 0 Hex 6: <4.24, 67.76, 40.02, 27.121>[/b]
[b]3 1 Dia 6: <4.37, 84.61, 83.28, 70.470>
3 1 Hex 6: <4.41, 84.56, 83.86, 70.907>
3-1-UMH-10: <15.80,100.00,100.00,100.000>[/b]
------------------------------------------------------------------------------------------------------------------

CONCLUSIONS
============

1 0 Dia 2: <1.43, 52.64, 17.90, 9.426>
<- As expected, 1-0-Dia-2 is quickest, but it doesn't actually do much.

1 1 Dia 3: <1.69, 82.20, 57.05, 46.895>
<- Only 1.7x as CPU intensive as the top encode, yet it gives 82% of the SSIM score.
This is my new setting for live-action encodes that I'm not too bothered about keeping.

1 1 Dia 6: <3.15, 84.02, 78.32, 65.806>
1 1 Hex 6: <3.27, 83.94, 79.03, 66.341>

<- Both give quality and compression close to 3-1-Dia-6 but in roughly 66% of the time.

3 0 Dia 6: <4.10, 67.73, 38.44, 26.036>
3 0 Hex 6: <4.24, 67.76, 40.02, 27.121>

:shock: ... I won't be using these two in a hurry...

3 1 Dia 6: <4.37, 84.61, 83.28, 70.470>
3 1 Hex 6: <4.41, 84.56, 83.86, 70.907>

<- Note we're still ~15% from the top encode in ssim and size...
But it's taken under a third of the time of a proper top encode to get there...

3 1 UMH 10: 3-1-UMH-10: <15.80,100.00,100.00,100.000>
<- Top scores for the top encode; but it takes almost 16x as long as base! :shock:

Re: Athlon x2 3800+ - 186 benchmarks @ diff settings...

Posted: Fri Sep 17, 2010 9:29 am
by Arlki
Just posting this so that I won't keep on thinking about doing so! m(_ _)m
Repeated the above procedure with 3 clean 1280*720 animated sources.

Once again, all 3 base encodes (REF=1, 8x8DCT=0, ME=Dia, SubQ=0, Trellis=0) were normalized to give a score of :
1.00 (31.583 FPS), 0.00db SSIM, 0.00% Size, 0.00 Score.

The most cpu intensive encodes (REF=4, 8x8DCT=1, ME=UMH, SubQ=10, Trellis=2) were averaged to give the following scores :
9.11 (3.468FPS), 100% SSIM increase (0.80db), 100% size reduction (16.92%), Score = 100.00

(And once again, I know that I'm sample-size limited, and that SSIM is only an approximation of the total quality. For that reason the subq=6 and subq=10 SSIM value should be taken in interest only or for those interested in maximum compression.)

The listing below shows everything else, sorted by FPS.

Encode Name = REF 8x8DCT ME SUBQ

Encode Name -> FPS, % Max SSIM Increase, % Max Size Reduction, Score
========================================================
2 0 Dia 2 -> 1.40 , 51.93 , 111.00 , 57.64
2 1 Dia 2 -> 1.43 , 64.16 , 103.41 , 66.35
2 0 Hex 2 -> 1.46 , 54.61 , 113.74 , 62.11
3 0 Dia 2 -> 1.47 , 55.01 , 120.38 , 66.21
2 1 Hex 2 -> 1.49 , 66.24 , 105.51 , 69.89
4 0 Dia 2 -> 1.50 , 55.32 , 121.13 , 67.01
3 1 Dia 2 -> 1.51 , 67.34 , 112.37 , 75.67
4 1 Dia 2 -> 1.53 , 67.62 , 113.40 , 76.69
3 0 Hex 2 -> 1.54 , 57.79 , 122.75 , 70.93
4 0 Hex 2 -> 1.58 , 58.10 , 123.65 , 71.84 -> Maximum size reduction
3 1 Hex 2 -> 1.58 , 69.73 , 114.83 , 80.07
4 1 Hex 2 -> 1.62 , 70.05 , 115.80 , 81.12
2 0 Dia 3 -> 1.73 , 81.96 , 105.05 , 86.09
2 1 Dia 3 -> 1.81 , 105.27 , 82.98 , 87.36
3 0 Dia 3 -> 1.82 , 84.32 , 114.11 , 96.22
2 0 Hex 3 -> 1.84 , 84.43 , 106.48 , 89.90
4 0 Dia 3 -> 1.85 , 84.46 , 115.22 , 97.32
3 1 Dia 3 -> 1.89 , 107.64 , 92.15 , 99.19
2 1 Hex 3 -> 1.92 , 106.89 , 84.23 , 90.03
4 1 Dia 3 -> 1.93 , 107.78 , 93.27 , 100.52
3 0 Hex 3 -> 1.94 , 86.98 , 115.43 , 100.40
4 0 Hex 3 -> 1.97 , 87.07 , 116.45 , 101.39
3 1 Hex 3 -> 2.04 , 109.54 , 93.61 , 102.55
4 1 Hex 3 -> 2.05 , 109.61 , 94.62 , 103.71
2 0 Dia 4 -> 2.08 , 86.50 , 106.52 , 92.14
2 1 Dia 4 -> 2.16 , 109.63 , 85.13 , 93.32
2 0 Hex 4 -> 2.19 , 89.27 , 107.51 , 95.98
3 0 Dia 4 -> 2.21 , 89.06 , 115.39 , 102.76
4 0 Dia 4 -> 2.24 , 89.23 , 116.46 , 103.91
2 1 Hex 4 -> 2.28 , 111.54 , 85.82 , 95.72
3 1 Dia 4 -> 2.29 , 112.15 , 94.11 , 105.54
3 0 Hex 4 -> 2.32 , 91.85 , 116.46 , 106.98
4 1 Dia 4 -> 2.33 , 112.16 , 95.20 , 106.78
4 0 Hex 4 -> 2.37 , 91.88 , 117.34 , 107.81
2 0 Dia 5 -> 2.39 , 89.36 , 105.15 , 93.96
3 1 Hex 4 -> 2.42 , 114.26 , 95.00 , 108.55
4 1 Hex 4 -> 2.46 , 114.43 , 96.17 , 110.05
2 1 Dia 5 -> 2.46 , 111.44 , 83.51 , 93.06
2 0 Hex 5 -> 2.50 , 91.82 , 105.72 , 97.08
3 0 Dia 5 -> 2.55 , 91.96 , 114.67 , 105.44
2 1 Hex 5 -> 2.59 , 113.31 , 84.07 , 95.26
4 0 Dia 5 -> 2.62 , 91.91 , 115.89 , 106.52
2 0 Dia 6 -> 2.66 , 70.08 , 65.26 , 45.73
3 1 Dia 5 -> 2.67 , 114.07 , 93.19 , 106.30
3 0 Hex 5 -> 2.69 , 94.63 , 115.24 , 109.05
4 1 Dia 5 -> 2.73 , 114.17 , 94.41 , 107.79
4 0 Hex 5 -> 2.75 , 94.78 , 116.41 , 110.33
2 0 Hex 6 -> 2.77 , 73.49 , 65.84 , 48.39
3 1 Hex 5 -> 2.80 , 116.20 , 93.58 , 108.73
2 1 Dia 6 -> 2.81 , 92.14 , 58.03 , 53.46
3 0 Dia 6 -> 2.83 , 73.38 , 74.16 , 54.42
4 1 Hex 5 -> 2.86 , 116.37 , 94.85 , 110.37 -> Maximum SSIM-only quality and score
4 0 Dia 6 -> 2.91 , 73.84 , 75.05 , 55.41
2 1 Hex 6 -> 2.95 , 95.50 , 58.17 , 55.55
3 0 Hex 6 -> 2.99 , 76.84 , 74.68 , 57.38
3 1 Dia 6 -> 3.02 , 95.65 , 66.66 , 63.77
4 0 Hex 6 -> 3.05 , 77.13 , 75.58 , 58.29
4 1 Dia 6 -> 3.09 , 96.12 , 67.65 , 65.03
3 1 Hex 6 -> 3.15 , 99.12 , 67.02 , 66.43
4 1 Hex 6 -> 3.22 , 99.51 , 67.77 , 67.44
4 1 UMH 10 -> 9.11 , 100.00 , 100.00 , 100.00

Of interest this time is that where the top encode won in (SSIM only) quality and % size reduction with the live action encodes above, the HD animated sources gave rise to encodes at lower settings that beat it in either size or ssim quality, though never both. Unlike the live-action stuff, the best setting for maximum size reduction was in fact 4-0-Hex-2, and took only 1.58/9.11 = 17% of the time that the top encode did.

In the interest of completeness I will state that a similar 186 benchmark test that downsampled some HD animated material to SD didn't show this; and so I wonder if perhaps the psy-opts that were activated at subq=10 tried to show detail that wouldn't normally be considered correct within an animated source (noise or variation within large areas of the same identical color) but which would be either significant in (a) live action material or (b) unlikely to occur in the downsampled animated stuff (due to the noise reduction that downsampling provides). This could explain why the top encode didn't have such an advantage this time.

Further work that I think would be interesting (but probably won't be able to carry out) :
Testing at even higher REF numbers.

Re: Athlon x2 3800+ - 186 benchmarks @ diff settings...

Posted: Fri Sep 17, 2010 1:51 pm
by Deleted User 11865
Arlki wrote:Encode Name -> FPS, % Max SSIM Increase, % Max Size Reduction, Score
========================================================
[…]
4 1 Dia 2 -> 1.53 , 67.62 , 113.40 , 76.69
[…]
4 0 Hex 2 -> 1.58 , 58.10 , 123.65 , 71.84 -> Maximum size reduction
As you pointed out, 4 0 Hex 2 gives you the smallest file at RF19. What about 4 1 Dia 2? It's a tad faster, gives a higher SSIM and a larger file size. But what would happen if you used a slightly different RF, trying to match the SSIM produced by 4 0 Hex 2?

One possibility is that, for the same quality/SSIM that you get from 4 0 Hex 2, the file size would be lower. Let's assume it's true: that gives us settings that are faster, and also result in a lower file size for the same quality. Wouldn't you say (assuming this holds true for a reasonable number of sources) that 4 1 Dia 2 are better settings?

Re: Athlon x2 3800+ - 186 benchmarks @ diff settings...

Posted: Fri Sep 17, 2010 8:03 pm
by Arlki
Rodeo wrote:
Arlki wrote:Encode Name -> FPS, % Max SSIM Increase, % Max Size Reduction, Score
========================================================
[…]
4 1 Dia 2 -> 1.53 , 67.62 , 113.40 , 76.69
[…]
4 0 Hex 2 -> 1.58 , 58.10 , 123.65 , 71.84 -> Maximum size reduction
As you pointed out, 4 0 Hex 2 gives you the smallest file at RF19. What about 4 1 Dia 2? It's a tad faster, gives a higher SSIM and a larger file size. But what would happen if you used a slightly different RF, trying to match the SSIM produced by 4 0 Hex 2?

One possibility is that, for the same quality/SSIM that you get from 4 0 Hex 2, the file size would be lower. Let's assume it's true: that gives us settings that are faster, and also result in a lower file size for the same quality. Wouldn't you say (assuming this holds true for a reasonable number of sources) that 4 1 Dia 2 are better settings?
Despite my failure when it came to assuming that rf=19 would give roughly the same quality across different encode settings, I am trying not to assume anything if at all possible. What you have stated assumes that the encode settings all react in the same way to a change in rf. It might in fact turn out that at a different value of rf another set of settings (say 3 0 Dia 1) are optimal, or that 4-0-hex-2 once again held an advantage. Finding out whether or not that is the case would require me to benchmark each group of setting at a whole range of rf values both above and below the target - probably far above and below - and so would drastically increase the size of the required data set and the time involved to make / process it.

Partly because of this reason I will not be looking into this, but I do understand what you are saying.

Secondly (as mentioned in another post) I chose rf=19 because it was slightly better than the recommended level of 20, but still far away enough from the archival target of 15 that I could anticipate some amount of size reduction. Whilst I have tried to keep my results accurate for source types (sd live, hd animated, sd animated) there is the chance that by approaching or going over 20 I might one day create an encode where certain scenes were of an unacceptable quality. Given that I delete the original once the encode has finished, I would rather stick to rf=19.

Thirdly; if I did all that testing and then admitted to doing so, I fear that the amount of posts scorning me for testing so many different settings and not just sticking to magic-preset-collection alpha would cause the internet to explode and thus doom mankind for an eternity. :)