Estimate file size

Random chit-chat and anything that doesn't belong elsewhere
Post Reply
rollin_eng
Veteran User
Posts: 4840
Joined: Wed May 04, 2011 11:06 pm

Estimate file size

Post by rollin_eng »

So over a year ago I had a thought to see if it was possible/reasonable to predict the final size of a single pass cq encode.

I have now got around to doing a few tests and here are my results.

First a few caveats:

I am only using blu-ray rips as this is all I am interested in.
I am not considering audio.
I am using HandbrakeCLI in Windows.

Method:

Its pretty simple really, I am taking the blu-ray and encoding each chapter as a separate file.
Each chapter will obviously have its own duration and bitrate.
That data is then used to calculate the average bitrate up to any chapter brake in the movie.

Settings:

A simple batch file was created like below, then a final encode of the complete movie was done.

Handbrakecli -i I:\BluRay\Deadpool -o d:\temp\deadpool\1.m4v -Z "Fast 1080p30" -c 1 -t 130 2>>d:\temp\deadpool\1.txt
Handbrakecli -i I:\BluRay\Deadpool -o d:\temp\deadpool\2.m4v -Z "Fast 1080p30" -c 2 -t 130 2>>d:\temp\deadpool\1.txt
...
Handbrakecli -i I:\BluRay\Deadpool -o d:\temp\deadpool\29.m4v -Z "Fast 1080p30" -c 29 -t 130 2>>d:\temp\deadpool\1.txt

Handbrakecli -i I:\BluRay\Deadpool -o d:\temp\deadpool\1-29.m4v -Z "Fast 1080p30" -c 1-29 -t 130 2>>d:\temp\deadpool\2.txt

Results:

In the example above you end up with 29 chapter files and 1 full length encode. While in theory the 29 chapters should equal the complete
encode they are not exactly the same, however I can live with this as we are just after an estimate.

What I ended up with is this:

Chapter H M S Time BitRate Comp_Final_BitRate Total_Time Size Total_Size Total_BitRate %_Final_Time %_Final_BitRate
0 0 0 0 0 0 0% 0 0 0 0.00 0%
1 0 3 4 184 3352 109% 184 616768 616768 3352.00 3% 109%
2 0 2 49 169 3197 103% 353 540293 1157061 3277.79 5% 106%
3 0 3 39 219 4325 140% 572 947175 2104236 3678.73 9% 119%
4 0 3 49 229 3888 126% 801 890352 2994588 3738.56 12% 121%
5 0 3 30 210 2578 83% 1011 541380 3535968 3497.50 16% 113%
6 0 3 25 205 3007 97% 1216 616435 4152403 3414.81 19% 111%
7 0 1 53 113 2390 77% 1329 270070 4422473 3327.67 20% 108%
8 0 5 4 304 2527 82% 1633 768208 5190681 3178.62 25% 103%
9 0 4 49 289 2856 92% 1922 825384 6016065 3130.11 30% 101%
10 0 4 52 292 1718 56% 2214 501656 6517721 2943.87 34% 95%
11 0 4 15 255 3029 98% 2469 772395 7290116 2952.66 38% 96%
12 0 3 39 219 2409 78% 2688 527571 7817687 2908.37 41% 94%
13 0 2 12 132 2328 75% 2820 307296 8124983 2881.20 43% 93%
14 0 3 15 195 2846 92% 3015 554970 8679953 2878.92 46% 93%
15 0 3 10 190 3005 97% 3205 570950 9250903 2886.40 49% 93%
16 0 2 16 136 7363 238% 3341 1001368 10252271 3068.62 52% 99%
17 0 3 34 214 3064 99% 3555 655696 10907967 3068.35 55% 99%
18 0 3 58 238 2916 94% 3793 694008 11601975 3058.79 58% 99%
19 0 4 4 244 3044 99% 4037 742736 12344711 3057.89 62% 99%
20 0 2 10 130 1878 61% 4167 244140 12588851 3021.08 64% 98%
21 0 3 14 194 2724 88% 4361 528456 13117307 3007.87 67% 97%
22 0 4 17 257 3199 104% 4618 822143 13939450 3018.50 71% 98%
23 0 4 0 240 3062 99% 4858 734880 14674330 3020.65 75% 98%
24 0 5 10 310 4667 151% 5168 1446770 16121100 3119.41 80% 101%
25 0 5 10 310 4185 135% 5478 1297350 17418450 3179.71 84% 103%
26 0 3 44 224 4114 133% 5702 921536 18339986 3216.41 88% 104%
27 0 5 7 307 2792 90% 6009 857144 19197130 3194.73 93% 103%
28 0 8 12 492 1626 53% 6501 799992 19997122 3076.01 100% 100%
29 0 0 2 2 9 0% 6503 18 19997140 3075.06 100% 100%

Without bothering with the details (although please ask if unsure) the 2 important columns are the last 2, as we move through the movie what is our current bitrate compared to the final bitrate.

%_Final_Time %_Final_BitRate
0%
3% 109%
5% 106%
9% 119%
12% 121%
16% 113%
19% 111%
20% 108%
25% 103%
30% 101%
34% 95%
38% 96%
41% 94%
43% 93%
46% 93%
49% 93%
52% 99%
55% 99%
58% 99%
62% 99%
64% 98%
67% 97%
71% 98%
75% 98%
80% 101%
84% 103%
88% 104%
93% 103%
100% 100%
100% 100%

Conclusion:

Early on in the encode the bitrate fluctuates more as there is a shorter period of time over which to average any spikes, as the encode continues bitrate spikes will have less of an effect as they are averaged out over the longer timeline.

Thoughts:

While it is impossible to predict 100% what the future part of an encode will be, in this case the encode appears to be fairly steady after the 25% point by not fluctuating more than 7% from the final bitrate. Obviously 7% may not be acceptable to all but from my personal point of view this is acceptable.

Now I just need to sample more movies to see if they exhibit similar patterns.

Known issues:

My times and bitrates are from Windows and thus are rounded up and down, this could lead to changes in my results but as I am only estimating the final size I am fine with this.
mduell
Veteran User
Posts: 8187
Joined: Sat Apr 21, 2007 8:54 pm

Re: Estimate file size

Post by mduell »

This assumes the correct video length is known, which is often not the case and one of the major reasons target size was removed.
rollin_eng
Veteran User
Posts: 4840
Joined: Wed May 04, 2011 11:06 pm

Re: Estimate file size

Post by rollin_eng »

Of course but I have never had handbrake be unable to correctly read the length of a Blu-Ray so I wouldn't consider this much of an issue.
mduell
Veteran User
Posts: 8187
Joined: Sat Apr 21, 2007 8:54 pm

Re: Estimate file size

Post by mduell »

...for people who constrain themselves to your Blu-ray sources only. HB's userbase is far broader than that.
rollin_eng
Veteran User
Posts: 4840
Joined: Wed May 04, 2011 11:06 pm

Re: Estimate file size

Post by rollin_eng »

mduell wrote:...for people who constrain themselves to your Blu-ray sources only. HB's userbase is far broader than that.
I'm sure it is, but I'm not sure of its relevancy in this thread?
rollin_eng
Veteran User
Posts: 4840
Joined: Wed May 04, 2011 11:06 pm

Re: Estimate file size

Post by rollin_eng »

Code: Select all

Transformers 4	
	
%_Final_Time	%_Final_BitRate
0%	
4%	74%
9%	93%
13%	95%
18%	88%
24%	99%
28%	105%
31%	104%
36%	103%
41%	99%
45%	97%
50%	99%
54%	100%
59%	101%
63%	101%
67%	100%
71%	100%
75%	99%
80%	100%
85%	102%
90%	103%
95%	104%
100%	100%
100%	100%
	
HP  5	
	
%_Final_Time	%_Final_BitRate
0%	
3%	139%
7%	123%
11%	110%
13%	104%
15%	108%
19%	101%
20%	103%
22%	101%
25%	99%
27%	100%
30%	101%
33%	101%
37%	102%
41%	100%
46%	102%
49%	102%
52%	102%
55%	100%
57%	100%
62%	102%
66%	101%
69%	100%
71%	103%
74%	102%
77%	102%
79%	101%
82%	102%
85%	101%
90%	105%
91%	105%
93%	105%
100%	100%
100%	100%
	
Aliens	
	
%_Final_Time	%_Final_BitRate
0%	
1%	27%
3%	62%
6%	89%
7%	96%
10%	91%
11%	89%
14%	86%
16%	84%
17%	84%
20%	87%
21%	90%
23%	91%
25%	92%
30%	95%
34%	100%
36%	101%
36%	103%
39%	104%
42%	105%
46%	105%
47%	104%
50%	105%
53%	105%
54%	105%
56%	105%
59%	104%
61%	103%
63%	102%
64%	102%
66%	102%
68%	101%
70%	101%
73%	100%
73%	100%
79%	99%
82%	99%
83%	99%
87%	99%
89%	99%
92%	100%
93%	100%
98%	101%
98%	101%
100%	100%
100%	100%
	
SPR	
	
%_Final_Time	%_Final_BitRate
0%	
3%	69%
8%	109%
11%	124%
16%	129%
21%	116%
27%	110%
34%	113%
39%	110%
44%	100%
46%	98%
50%	99%
56%	100%
63%	100%
71%	103%
75%	101%
81%	101%
87%	102%
90%	103%
96%	103%
100%	100%
100%	100%
So with 4 more movies done the following is found:

Transformers 4: After 24% the bitrate only fluctuates +-5% from its final value.
Harry Potter 5: After 19% the bitrate only fluctuates +-5% from its final value.
Aliens: : After 30% the bitrate only fluctuates +-5% from its final value.
Saving Private Ryan:: After 44% the bitrate only fluctuates +-3% from its final value.
Post Reply