Page 1 of 1

Different file sizes in same encodings on same machine?

Posted: Wed Sep 14, 2022 6:05 pm
by zak
Description of problem or question:
Not really a problem (I hope) but I would like clarity about this:
After encoding the same video with the same command line execution on the same machine (Intel Core i7-12700) with the same HandBrake version the resulting file sizes slightly differ every time. This did not happen on my old machine with an Intel Xeon E3-1240L.

b_deterministic is set to 1 in my source:

Code: Select all

build/contrib/x264/x264-snapshot-20210613-3065/common/base.c:    param->b_deterministic = 1;
Thank you for your work!


Steps to reproduce the problem (If Applicable):
1. start the encoding, for example:

Code: Select all

"/usr/local/bin/HandBrakeCLI" --input="VIDEO/" --title=1 --output="test.mkv" --markers --encoder=x264 --encoder-preset=veryslow --quality=20 --audio=4,3,4,2,1,2 --aencoder=ffac3,copy:dts,copy:dtshd,ffac3,copy:dts,copy:dtshd --ab=640,,,640,, --crop=0:0:64:64 --auto-anamorphic --decomb --subtitle=1,2,1,2 --subtitle-forced=3,4
2. Move the file to another name, e.g. "mv test.mkv test1.mkv"
3. start the same encoding again.
4. Compare file sizes.


HandBrake version (e.g., 1.0.0):
HandBrake 1.5.1, compiled from source while activating MODULES += contrib/x264 in make/include/main.defs

Operating system and version (e.g., Ubuntu 16.04 LTS, macOS 10.13 High Sierra, Windows 10 Creators Update):
Debian GNU/Linux 11 (bullseye)

(with latest updates installed of course)

HandBrake Activity Log ***required*** (see How-to get an activity log)

https://pastebin.com/UNq6zYMZ

Re: Different file sizes in same encodings on same machine?

Posted: Wed Sep 14, 2022 7:39 pm
by rollin_eng
A small difference is expected, can you post 2 logs so we can see the difference.

Re: Different file sizes in same encodings on same machine?

Posted: Thu Sep 15, 2022 7:22 am
by zak
Another log: https://pastebin.com/tgZRQ0VL

As I said it did not happen with my other CPU. Would there be a way to create completely identical video files? (not counting meta-data of course)

Re: Different file sizes in same encodings on same machine?

Posted: Thu Sep 15, 2022 7:28 am
by zak
As far as I see the only relevant difference is in this line:

Code: Select all

[19:20:00] mux: track 0, 139205 frames, 6800437243 bytes, 9370.07 kbps, fifo 512
[22:12:01] mux: track 0, 139205 frames, 6800436109 bytes, 9370.07 kbps, fifo 512
Comparing with further logs I see that it is always this line, i.e. track 0. This applies to another video I tested as well.

Re: Different file sizes in same encodings on same machine?

Posted: Thu Sep 15, 2022 11:04 am
by Deleted User 11865
That's a little odd indeed, but it won't be easy to figure out where the nondeterminism comes from.

Is the issue reproducible on a shorter samples (e.g. encoding the same chapter, say, #12, multiple times with the same HandBrake instance)?

Re: Different file sizes in same encodings on same machine?

Posted: Thu Sep 15, 2022 2:27 pm
by rollin_eng
Isn't x264 non-deterministic? Thus small differences are normal?

Re: Different file sizes in same encodings on same machine?

Posted: Thu Sep 15, 2022 4:14 pm
by mduell
x264 is non-deterministic with multithreading and VBV.

Re: Different file sizes in same encodings on same machine?

Posted: Thu Sep 15, 2022 7:11 pm
by zak
I just did some tests and encoded chapter 12 three times and then chapter 1 two times (just added -c12 resp. -c1 to the command line)
All the respective files were identical. (I did a "mkvextract tracks test.mkv 0:test.x264" and compared those files using "diff".)

Maybe the differences don't happen that often and only on longer files?
I will probably do some more tests later...

Re: Different file sizes in same encodings on same machine?

Posted: Thu Sep 15, 2022 9:35 pm
by Deleted User 11865
mduell wrote: Thu Sep 15, 2022 4:14 pm x264 is non-deterministic with multithreading and VBV.
s/x264/x265 (hence why x265 does or did print a warning to that effect when you enable VBV)

x264 is deterministic on my end at least.

Re: Different file sizes in same encodings on same machine?

Posted: Thu Sep 15, 2022 11:13 pm
by mduell
x264 dev mailing list says it's the intended behavior https://mailman.videolan.org/pipermail/ ... 11036.html

Re: Different file sizes in same encodings on same machine?

Posted: Fri Sep 16, 2022 1:54 pm
by Deleted User 11865
Fair enough :-)