Why are the encoded movies larger than their originals? (batch script)

[michael.heuberger]

Hello guys

I wrote this batch script to encode all videos recursively from a start directory. This for small videos recorded from various different mobiles; with the aim to reduce file sizes:

Code: Select all

#!/bin/bash

export FLAGS="
    --format mkv
    --decomb=bob
    --loose-anamorphic
    --two-pass
    --modulus 2
    --encoder x264
    --x264-preset veryslow
    --quality 17
    --aencoder copy
    --audio-fallback ac3
    --encoder-preset slow
    --optimize"

if [ -z "$1" ] ; then
    TRANSCODEDIR="."
else
    TRANSCODEDIR="$1"
fi

find "$TRANSCODEDIR"/* \
  -type f \
  -iregex '.*\.\(avi\|mpg\|flv\|wmv\|mov\|3gp\|mp4\)$' \
  -exec bash \
  -c 'HandBrakeCLI -i "$1" -o "${1%\.*}".mkv $FLAGS' __ {} \;

Maybe MKV is the wrong format to achieve this? Maybe the x264 encoder is the wrong one? Maybe one of these presets is overwriting some custom arguments? I don't know.

Any tips/advice very welcome.

[rollin_eng]

Could you please post your logs, instructions can be found here:

viewtopic.php?f=6&t=31236

[michael.heuberger]

Ok, have pasted the stdout to a Gist
https://gist.github.com/binarykitchen/c ... 32a5b50a6a

I aborted it after about 30 sec but I think that should be sufficient.

Hope you can give me some clues?

[rollin_eng]

Your source has incredibly low bit rate audio and video.

Try changing your quality to 22 or 24.

Also try changing your audio to AAC at 12 or 24kbps.

[mduell]

Comically low bitrate source + inappropriate quality target for such a source.

[damtor]

Try changing your quality to 22 or 24.

I use quality 16, and my encoded videos are approximately 25% the size of the source;
I'm no expert, but I don't think that's your problem.

Since your videos are small, and you clearly know how to write a shell script,
I would suggest you get rid of all you custom flags, and just call Handbrake
using --input and --output and see what the size turns out to be. Assuming
it's smaller, then start adding your custom flags back in one at a time until you
figure out what flag or combination is causing the problem. I hope you will
post your results, I'd like to know what happened.

In your log file, I did see this:

udfread ERROR: ECMA 167 Volume Recognition failed

If use or non-use of any of the flags gets rid of that error, also please
post your results on that: I'd sure like to know if this error should be
ignored or not.

[mduell]

Try changing your quality to 22 or 24.

I use quality 16, and my encoded videos are approximately 25% the size of the source;

Are your sources comically low bitrate like his? Or are your sources bloated DVD/Bluray?

I'm no expert

No kiddin.

In your log file, I did see this:

udfread ERROR: ECMA 167 Volume Recognition failed

If use or non-use of any of the flags gets rid of that error, also please
post your results on that: I'd sure like to know if this error should be
ignored or not.

Completely normal when your source is not a Bluray. No flags will change this.

[damtor]

>> Or are your sources bloated DVD/Bluray?

Original source is a Blu-Ray disc, processed by makemkv to
create a *mkv file. The *mkv file is the input to HandBrakeCLI.
I use Don Melton's scripts to create the HandBrakeCLI command line.

>> Completely normal when your source is not a Bluray. No flags will change this.

Could you please explain why I get this error:

>> udfread ERROR: ECMA 167 Volume Recognition failed

when processing the same (Blu-Ray) input as described above,
and in this thread which no one has yet replied to:

viewtopic.php?f=10&t=35588 ?

[mduell]

>> Or are your sources bloated DVD/Bluray?

Original source is a Blu-Ray disc, processed by makemkv to
create a *mkv file. The *mkv file is the input to HandBrakeCLI.
I use Don Melton's scripts to create the HandBrakeCLI command line.

Yea, those are vastly higher quality sources than what OP is working with, and the lower RF can be appropriate.

>> Completely normal when your source is not a Bluray. No flags will change this.

Could you please explain why I get this error:

>> udfread ERROR: ECMA 167 Volume Recognition failed

when processing the same (Blu-Ray) input as described above,

Because your source is a MKV file, not a Bluray UDF disk filesystem structure!

[michael.heuberger]

thanks guys, you are entertaining

so yes, i intentionally aim for less quality to save file size. will check out at home tonight some of your advice.

but lets take a step back. what about my initial questions:
a) Maybe MKV is the wrong output format to achieve lowest file size?
b) Maybe the x264 encoder is the wrong one?
c) Maybe one of these presets is overwriting some custom arguments?

[mduell]

Container is irrelevant, you could use x265 if your playback environment supports it and you can tolerate the encode sizes, and you're not specifying a HB preset.

[michael.heuberger]

@mduell well yeah, will consider x265

but regarding presets, i am using --x264-preset and --encoder-preset slow together here with other stuff like --quality 17 ... so i wonder here if they don't mess with each other and override something?

ultimately i want very small file sizes. any other clues?

[mduell]

Those are the same thing for different versions of HB. The former is ineffective on 1.0.1.

Quality has nothing to do with x264/encoder preset.

Best way to get small files is to reduce quality. An RF in the mid 20s is probably about right, but you can go as high as low 30s before getting ridiculous. Although with such a low quality source it's gonna be ugly.

[michael.heuberger]

but still, i would like to know how i can reduce file size further to the extreme. any other clues?

[rollin_eng]

There is no "secret" super compression, as mduell said just keep lowering the RF value until you are happy.

[michael.heuberger]

ok - how do you manually set the rf value via command line?

[rollin_eng]

--quality

You have it at 17, try 30 and see what it looks like.

And change your audio.