Tutorial: x264 presets/tunes and HandBrake

General questions or discussion about HandBrake, Video and/or audio transcoding, trends etc.
Locked
Deleted User 11865

Tutorial: x264 presets/tunes and HandBrake

Post by Deleted User 11865 »

This information is up-to-date for HandBrake 0.9.8 (which uses x264 r2146 internally). It may no longer apply to future versions.

HandBrake 0.9.8 does not natively support x264 presets and tunes (except via the CLI). However, nothing prevents you from re-creating them manually.

Note: the HandBrake nightly builds now support x264 presets & tunes natively, so they don't need to be re-created.

You can specify most x264 options (nearly all options parseable by libx264's x264_param_parse function) via HandBrake's advanced panel:
  • --option value becomes option=value
  • Options are separated by a colon.
  • Options that do not require a parameter can be specified as either option or option=1
    The MacGUI's advanced panel will automatically add =1 if you don't specify a value.
  • In most cases, no-<something>=1 and <something>=0 are equivalent.
    This isn't true for deblock - to disable it, you must use no-deblock=1; deblock=0 is the same as deblock=0,0 (the default)
A list of all available x264 options can be found on MeWiki:

As of March 19, 2012 (applicable to x264 r2146)
Latest Revision

Note: some options are only available to the x264 CLI (x264.exe) and are therefore not applicable to HandBrake.
A few options (pass, slow-firstpass, bitrate, crf ) are set from the Video panel rather than the Advanced panel.

1. x264 presets

The official x264 presets (as of r2146) are:

Code: Select all

      --preset <string>       Use a preset to select encoding settings [medium]
                                  Overridden by user settings.
                                  - ultrafast:
                                    --no-8x8dct --aq-mode 0 --b-adapt 0
                                    --bframes 0 --no-cabac --no-deblock
                                    --no-mbtree --me dia --no-mixed-refs
                                    --partitions none --rc-lookahead 0 --ref 1
                                    --scenecut 0 --subme 0 --trellis 0
                                    --no-weightb --weightp 0
                                  - superfast:
                                    --no-mbtree --me dia --no-mixed-refs
                                    --partitions i8x8,i4x4 --rc-lookahead 0
                                    --ref 1 --subme 1 --trellis 0 --weightp 1
                                  - veryfast:
                                    --no-mixed-refs --rc-lookahead 10
                                    --ref 1 --subme 2 --trellis 0 --weightp 1
                                  - faster:
                                    --no-mixed-refs --rc-lookahead 20
                                    --ref 2 --subme 4 --weightp 1
                                  - fast:
                                    --rc-lookahead 30 --ref 2 --subme 6
                                    --weightp 1
                                  - medium:
                                    Default settings apply.
                                  - slow:
                                    --b-adapt 2 --direct auto --me umh
                                    --rc-lookahead 50 --ref 5 --subme 8
                                  - slower:
                                    --b-adapt 2 --direct auto --me umh
                                    --partitions all --rc-lookahead 60
                                    --ref 8 --subme 9 --trellis 2
                                  - veryslow:
                                    --b-adapt 2 --bframes 8 --direct auto
                                    --me umh --merange 24 --partitions all
                                    --ref 16 --subme 10 --trellis 2
                                    --rc-lookahead 60
                                  - placebo:
                                    --bframes 16 --b-adapt 2 --direct auto
                                    --slow-firstpass --no-fast-pskip
                                    --me tesa --merange 24 --partitions all
                                    --rc-lookahead 60 --ref 16 --subme 11
                                    --trellis 2
Pastebin: http://paste.handbrake.fr/pastebin.php?show=3038

Formatted for HandBrake's advanced panel (getting rid of redundant options; e.g. --bframes 0 implies --b-adapt 0), we get:

ultrafast

Code: Select all

ref=1:bframes=0:cabac=0:8x8dct=0:weightp=0:me=dia:subq=0:rc-lookahead=0:mbtree=0:analyse=none:trellis=0:aq-mode=0:scenecut=0:no-deblock=1
superfast

Code: Select all

ref=1:weightp=1:me=dia:subq=1:rc-lookahead=0:mbtree=0:analyse=i4x4,i8x8:trellis=0
veryfast

Code: Select all

ref=1:weightp=1:subq=2:rc-lookahead=10:trellis=0
faster

Code: Select all

ref=2:mixed-refs=0:weightp=1:subq=4:rc-lookahead=20
fast

Code: Select all

ref=2:weightp=1:subq=6:rc-lookahead=30
medium: default x264 settings apply (empty advanced options string).

slow

Code: Select all

ref=5:b-adapt=2:direct=auto:me=umh:subq=8:rc-lookahead=50
slower

Code: Select all

ref=8:b-adapt=2:direct=auto:me=umh:subq=9:rc-lookahead=60:analyse=all:trellis=2
veryslow

Code: Select all

ref=16:bframes=8:b-adapt=2:direct=auto:me=umh:merange=24:subq=10:rc-lookahead=60:analyse=all:trellis=2
placebo

Code: Select all

ref=16:bframes=16:b-adapt=2:direct=auto:me=tesa:merange=24:subq=11:rc-lookahead=60:analyse=all:trellis=2:no-fast-pskip=1
--slow-firstpass can be set by disabling "Turbo First Pass" in the Video panel.

The official x264 presets are meant to be good speed / compression efficiency tradeoffs. The options that are most useful to compression efficiency are enabled first; the less useful options are enabled in the slowest presets (i.e. once all the most useful options are already enabled).

If you're using settings slower than the placebo preset, you're doing it wrong (the difference won't be noticeable).

Note: if you're starting with an official HandBrake preset for your device, be sure to add these settings before the preset's options (so that the HandBrake options override the x264 preset - the other way around will likely break compatibility with your device).

Example. Starting with the AppleTV preset:

Code: Select all

cabac=0:ref=2:me=umh:b-pyramid=none:b-adapt=2:weightb=0:trellis=0:weightp=0:vbv-maxrate=9500:vbv-bufsize=9500
combined with the veryslow x264 preset, for better compression efficiency and much slower encoding:

Code: Select all

ref=16:bframes=8:b-adapt=2:direct=auto:me=umh:merange=24:subq=10:rc-lookahead=60:analyse=all:trellis=2
gives us:

Code: Select all

ref=16:bframes=8:b-adapt=2:direct=auto:me=umh:merange=24:subq=10:rc-lookahead=60:analyse=all:trellis=2:cabac=0:ref=2:me=umh:b-pyramid=none:b-adapt=2:weightb=0:trellis=0:weightp=0:vbv-maxrate=9500:vbv-bufsize=9500
Some settings should be safe to override with the x264 preset values:
  • b-adapt (Adaptive B-Frames)
  • direct (Adaptive Direct Mode)
  • me (Motion Estimation Method)
  • subme/subq (Subpixel ME & Mode Decision)
  • merange (Motion Estimation Range)
  • aq-strength (Adaptive Quantization)
  • psy-rd (Psychovisual Rate Distorsion, Psychovisual Trellis)
  • partitions/analyse (Partition Types)
  • trellis (Trellis)
  • deblock (Deblocking)*
  • rc-lookahead
  • no-fast-pskip
  • mixed-refs
* Some very weak devices may require that deblocking be disabled

The following settings are NOT safe to override with the x264 preset values:
  • 8x8dct (8x8 Transform)
    only works with devices that support H.264 High Profile - see H.264 Profiles
    some devices are know not to honor the pixel aspect ratio when 8x8dct is used (e.g. Xbox 360)
  • cabac (CABAC Entropy Coding)
    should work with devices that support H.264 Main Profile or higher - see H.264 Profiles
    some devices that support Main Profile H.264 still have trouble playing H.264 fast enough when CABAC is used, especially during bitrate peaks (e.g. 1st gen. AppleTV)
  • bframes (Maximum B-Frames)
    0 means B-frames (and thus b-ypramid, weightb) are disabled
    should work with devices that support H.264 Main Profile or higher - see H.264 Profiles
  • b-pyramid (Pyramidal B-Frames)
    should work with devices that support H.264 Main Profile or higher - see H.264 Profiles
    some devices that support Main Profile H.264 still fail to play H.264 streams when b-pyramid is used (e.g. 1st gen. AppleTV)
    also, some software has trouble decoding H.264 fast enough when b-pyramid is used (e.g. QuickTime 7)
  • weightb
    should work with devices that support H.264 Main Profile or higher - see H.264 Profiles
    some software and devices have trouble decoding H.264 fast enough when weightb is used (e.g. QuickTime 7, 1st gen. AppleTV with original software)
  • weightp (Weighted P-Frames)
    should work with devices that support H.264 Main Profile and higher - see H.264 Profiles
    some devices that support Main Profile H.264 still fail to play H.264 streams when weightp is used (e.g. 1st gen. AppleTV)
    also, some software has trouble decoding H.264 fast enough when weightp is used (e.g. QuickTime 7)
  • ref (Reference Frames)
    see H.264 Levels
  • vbv-bufsize and vbv-maxrate
    some devices have trouble when the peak bitrate exceeds a given value and require VBV in order to limit such peaks - change VBV parameters at your own risk
meaning:

1) you shouldn't enable them if they are disabled in the HandBrake preset, unless you know what you're doing
2) you shouldn't increase their value from what is specified in the HandBrake preset, unless you know what you're doing

When no number of reference frames is specified, the x264 default of 3 is used. You will need to make sure that you don't exceed your device's maximum supported level by using too many refs; also, some weak devices (e.g. old iPods, and possibly the first-generation AppleTV) may struggle when using many refs, even though the level limits allow a fairly large number - so be sure not to override the preset's ref value accidentally when using an x264 preset.

Using the example of the AppleTV preset above, AppleTV + veryslow would become:

Code: Select all

bframes=8:b-adapt=2:direct=auto:me=umh:merange=24:subq=10:rc-lookahead=60:analyse=all:trellis=2:cabac=0:ref=2:b-pyramid=none:weightb=0:weightp=0:vbv-maxrate=9500:vbv-bufsize=9500
2. x264 tunes

The official x264 tunes (as of r2146) are:

Code: Select all

      --tune <string>         Tune the settings for a particular type of source
                              or situation
                                  Overridden by user settings.
                                  Multiple tunings are separated by commas.
                                  Only one psy tuning can be used at a time.
                                  - film (psy tuning):
                                    --deblock -1:-1 --psy-rd <unset>:0.15
                                  - animation (psy tuning):
                                    --bframes {+2} --deblock 1:1
                                    --psy-rd 0.4:<unset> --aq-strength 0.6
                                    --ref {Double if >1 else 1}
                                  - grain (psy tuning):
                                    --aq-strength 0.5 --no-dct-decimate
                                    --deadzone-inter 6 --deadzone-intra 6
                                    --deblock -2:-2 --ipratio 1.1 
                                    --pbratio 1.1 --psy-rd <unset>:0.25
                                    --qcomp 0.8
                                  - stillimage (psy tuning):
                                    --aq-strength 1.2 --deblock -3:-3
                                    --psy-rd 2.0:0.7
                                  - psnr (psy tuning):
                                    --aq-mode 0 --no-psy
                                  - ssim (psy tuning):
                                    --aq-mode 2 --no-psy
                                  - fastdecode:
                                    --no-cabac --no-deblock --no-weightb
                                    --weightp 0
                                  - zerolatency:
                                    --bframes 0 --force-cfr --no-mbtree
                                    --sync-lookahead 0 --sliced-threads
                                    --rc-lookahead 0
Pastebin: http://paste.handbrake.fr/pastebin.php?show=3039

In HandBrake's advanced panel:
  • film: optimize settings for most non-animated video content (not only feature films)
    Set Deblocking to -1,-1
    Set Psychovisual Trellis to 0.15
  • animation: optimize settings for traditional animation - note that most 3D animation behaves more like film, so only use this for hand-drawn animation (anime, classic Disney, etc.)
    Set Deblocking to 1,1
    Set Psychovisual Rate Distorsion to 0.4
    Set Adaptive Quantization to 0.6
    If B-Frames are enabled (non-zero), add 2 B-Frames to Maximum B-Frames
    If using more than one Reference Frame(s), double the number of Reference Frames*
    * Make sure not to exceed level constraints - most devices only support H.264 up to a given level
  • grain: optimize settings for film with high levels of grain
    (better preserve film grain, at the expense of - often significantly - higher bitrate)
    Set Deblocking to -2,-2
    Set Psychovisual Trellis to 0.25
    Set Adaptive Quantization to 0.5
    Enable No DCT Decimation
    Add the following options to the option string:

    Code: Select all

    deadzone-inter=6:deadzone-intra=6:ipratio=1.1:pbratio=1.1:qcomp=0.8
  • stillimage: there has been some demand for this from companies looking to use x264 for still image compression (it can outperform JPEG or JPEG-2000 by a factor of 2 or more) - not useful for video content so irrelevant to HandBrake
    Set Deblocking to -3,-3
    Set Adaptive Quantization to 1.2
    Set Psychovisual Rate Distorsion to 2.0
    Set Psychovisual Trellis to 0.7
  • psnr: optimizes the settings for best PSNR (hurts perceived quality, not useful for encodes you intend to watch)
    Add the following options to the option string:

    Code: Select all

    aq-mode=0:psy=0
  • ssim: optimizes the settings for best SSIM (hurts perceived quality, not useful for encodes you intend to watch)
    Add the following options to the option string:

    Code: Select all

    aq-mode=2:psy=0
  • fastdecode: optimizes settings for faster decoding on weak devices
    Disable CABAC Entropy Coding
    Disable Weighted P-Frames
    Add the following options to the option string:

    Code: Select all

    weightb=0:no-deblock=1
  • zerotatency: minimizes encoding latency (useful for live streaming, which HandBrake doesn't do)
    Set Maximum B-Frames to 0
    Add the following options to the option string:

    Code: Select all

    mbtree=0:rc-lookahead=0:sync-lookahead=0:sliced-threads=1
    CFR (Constant Framerate) is set via HandBrake's video tab
If you modify a built-in preset and the output doesn't work on your device, try again with the built-in preset unmodified; if it works, you're changing something you shouldn't be. It's up to you to figure out what.
Last edited by Deleted User 11865 on Wed Jan 30, 2013 9:02 pm, edited 9 times in total.
Reason: Nightly builds
Locked