Audio delay relative to video

[KnightAzul]

Description of problem or question:

It seems that whenever I encode an MKV format A/V file with Handbrake to format MKV and reencode the audio (no passthru) then the resultant MKV audio track always has a delay relative to video of a few ms as shown by Mediainfo or MKVToolNix Info Tool. In the orignal source A/V file there is no audio delay relative to video shown by Mediainfo or MKVToolNix Info Tool. It seems that for some reason Handbrake is introducing the delay.

If I choose format MP4 then I have the Handbrake option "Align A/V Start". With exactly the same Handbrake encode options (except to format MP4 instead of MKV format) and the Handbrake "Align A/V Start" option checked then there is no audio track delay relative to video in the resultant MP4 audio track. From the Handbrake documentation the "Align A/V Start" option aligns the initial timestamps of all audio and video streams by inserting blank frames or dropping frames, and this may improve audio/video sync for broken players that do not honor MP4 edit lists.

However, the Handbrake option "Align A/V Start" is only available for MP4 and not MKV format.

I know that the audio and video in an A/V file don't have to start at the same time. One or the other can have a delay flagged at the container level. Unfortunately, many players don't respect the delay flag in the MP4 container. Most players that support MKV do support the delay flag.

Based on all of this it would seem that Handbrake only gives the "Align A/V Start" option for MP4 format due to many players not respecting the delay flag in the MP4 container, and therefore giving more compatibility between players. However:

• Why is the "Align A/V Start" option not given for MKV format to provide this same compatibility between players, and

• Why, if the the orignal source A/V file there is no audio delay relative to video shown by Mediainfo or MKVToolNix Info Tool, does Handbrake produce a resultant MKV audio track with a delay relative to video of a few ms as shown by Mediainfo or MKVToolNix Info Tool.

Thanks for any help!
KnightAzul



Steps to reproduce the problem (If Applicable):




HandBrake version (e.g., 1.0.0):
1.3.3



Operating system and version (e.g., Ubuntu 16.04 LTS, macOS 10.13 High Sierra, Windows 10 Creators Update):
Windows 10 Pro 1909



HandBrake Activity Log ***required*** (see How-to get an activity log)

Code: Select all

HandBrake 1.3.3 (2020061300)
OS: Microsoft Windows NT 10.0.18363.0
CPU: AMD Ryzen 5 3600 6-Core Processor              
Ram: 16303 MB, 
GPU Information:
  NVIDIA GeForce GT 1030 - 26.21.14.4120
Screen: 1920x1080
Temp Dir: C:\Users\ynot\AppData\Local\Temp\
Install Dir: C:\Program Files\HandBrake
Data Dir: C:\Users\ynot\AppData\Roaming\HandBrake

-------------------------------------------


# Starting Encode ...

[11:08:29] base preset: Fast 1080p30 (Modified)
[11:08:29] hb_init: starting libhb thread
[11:08:29] Starting work at: Sun Oct 18 11:08:29 2020
[11:08:29] 1 job(s) to process
[11:08:29] json job:
{
  "Audio": {
    "AudioList": [
      {
        "Bitrate": 512,
        "DRC": 0.0,
        "Encoder": "av_aac",
        "Gain": 0.0,
        "Mixdown": 7,
        "NormalizeMixLevel": false,
        "Samplerate": 0,
        "Track": 0,
        "DitherMethod": 0
      }
    ],
    "CopyMask": [
      "copy:aac",
      "copy:ac3",
      "copy:dtshd",
      "copy:dts",
      "copy:eac3",
      "copy:flac",
      "copy:mp3",
      "copy:truehd"
    ],
    "FallbackEncoder": "ac3"
  },
  "Destination": {
    "ChapterList": [
      {
        "Name": "Chapter 1"
      }
    ],
    "ChapterMarkers": true,
    "AlignAVStart": false,
    "File": "D:\\ynot\\Program Data\\Handbrake\\Gran Torino-4.mkv",
    "Mp4Options": {
      "IpodAtom": false,
      "Mp4Optimize": false
    },
    "Mux": "av_mkv"
  },
  "Filters": {
    "FilterList": [
      {
        "ID": 12,
        "Settings": {
          "crop-bottom": "0",
          "crop-left": "0",
          "crop-right": "0",
          "crop-top": "0",
          "height": "536",
          "width": "1280"
        }
      },
      {
        "ID": 6,
        "Settings": {
          "mode": "1"
        }
      }
    ]
  },
  "PAR": {
    "Num": 1,
    "Den": 1
  },
  "Metadata": {},
  "SequenceID": 0,
  "Source": {
    "Angle": 1,
    "Range": {
      "Type": "chapter",
      "Start": 1,
      "End": 1
    },
    "Title": 1,
    "Path": "E:\\ynot\\Multimedia\\Video\\Movies\\Gran Torino.mkv"
  },
  "Subtitle": {
    "Search": {
      "Burn": false,
      "Default": false,
      "Enable": false,
      "Forced": false
    },
    "SubtitleList": []
  },
  "Video": {
    "Encoder": "x264",
    "Level": "4.0",
    "TwoPass": false,
    "Turbo": false,
    "ColorMatrixCode": 0,
    "Options": "",
    "Preset": "ultrafast",
    "Profile": "main",
    "Quality": 22.0,
    "QSV": {
      "Decode": false,
      "AsyncDepth": 0
    }
  }
}
[11:08:29] CPU: AMD Ryzen 5 3600 6-Core Processor
[11:08:29]  - logical processor count: 12
[11:08:29] Intel Quick Sync Video support: no
[11:08:29] hb_scan: path=E:\ynot\Multimedia\Video\Movies\Gran Torino.mkv, title_index=1
udfread ERROR: ECMA 167 Volume Recognition failed
src/libbluray/disc/disc.c:323: failed opening UDF image E:\ynot\Multimedia\Video\Movies\Gran Torino.mkv
src/libbluray/disc/disc.c:424: error opening file BDMV\index.bdmv
src/libbluray/disc/disc.c:424: error opening file BDMV\BACKUP\index.bdmv
src/libbluray/bluray.c:2585: nav_get_title_list(E:\ynot\Multimedia\Video\Movies\Gran Torino.mkv\) failed
[11:08:29] bd: not a bd - trying as a stream/file instead
libdvdnav: Using dvdnav version 6.0.1
libdvdread: Encrypted DVD support unavailable.
libdvdread:DVDOpenFileUDF:UDFFindFile /VIDEO_TS/VIDEO_TS.IFO failed
libdvdread:DVDOpenFileUDF:UDFFindFile /VIDEO_TS/VIDEO_TS.BUP failed
libdvdread: Can't open file VIDEO_TS.IFO.
libdvdnav: vm: failed to read VIDEO_TS.IFO
[11:08:29] dvd: not a dvd - trying as a stream/file instead
Input #0, matroska,webm, from 'E:\ynot\Multimedia\Video\Movies\Gran Torino.mkv':
  Metadata:
    encoder         : libebml v1.3.0 + libmatroska v1.4.0
    creation_time   : 2015-02-15T09:16:48.000000Z
  Duration: 01:56:40.64, start: 0.000000, bitrate: 5369 kb/s
    Stream #0:0(eng): Video: h264 (High), yuv420p(progressive), 1280x536, SAR 1:1 DAR 160:67, 23.98 fps, 23.98 tbr, 1k tbn, 47.95 tbc (default)
    Stream #0:1(eng): Audio: ac3, 48000 Hz, 5.1(side), fltp, 640 kb/s (default)
    Stream #0:2(eng): Subtitle: subrip
    Stream #0:3(spa): Subtitle: subrip
[11:08:29] scan: decoding previews for title 1
[11:08:29] scan: audio 0x1: ac3, rate=48000Hz, bitrate=640000 English (AC3) (5.1 ch) (640 kbps)
[11:08:30] scan: 10 previews, 1280x536, 23.976 fps, autocrop = 0/0/0/0, aspect 2.39:1, PAR 1:1
[11:08:30] scan: supported video decoders: avcodec qsv
[11:08:30] libhb: scan thread found 1 valid title(s)
[11:08:30] Starting Task: Encoding Pass
[11:08:30] Skipping crop/scale filter
[11:08:30] work: only 1 chapter, disabling chapter markers
[11:08:30] job configuration:
[11:08:30]  * source
[11:08:30]    + E:\ynot\Multimedia\Video\Movies\Gran Torino.mkv
[11:08:30]    + title 1, chapter(s) 1 to 1
[11:08:30]    + container: matroska,webm
[11:08:30]    + data rate: 5369 kbps
[11:08:30]  * destination
[11:08:30]    + D:\ynot\Program Data\Handbrake\Gran Torino-4.mkv
[11:08:30]    + container: Matroska (libavformat)
[11:08:30]  * video track
[11:08:30]    + decoder: h264
[11:08:30]    + filter
[11:08:30]      + Framerate Shaper (mode=1)
[11:08:30]        + frame rate: 23.976 fps -> constant 23.976 fps
[11:08:30]    + Output geometry
[11:08:30]      + storage dimensions: 1280 x 536
[11:08:30]      + pixel aspect ratio: 1 : 1
[11:08:30]      + display dimensions: 1280 x 536
[11:08:30]    + encoder: H.264 (libx264)
[11:08:30]      + preset:  ultrafast
[11:08:30]      + profile: main
[11:08:30]      + level:   4.0
[11:08:30]      + quality: 22.00 (RF)
[11:08:30]      + color profile: 1-1-1
[11:08:30]  * audio track 1
[11:08:30]    + decoder: English (AC3) (5.1 ch) (640 kbps) (track 1, id 0x1)
[11:08:30]      + bitrate: 640 kbps, samplerate: 48000 Hz
[11:08:30]    + mixdown: 5.1 Channels
[11:08:30]    + dither: none
[11:08:30]    + encoder: AAC (libavcodec)
[11:08:30]      + bitrate: 512 kbps, samplerate: 48000 Hz
[11:08:30] sync: expecting 167847 video frames
[11:08:30] encx264: min-keyint: 24, keyint: 240
[11:08:30] encx264: encoding at constant RF 22.000000
[11:08:30] encx264: unparsed options: level=4.0:ref=1:scenecut=0:bframes=0:no-deblock=1:cabac=0:analyse=none:8x8dct=0:weightp=0:me=dia:subme=0:mixed-refs=0:vbv-bufsize=25000:vbv-maxrate=20000:aq-mode=0:mbtree=0:rc-lookahead=0
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
x264 [info]: profile Constrained Baseline, level 4.0, 4:2:0, 8-bit
[11:08:30] sync: first pts video is 0
[11:08:30] sync: "Chapter 1" (1) at frame 1 time 0
[11:08:30] sync: first pts audio 0x1 is 0
[11:13:51] reader: done. 1 scr changes
[11:13:52] work: average encoding speed for job is 521.906921 fps
[11:13:52] vfr: 167707 frames output, 0 dropped and 0 duped for CFR/PFR
[11:13:52] vfr: lost time: 0 (0 frames)
[11:13:52] vfr: gained time: 0 (0 frames) (0 not accounted for)
[11:13:52] ac3-decoder done: 218770 frames, 0 decoder errors
[11:13:52] h264-decoder done: 167707 frames, 0 decoder errors
[11:13:52] sync: got 167707 frames, 167847 expected
[11:13:52] sync: framerate min 23.976 fps, max 23.976 fps, avg 23.976 fps
x264 [info]: frame I:699   Avg QP:18.57  size: 76856
x264 [info]: frame P:167008 Avg QP:21.52  size: 19652
x264 [info]: mb I  I16..4: 100.0%  0.0%  0.0%
x264 [info]: mb P  I16..4:  7.3%  0.0%  0.0%  P16..4: 56.3%  0.0%  0.0%  0.0%  0.0%    skip:36.4%
x264 [info]: coded y,uvDC,uvAC intra: 38.6% 44.8% 7.7% inter: 32.8% 28.7% 0.4%
x264 [info]: i16 v,h,dc,p: 41% 25% 18% 16%
x264 [info]: i8c dc,h,v,p: 44% 22% 27%  7%
x264 [info]: kb/s:3815.24
[11:13:52] mux: track 0, 167707 frames, 3335815563 bytes, 3811.98 kbps, fifo 2048
[11:13:52] mux: track 1, 328156 frames, 444493676 bytes, 507.94 kbps, fifo 4096
[11:13:52] Finished work at: Sun Oct 18 11:13:52 2020
[11:13:52] libhb: work result = 0

# Encode Completed ...

[Deleted User 13735]

Audio delay by -- what is reported or what you can hear and see?
It is necessary with long GOP renders, such as Handbrake. All players, to my knowledge, honor the delay metadata, regardless of container.
Unless it looks and sounds like a spaghetti western, you're OK with the defaults.

[KnightAzul]

The audio delay introduced when encoding to an MKV is a few ms (-21 ms, -5 ms, etc.) which is reported by Mediainfo and MKVToolNix Info Tool. It is not noticeable when watching. So why I am asking the question? Well, call it curiosity. Also, rather than relying on players to respect flags, inserting silence/video frames to make sure that audio and video is in sync seems to be a more resilient solution.

The question really is: why does Handbrake only offer the option "Align A/V Start" when encoding to an MP4 and not when encoding to an MKV

My workaround for getting an audio stream with no audio delay could be to encode to an MP4 first with the option "Align A/V Start" checked and then remux the video and audio streams into an MKV using MKVToolNIX GUI.

Reading around on the net the best explanation as to why there is a delay introduced when the audio is reencoded is:

Eugen Rieck:

As counterintuitive as it sounds: This is not an error

The point is, that many audio codecs, can not start at PTS zero - and AAC is one of them. There is a "settling phase" at the beginning (I don't know the exact english word, in my native German it is "Einschwingphase"), where the audio output can not yet carry payload (silence is presented instead).

What all correct encoding tools do, is to start the audio track just so much earlier than the corresponding video track, that the PTS of the first image coincides with the PTS of the first payload carrying audio packet. This implies, that there are earlier audio packets rendered as silence.

Things start to become even messier, when you encode from one codec with a settling phase of X ms into another codec with a settling phase of Y ms - in this case, there might even be a positive delay, when the old codec needed longer to sttle than the new codec does.

This is BTW the reason, why exact edits are always done in a non-settling codec (typically a member of the PCM family).

With exactly the same Handbrake encode options but reencoding the audio to FLAC (lossless - which I guess counts as a non-settling codec) then in in the resultant MKV there is no audio delay relative to video shown by Mediainfo or MKVToolNix Info Tool. This suggests that the above quote could very well be the reason for delays.

KnightAzul

[Deleted User 13735]

MKV is a wrapper, not a format. The video format INSIDE the MKV wrapper, depending on which format is used, may or may not have a delay. If the video format inside the MKV wrapper is delayed correctly, you do not need a redundant flag in the MKV properties, which would be additive, if present. That's really all that is puzzling you, I think.

All modern long-GOP interframe formats, including mp4, do not use a/v interleave, because that is only possible in intraframe formats like AVI. So sync must be accomplished by another means, in this case by using a necessary delay factor, which approximates frame alignment, hopefully unnoticeably. You can change the delay amount in Yamb/MP4Box if you need to, but doing so is fraught with serpents and snakes. For investigational questions such as yours, Wikipedia is your friend.
https://en.wikipedia.org/wiki/Audio-to- ... ronization

Sufficient explanation for you, I hope

[Ritsuka]

Eugen Rieck is right. HandBrake explicitly writes the encoder delay in the mkv and mp4 files.
AAC and others format will always have an encoder delay, even if not explicitly stated in the container. The difference is that a player will not have to guess what the delay is because HandBrake wrote the exact value.

Here's some technical info on AAC encoder delay in mp4/mov: https://developer.apple.com/library/arc ... ppenG.html

[KnightAzul]

Ok, got it! Thanks for all the information.

Just one final question: why does Handbrake only offer the option "Align A/V Start" when encoding to an MP4 container and not when encoding to an MKV container?

Thanks,
KnightAzul

[Ritsuka]

The Align A/V Start is unrelated to the audio encoder delay. MP4 has got a feature called "edit lists" to change the presentation time of video and audio. Some players (mainly old tv built-in players) don't support it correctly, so the Align A/V Start feature tries to minimise the usage of edit lists.

Matroska is a completely different format, so this option is not applicable.