Handle Occasional Interlaced Frames in Progressive Film
Posted: Thu Jan 17, 2008 6:43 pm
I've found that many feature-film DVDs have interlaced frames right at or near chapter beginnings.
On a standalone player, one never sees those frames because a 2:3 pulldown is running the ntsc film rate movie at 29.97 fps.
I've verified that the actual DVD is mastered this way by turning off de-interlace in Apple DVD player, choosing a chapter, pausing quickly (space bar) and stepping forward (right arrow). The interlaced frames come in pairs near/surrounding chapter breaks.
HandBrake encodes these interlaced frames as it sees them and the result is progressive frames with mice teeth as shown here:
See images from HandBrake encodes:
During playback, they are hard to spot, but I always seem to notice them!
My pony is make HandBrake deal with these frames. Using a de-interlace I suspect.
I don't want to turn on de-interlace for the whole film because it unnecessarily softens progressive frames.
I've been told by one of the developers that the slow, slower and slowest de-interlace algorithms are not applied to progressive frames, but I can see the subtle difference, and I can show with screen grabs if necessary.
Any chance or thoughts on this?
Here is an activity log from a two chapter encode on a Mac Pro using a recent HandBrake SVN's Classic Preset:
[11:38:09] hb_init: checking for updates
[11:38:09] thread b013e000 started ("update")
[11:38:09] latest stable: 0.9.1, build 2007100800
[11:38:09] latest unstable: 0.9.1, build 2007100800
[11:38:09] thread b013e000 exited ("update")
[11:38:09] thread b013e000 joined ("update")
[11:38:09] hb_init: checking cpu count
[11:38:09] hb_init: starting libhb thread
[11:38:09] thread b013e000 started ("libhb")
[11:38:48] hb_scan: path=/Volumes/BIG MEDIA SHARE/Movies to be added/DVD Folders/Surfs Up, title_index=0
[11:38:48] thread b09c3000 started ("scan")
[11:38:48] scan: trying to open with libdvdread
[11:38:48] scan: DVD has 1 title(s)
[11:38:48] scan: scanning title 1
[11:38:48] scan: opening IFO for VTS 1
[11:38:48] pgc_id: 1, pgn: 1: pgc: 0x18618690
[11:38:48] scan: vts=1, ttn=1, cells=0->29, blocks=0->1466995, 1466996 blocks
[11:38:48] scan: duration is 01:25:25 (5125210 ms)
[11:38:48] scan: checking audio 1
[11:38:48] scan: id=80bd, lang=English (AC3), 3cc=eng
[11:38:48] scan: title 1 has 28 chapters
[11:38:48] scan: chap 1 c=0->2, b=0->74271 (74272), 256053 ms
[11:38:48] scan: chap 2 c=3->3, b=74272->101577 (27306), 114299 ms
[11:38:48] scan: chap 3 c=4->4, b=101578->163479 (61902), 235614 ms
[11:38:48] scan: chap 4 c=5->5, b=163480->201113 (37634), 121347 ms
[11:38:48] scan: chap 5 c=6->6, b=201114->254354 (53241), 133537 ms
[11:38:48] scan: chap 6 c=7->7, b=254355->304102 (49748), 200692 ms
[11:38:48] scan: chap 7 c=8->8, b=304103->336481 (32379), 117405 ms
[11:38:48] scan: chap 8 c=9->9, b=336482->403418 (66937), 205678 ms
[11:38:48] scan: chap 9 c=10->10, b=403419->464629 (61211), 162584 ms
[11:38:48] scan: chap 10 c=11->11, b=464630->500854 (36225), 133527 ms
[11:38:48] scan: chap 11 c=12->12, b=500855->592204 (91350), 298827 ms
[11:38:48] scan: chap 12 c=13->13, b=592205->642601 (50397), 225594 ms
[11:38:48] scan: chap 13 c=14->14, b=642602->663274 (20673), 87280 ms
[11:38:48] scan: chap 14 c=15->15, b=663275->711468 (48194), 185573 ms
[11:38:48] scan: chap 15 c=16->16, b=711469->751154 (39686), 173439 ms
[11:38:48] scan: chap 16 c=17->17, b=751155->794100 (42946), 149612 ms
[11:38:48] scan: chap 17 c=18->18, b=794101->898621 (104521), 298594 ms
[11:38:48] scan: chap 18 c=19->19, b=898622->924263 (25642), 85276 ms
[11:38:48] scan: chap 19 c=20->20, b=924264->967232 (42969), 167447 ms
[11:38:48] scan: chap 20 c=21->21, b=967233->1018877 (51645), 172447 ms
[11:38:48] scan: chap 21 c=22->22, b=1018878->1082058 (63181), 200402 ms
[11:38:48] scan: chap 22 c=23->23, b=1082059->1124374 (42316), 128527 ms
[11:38:48] scan: chap 23 c=24->24, b=1124375->1197529 (73155), 217501 ms
[11:38:48] scan: chap 24 c=25->25, b=1197530->1229063 (31534), 85286 ms
[11:38:48] scan: chap 25 c=26->26, b=1229064->1296091 (67028), 169551 ms
[11:38:48] scan: chap 26 c=27->27, b=1296092->1356697 (60606), 172357 ms
[11:38:48] scan: chap 27 c=28->28, b=1356698->1379632 (22935), 80377 ms
[11:38:48] scan: chap 28 c=29->29, b=1379633->1466995 (87363), 546369 ms
[11:38:48] scan: aspect = 16
[11:38:48] scan: decoding previews for title 1
[11:38:53] scan: preview 1
[11:38:53] scan: AC3, rate=48000Hz, bitrate=384000
[11:38:53] scan: preview 2
[11:38:53] scan: preview 3
[11:38:53] scan: preview 4
[11:38:54] scan: preview 5
[11:38:54] scan: preview 6
[11:38:54] Title's mostly progressive NTSC, setting fps to 23.976
[11:38:54] scan: preview 7
[11:38:54] scan: preview 8
[11:38:54] scan: preview 9
[11:38:54] scan: preview 10
[11:38:54] scan: 720x480, 23.976 fps, autocrop = 10/8/0/0
[11:38:54] scan: title (0) job->width:720, job->height:384
[11:38:54] thread b09c3000 exited ("scan")
[11:38:54] thread b09c3000 joined ("scan")
[11:38:54] libhb: scan thread found 1 valid title(s)
[11:39:33] thread b083d000 started ("work")
[11:39:33] 1 job(s) to process
[11:39:33] starting job
[11:39:33] + device /Volumes/BIG MEDIA SHARE/Movies to be added/DVD Folders/Surfs Up
[11:39:33] + title 1, chapter(s) 16 to 17
[11:39:33] + 720x480 -> 720x384, crop 10/8/0/0
[11:39:33] + grayscale off
[11:39:33] + filters
[11:39:33] + 23.976 fps, video bitrate 1000 kbps, pass 0
[11:39:33] + PixelRatio: 0, width:720, height: 384
[11:39:33] + encoder FFmpeg
[11:39:33] + audio 160 kbps, 48000 Hz
[11:39:33] + encoder faac
[11:39:33] + dynamic range compression: 1.000000
[11:39:33] + 80bd, English (AC3) (5.1 ch)
[11:39:33] + Requested mixdown: Dolby Pro Logic II (HB_AMIXDOWN_DOLBYPLII)
[11:39:33] + Actual mixdown: Dolby Pro Logic II (HB_AMIXDOWN_DOLBYPLII)
[11:39:33] thread b0941000 started ("reader")
[11:39:33] + output: /Volumes/Funky Monkey/Video Temp/Movies/Surfs Up-16-17.m4v
[11:39:33] thread b09c3000 started ("muxer")
[11:39:33] thread b0a45000 started ("MPEG-2 decoder (libmpeg2)")
[11:39:33] thread b0ac7000 started ("Renderer")
[mpeg4 @ 0x485470]removing common factors from framerate
[11:39:33] thread b0b49000 started ("MPEG-4 encoder (libavcodec)")
No accelerated IMDCT transform found
[11:39:33] thread b0bcb000 started ("AC3 decoder")
[11:39:33] thread b0c4d000 started ("AAC encoder (libfaac)")
[11:39:33] sync: expecting 10770 video frames
[11:39:37] DVD: Beginning of Cell (17) at block 751155
[11:39:37] sync: first pts is 217961672
[11:39:37] Sync: Audio PTS (217940540) < Video PTS (217961672) by greater than 100ms, trashing audio to reconverge
[11:39:37] Sync: Audio back in Sync at PTS 217954940
[11:40:09] DVD: End of Cell (17) at block 793934
[11:40:09] DVD: Beginning of Cell (18) at block 794101
[11:40:09] DVD: Chapter Break Cell Found
[11:40:09] MPEG2: Chapter Break Cell Found, searching for GOP
[11:40:09] MPEG2: Group of pictures found, searching for I-Frame
[11:40:09] MPEG2: I-Frame Found
[11:40:09] MPEG2: Chapter Break Inserted
[11:40:09] 2571.913086s: Progressive -> Interlaced
[11:40:09] 2571.946289s: Interlaced -> Progressive
[11:40:09] WORK: Copying Chapter Break
[11:40:09] WORK: Copying Chapter Break
[11:40:09] WORK: Copying Chapter Break
[11:41:27] DVD: End of Cell (18) at block 898369
[11:41:27] DVD: Beginning of Cell (19) at block 898622
[11:41:27] DVD: Chapter Break Cell Found
[11:41:27] reader: end of chapter 17 (media 17) reached at media chapter 18
[11:41:27] reader: done
[11:41:27] thread b0941000 exited ("reader")
[11:41:27] sync: got 10757 frames, 10770 expected
[11:41:27] Reader has exited early, inserting silence.
[11:41:27] thread b0c4d000 exited ("AAC encoder (libfaac)")
[11:41:27] thread b0b49000 exited ("MPEG-4 encoder (libavcodec)")
[11:41:27] thread b0ac7000 exited ("Renderer")
[11:41:27] thread b0a45000 exited ("MPEG-2 decoder (libmpeg2)")
[11:41:27] thread b0bcb000 exited ("AC3 decoder")
[11:41:27] thread b0a45000 joined ("MPEG-2 decoder (libmpeg2)")
[11:41:27] thread b0ac7000 joined ("Renderer")
[11:41:27] render: dropped frames: 0 (0 ticks)
[11:41:27] render: extended frames: 0 (0 ticks)
[11:41:27] render: Lost time: 0 frames (0 ticks)
[11:41:27] fifo_close: trashing 0 buffer(s)
[11:41:27] fifo_close: trashing 2 buffer(s)
[11:41:27] thread b0b49000 joined ("MPEG-4 encoder (libavcodec)")
[11:41:27] encavcodec: closing libavcodec
[11:41:27] thread b0bcb000 joined ("AC3 decoder")
[11:41:27] thread b0c4d000 joined ("AAC encoder (libfaac)")
[11:41:27] thread b0941000 joined ("reader")
[11:41:28] mux: file size, 65465094 bytes
[11:41:28] mux: track 0, 56298527 bytes, 1004.05 kbps
[11:41:28] mux: video bitrate error, +226887 bytes
[11:41:28] mux: track 1, 8969578 bytes, 159.97 kbps
[11:41:28] mux: overhead, 6.20 bytes per frame
[11:41:28] thread b09c3000 exited ("muxer")
[11:41:28] thread b09c3000 joined ("muxer")
[11:41:28] fifo_close: trashing 0 buffer(s)
[11:41:28] fifo_close: trashing 0 buffer(s)
[11:41:28] fifo_close: trashing 0 buffer(s)
[11:41:28] fifo_close: trashing 0 buffer(s)
[11:41:28] fifo_close: trashing 0 buffer(s)
[11:41:28] fifo_close: trashing 0 buffer(s)
[11:41:28] fifo_close: trashing 0 buffer(s)
[11:41:28] fifo_close: trashing 0 buffer(s)
[11:41:28] fifo_close: trashing 1 buffer(s)
[11:41:28] Freed 1 buffers of size 512
[11:41:28] Freed 0 buffers of size 1024
[11:41:28] Freed 45 buffers of size 2048
[11:41:28] Freed 0 buffers of size 4096
[11:41:28] Freed 0 buffers of size 8192
[11:41:28] Freed 22 buffers of size 16384
[11:41:28] Freed 0 buffers of size 32768
[11:41:28] Freed 29 buffers of size 518400
[11:41:28] Allocated 15486720 bytes of buffers on this pass and Freed 15486720 bytes, 0 bytes leaked
[11:41:28] thread b083d000 exited ("work")
[11:41:28] thread b083d000 joined ("work")
[11:41:28] libhb: work result = 0
Thanks
--sdm.
On a standalone player, one never sees those frames because a 2:3 pulldown is running the ntsc film rate movie at 29.97 fps.
I've verified that the actual DVD is mastered this way by turning off de-interlace in Apple DVD player, choosing a chapter, pausing quickly (space bar) and stepping forward (right arrow). The interlaced frames come in pairs near/surrounding chapter breaks.
HandBrake encodes these interlaced frames as it sees them and the result is progressive frames with mice teeth as shown here:
See images from HandBrake encodes:
During playback, they are hard to spot, but I always seem to notice them!
My pony is make HandBrake deal with these frames. Using a de-interlace I suspect.
I don't want to turn on de-interlace for the whole film because it unnecessarily softens progressive frames.
I've been told by one of the developers that the slow, slower and slowest de-interlace algorithms are not applied to progressive frames, but I can see the subtle difference, and I can show with screen grabs if necessary.
Any chance or thoughts on this?
Here is an activity log from a two chapter encode on a Mac Pro using a recent HandBrake SVN's Classic Preset:
[11:38:09] hb_init: checking for updates
[11:38:09] thread b013e000 started ("update")
[11:38:09] latest stable: 0.9.1, build 2007100800
[11:38:09] latest unstable: 0.9.1, build 2007100800
[11:38:09] thread b013e000 exited ("update")
[11:38:09] thread b013e000 joined ("update")
[11:38:09] hb_init: checking cpu count
[11:38:09] hb_init: starting libhb thread
[11:38:09] thread b013e000 started ("libhb")
[11:38:48] hb_scan: path=/Volumes/BIG MEDIA SHARE/Movies to be added/DVD Folders/Surfs Up, title_index=0
[11:38:48] thread b09c3000 started ("scan")
[11:38:48] scan: trying to open with libdvdread
[11:38:48] scan: DVD has 1 title(s)
[11:38:48] scan: scanning title 1
[11:38:48] scan: opening IFO for VTS 1
[11:38:48] pgc_id: 1, pgn: 1: pgc: 0x18618690
[11:38:48] scan: vts=1, ttn=1, cells=0->29, blocks=0->1466995, 1466996 blocks
[11:38:48] scan: duration is 01:25:25 (5125210 ms)
[11:38:48] scan: checking audio 1
[11:38:48] scan: id=80bd, lang=English (AC3), 3cc=eng
[11:38:48] scan: title 1 has 28 chapters
[11:38:48] scan: chap 1 c=0->2, b=0->74271 (74272), 256053 ms
[11:38:48] scan: chap 2 c=3->3, b=74272->101577 (27306), 114299 ms
[11:38:48] scan: chap 3 c=4->4, b=101578->163479 (61902), 235614 ms
[11:38:48] scan: chap 4 c=5->5, b=163480->201113 (37634), 121347 ms
[11:38:48] scan: chap 5 c=6->6, b=201114->254354 (53241), 133537 ms
[11:38:48] scan: chap 6 c=7->7, b=254355->304102 (49748), 200692 ms
[11:38:48] scan: chap 7 c=8->8, b=304103->336481 (32379), 117405 ms
[11:38:48] scan: chap 8 c=9->9, b=336482->403418 (66937), 205678 ms
[11:38:48] scan: chap 9 c=10->10, b=403419->464629 (61211), 162584 ms
[11:38:48] scan: chap 10 c=11->11, b=464630->500854 (36225), 133527 ms
[11:38:48] scan: chap 11 c=12->12, b=500855->592204 (91350), 298827 ms
[11:38:48] scan: chap 12 c=13->13, b=592205->642601 (50397), 225594 ms
[11:38:48] scan: chap 13 c=14->14, b=642602->663274 (20673), 87280 ms
[11:38:48] scan: chap 14 c=15->15, b=663275->711468 (48194), 185573 ms
[11:38:48] scan: chap 15 c=16->16, b=711469->751154 (39686), 173439 ms
[11:38:48] scan: chap 16 c=17->17, b=751155->794100 (42946), 149612 ms
[11:38:48] scan: chap 17 c=18->18, b=794101->898621 (104521), 298594 ms
[11:38:48] scan: chap 18 c=19->19, b=898622->924263 (25642), 85276 ms
[11:38:48] scan: chap 19 c=20->20, b=924264->967232 (42969), 167447 ms
[11:38:48] scan: chap 20 c=21->21, b=967233->1018877 (51645), 172447 ms
[11:38:48] scan: chap 21 c=22->22, b=1018878->1082058 (63181), 200402 ms
[11:38:48] scan: chap 22 c=23->23, b=1082059->1124374 (42316), 128527 ms
[11:38:48] scan: chap 23 c=24->24, b=1124375->1197529 (73155), 217501 ms
[11:38:48] scan: chap 24 c=25->25, b=1197530->1229063 (31534), 85286 ms
[11:38:48] scan: chap 25 c=26->26, b=1229064->1296091 (67028), 169551 ms
[11:38:48] scan: chap 26 c=27->27, b=1296092->1356697 (60606), 172357 ms
[11:38:48] scan: chap 27 c=28->28, b=1356698->1379632 (22935), 80377 ms
[11:38:48] scan: chap 28 c=29->29, b=1379633->1466995 (87363), 546369 ms
[11:38:48] scan: aspect = 16
[11:38:48] scan: decoding previews for title 1
[11:38:53] scan: preview 1
[11:38:53] scan: AC3, rate=48000Hz, bitrate=384000
[11:38:53] scan: preview 2
[11:38:53] scan: preview 3
[11:38:53] scan: preview 4
[11:38:54] scan: preview 5
[11:38:54] scan: preview 6
[11:38:54] Title's mostly progressive NTSC, setting fps to 23.976
[11:38:54] scan: preview 7
[11:38:54] scan: preview 8
[11:38:54] scan: preview 9
[11:38:54] scan: preview 10
[11:38:54] scan: 720x480, 23.976 fps, autocrop = 10/8/0/0
[11:38:54] scan: title (0) job->width:720, job->height:384
[11:38:54] thread b09c3000 exited ("scan")
[11:38:54] thread b09c3000 joined ("scan")
[11:38:54] libhb: scan thread found 1 valid title(s)
[11:39:33] thread b083d000 started ("work")
[11:39:33] 1 job(s) to process
[11:39:33] starting job
[11:39:33] + device /Volumes/BIG MEDIA SHARE/Movies to be added/DVD Folders/Surfs Up
[11:39:33] + title 1, chapter(s) 16 to 17
[11:39:33] + 720x480 -> 720x384, crop 10/8/0/0
[11:39:33] + grayscale off
[11:39:33] + filters
[11:39:33] + 23.976 fps, video bitrate 1000 kbps, pass 0
[11:39:33] + PixelRatio: 0, width:720, height: 384
[11:39:33] + encoder FFmpeg
[11:39:33] + audio 160 kbps, 48000 Hz
[11:39:33] + encoder faac
[11:39:33] + dynamic range compression: 1.000000
[11:39:33] + 80bd, English (AC3) (5.1 ch)
[11:39:33] + Requested mixdown: Dolby Pro Logic II (HB_AMIXDOWN_DOLBYPLII)
[11:39:33] + Actual mixdown: Dolby Pro Logic II (HB_AMIXDOWN_DOLBYPLII)
[11:39:33] thread b0941000 started ("reader")
[11:39:33] + output: /Volumes/Funky Monkey/Video Temp/Movies/Surfs Up-16-17.m4v
[11:39:33] thread b09c3000 started ("muxer")
[11:39:33] thread b0a45000 started ("MPEG-2 decoder (libmpeg2)")
[11:39:33] thread b0ac7000 started ("Renderer")
[mpeg4 @ 0x485470]removing common factors from framerate
[11:39:33] thread b0b49000 started ("MPEG-4 encoder (libavcodec)")
No accelerated IMDCT transform found
[11:39:33] thread b0bcb000 started ("AC3 decoder")
[11:39:33] thread b0c4d000 started ("AAC encoder (libfaac)")
[11:39:33] sync: expecting 10770 video frames
[11:39:37] DVD: Beginning of Cell (17) at block 751155
[11:39:37] sync: first pts is 217961672
[11:39:37] Sync: Audio PTS (217940540) < Video PTS (217961672) by greater than 100ms, trashing audio to reconverge
[11:39:37] Sync: Audio back in Sync at PTS 217954940
[11:40:09] DVD: End of Cell (17) at block 793934
[11:40:09] DVD: Beginning of Cell (18) at block 794101
[11:40:09] DVD: Chapter Break Cell Found
[11:40:09] MPEG2: Chapter Break Cell Found, searching for GOP
[11:40:09] MPEG2: Group of pictures found, searching for I-Frame
[11:40:09] MPEG2: I-Frame Found
[11:40:09] MPEG2: Chapter Break Inserted
[11:40:09] 2571.913086s: Progressive -> Interlaced
[11:40:09] 2571.946289s: Interlaced -> Progressive
[11:40:09] WORK: Copying Chapter Break
[11:40:09] WORK: Copying Chapter Break
[11:40:09] WORK: Copying Chapter Break
[11:41:27] DVD: End of Cell (18) at block 898369
[11:41:27] DVD: Beginning of Cell (19) at block 898622
[11:41:27] DVD: Chapter Break Cell Found
[11:41:27] reader: end of chapter 17 (media 17) reached at media chapter 18
[11:41:27] reader: done
[11:41:27] thread b0941000 exited ("reader")
[11:41:27] sync: got 10757 frames, 10770 expected
[11:41:27] Reader has exited early, inserting silence.
[11:41:27] thread b0c4d000 exited ("AAC encoder (libfaac)")
[11:41:27] thread b0b49000 exited ("MPEG-4 encoder (libavcodec)")
[11:41:27] thread b0ac7000 exited ("Renderer")
[11:41:27] thread b0a45000 exited ("MPEG-2 decoder (libmpeg2)")
[11:41:27] thread b0bcb000 exited ("AC3 decoder")
[11:41:27] thread b0a45000 joined ("MPEG-2 decoder (libmpeg2)")
[11:41:27] thread b0ac7000 joined ("Renderer")
[11:41:27] render: dropped frames: 0 (0 ticks)
[11:41:27] render: extended frames: 0 (0 ticks)
[11:41:27] render: Lost time: 0 frames (0 ticks)
[11:41:27] fifo_close: trashing 0 buffer(s)
[11:41:27] fifo_close: trashing 2 buffer(s)
[11:41:27] thread b0b49000 joined ("MPEG-4 encoder (libavcodec)")
[11:41:27] encavcodec: closing libavcodec
[11:41:27] thread b0bcb000 joined ("AC3 decoder")
[11:41:27] thread b0c4d000 joined ("AAC encoder (libfaac)")
[11:41:27] thread b0941000 joined ("reader")
[11:41:28] mux: file size, 65465094 bytes
[11:41:28] mux: track 0, 56298527 bytes, 1004.05 kbps
[11:41:28] mux: video bitrate error, +226887 bytes
[11:41:28] mux: track 1, 8969578 bytes, 159.97 kbps
[11:41:28] mux: overhead, 6.20 bytes per frame
[11:41:28] thread b09c3000 exited ("muxer")
[11:41:28] thread b09c3000 joined ("muxer")
[11:41:28] fifo_close: trashing 0 buffer(s)
[11:41:28] fifo_close: trashing 0 buffer(s)
[11:41:28] fifo_close: trashing 0 buffer(s)
[11:41:28] fifo_close: trashing 0 buffer(s)
[11:41:28] fifo_close: trashing 0 buffer(s)
[11:41:28] fifo_close: trashing 0 buffer(s)
[11:41:28] fifo_close: trashing 0 buffer(s)
[11:41:28] fifo_close: trashing 0 buffer(s)
[11:41:28] fifo_close: trashing 1 buffer(s)
[11:41:28] Freed 1 buffers of size 512
[11:41:28] Freed 0 buffers of size 1024
[11:41:28] Freed 45 buffers of size 2048
[11:41:28] Freed 0 buffers of size 4096
[11:41:28] Freed 0 buffers of size 8192
[11:41:28] Freed 22 buffers of size 16384
[11:41:28] Freed 0 buffers of size 32768
[11:41:28] Freed 29 buffers of size 518400
[11:41:28] Allocated 15486720 bytes of buffers on this pass and Freed 15486720 bytes, 0 bytes leaked
[11:41:28] thread b083d000 exited ("work")
[11:41:28] thread b083d000 joined ("work")
[11:41:28] libhb: work result = 0
Thanks
--sdm.