Combing Detection

Archive of historical development discussions
Discussions / Development has moved to GitHub
Forum rules
*******************************
Please be aware we are now using GitHub for issue tracking and feature requests.
- This section of the forum is now closed to new topics.

*******************************
jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Rewind

Post by jbrjake »

"We have the most [thorough] test guy in the world... [I showed him this program and he asked,] 'but Rob, what if time runs backward?'" -- traditional nerd proverb

....soooooo....

I guess no one, myself included, ever actually, um, looked at the output of threaded decomb.

At the top of yadif_decomb_filter_thread() was:

Code: Select all

int is_combed = pv->yadif_arguments[segment].is_combed;
Just a *slight* problem there -- yadif_decomb_filter_thread() is called only once, when the filter initializes. Before filtering starts. The part of the function that actually gets called on each frame is inside the while( run ) loop.

End result: threaded decomb is so fast because it was never actually calling yadif. It set up threads for comb detection and deinterlacing, performed threaded comb detection, and handed the frame off to the yadif threads....which proceeded to see that is_combed was 0 (or worse, an uninitialized value) and promptly copied each line of input to output, unfiltered. :/

So all those fps stats? Totally useless. They need to be redone with this patch:

http://handbrake.fr/pastebin/pastebin.php?show=45

...and results will definitely be worse. We're back to asking: "Is parallelizing this stuff worth the pthreads overhead at all?" I can't even say whether or not it'll be faster than single-threaded. It seems to be a little bit faster on my tired old G5 but it's hard to tell for sure...
eddyg
Veteran User
Posts: 798
Joined: Mon Apr 23, 2007 3:34 am

Re: Combing Detection

Post by eddyg »

oops.
eddyg
Veteran User
Posts: 798
Joined: Mon Apr 23, 2007 3:34 am

Re: Combing Detection

Post by eddyg »

New stats on Mac Pro with 1080i input resizing down to 720p:

Single Threaded Decomb: 6.83fps
Multi Threaded Decomb (new patch): 13.10fps
Multi Threaded Yadif (slower): 15.7fps

Note that this is for 1080i input, so pretty much every frame is deinterlaced, with mixed input decomb would probably fair better against having deinterlace on all the time.

Cheers, Ed.
dynaflash
Veteran User
Posts: 3820
Joined: Thu Nov 02, 2006 8:19 pm

Re: Combing Detection

Post by dynaflash »

Quick test using two chapters of a Def leppard Dvd.

Def Leppard ch. 1-2 Time: 07:49

No Decomb:
[08:18:38] work: average encoding speed for job is 192.02 fps

Unthreaded:
[08:15:09] decomb: yadif deinterlaced 9704 | blend deinterlaced 1329 | unfiltered 3056 | total 14089
[08:15:06] work: average encoding speed for job is 38.40 fps

Threaded:
[08:23:54] decomb: yadif deinterlaced 9704 | blend deinterlaced 1329 | unfiltered 3056 | total 14089
[08:23:51] work: average encoding speed for job is 55.33 fps

Definitely an improvement over unthreaded here.
jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Re: Combing Detection

Post by jbrjake »

Now that EEDI2's in there, I'm going to get started with some big changes for decomb. I haven't figured out what, exactly, yet, but ideas include:

- Using overlapping windows for checking the combing mask
- Looking in more directions for adjacent combed pixels in the mask
- Generating a full motion mask
- Running the erosion+dilation combo from EEDI2 on the combing (+ motion) masks to reduce noise, so I can raise sensitivity
- Only running EEDI2 on frames at the beginning and end of motion sequences, since that's when the visual improvement will be most noticeable.
- Only running EEDI2 on the Y plane, and letting chroma suffice with yadif, to save time.
- If someone wanted to help me with setting up the signal processing functions to make them, I'd love to see what frequency analysis can do for this sort of filter...

Anyway, that's just some long-delayed brainstorming.

Here's a patch that reorganizes the way modes work in decomb, to make it more flexible, albeit a lot more nerdy:
http://handbrake.fr/pastebin/pastebin.php?show=498

Mode is the first parameter decomb accepts in its settings strings. Old modes were:

Code: Select all

0: Yadif and blending
1: Yadif and blending and cubic interpolation
2: Yadif and blending and mcdeint
3: Yadif and blending and cubic interpolation and mcdeint
4: Blending
5: Yadif and blending and eedi2
The numbers were pretty much arbitrary.

New system looks like this:

Code: Select all

Mode    128    64    32    16    8    4    2    1
Yadif                                           1
Blend                                      1
Cubic                                 1
EEDI2                            1
Mcdint                      1
The idea is that this way it's easy to mix and match features without defining a specific mode for every combination. The old modes end up as:

Code: Select all

3: Yadif and blending
7: Yadif and blending and cubic interpolation (default)
19: Yadif and blending and mcdeint (not working quite yet)
23: Yadif and blending and cubic interpolation and mcdeint (not working quite yet)
2: Blending
11: Yadif and blending and eedi2
So for those who don't speak binary, every feature has a score. You add the scores up to tell the filter which modes to employ.

Code: Select all

Yadif is worth 1 point.
Blending is worth 2 points.
Cubic interpolation is worth 4 points.
EEDI2 interpolation is worth 8 points.
Mcdeint filtering is worth 16 points.

Yadif + Blending + EEDI2 = 1 + 2 + 8 = 11
You can add together the ones you want, just be aware that mcdeint currently doesn't work right, eedi2 and cubic are mutually exclusive for now, and if you don't include at least a 1 or a 2 in there, no frames are going to get filtered in the output. Also, when blending and yadif are both enabled (if you've added in both 1 and 2) decomb uses its traditional behavior of blending progressive frames and blending video frames that are greater than or equal to half of the blocking threshold.

@Elan, to continue from the EEDI2 thread, this means you force its use on all frames detected as combed by sending 9. Yadif + EEDI2 = 1 + 8 = 9.

Oh yeah, and for anyone who didn't follow the EEDI2 thread, in r2264 I added a new value for the spatial metric (the second parameter, whereas the mode I was just talking about is the first parameter in the settings string). The new value is -1, and it forces detection of all frames as combed and sends them to yadif if it's enabled or blend if it isn't. So with the above patch, --decomb=9:-1 will use yadif and EEDI2 on every frame of the source.
eddyg
Veteran User
Posts: 798
Joined: Mon Apr 23, 2007 3:34 am

Re: Combing Detection

Post by eddyg »

jbrjake wrote: Oh yeah, and for anyone who didn't follow the EEDI2 thread, in r2264 I added a new value for the spatial metric (the second parameter, whereas the mode I was just talking about is the first parameter in the settings string). The new value is -1, and it forces detection of all frames as combed and sends them to yadif if it's enabled or blend if it isn't. So with the above patch, --decomb=9:-1 will use yadif and EEDI2 on every frame of the source.
So what you're describing is a new deinterlace mode, that's pretty cool! I wonder if that will make a reasonable (faster) alternative to "slower" for stuff that is 100% interlaced??

Cheers, Ed.
jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Re: Combing Detection

Post by jbrjake »

eddyg wrote:So what you're describing is a new deinterlace mode, that's pretty cool! I wonder if that will make a reasonable (faster) alternative to "slower" for stuff that is 100% interlaced??
No, not at all. EEDI2's rather more sluggish than yadif. I ported it for its quality, not its speed. And actually, it's still using yadif in slower mode. It just skips the YADIF_CHECK spatial prediction and replaces it with a better one before doing the spatio-temporal adjustments.

Temporal adjustments in Slow mode:
- Compares current pixel against that location 1 frame back
- Compares the pixels directly above and below against those locations 1 frame back
- Compares the pixels directly above and below against those locations 1 frame forward
- Maximum value becomes diff.

Additional spatio-temporal adjustments in Slower mode:
- Averages the pixel two lines above with the value in that location 2 fields back as variable B
- Averages the pixel two lines below with the value in that location 2 fields back as variable F
- Finds the difference between B and the pixel directly above, and the difference between F and the pixel directly below. The smaller value will be fed to max, the larger to min.
- Max then sees which is largest: that smaller value from the last step, the difference between the average of the current pixel and that location 1 frame back and the pixel below, or the difference between the average of the current pixel and that location 1 frame back and the pixel above.
- Min then sees which is smallest: that larger value from 2 steps back, the difference between the average of the current pixel and that location 1 frame back and the pixel below, or the difference between the average of the current pixel and that location 1 frame back and the pixel above.
- The largest of absolute value of the diff from slow mode, min, and max, becomes the new diff.

In both modes:
- If the spatial prediction is less than the average of the pixel in the previous and current frames plus the diff, it gets replaced by it.
- If the spatial prediction is more than the average of the pixel in the previous and current frames minus the diff, it gets replaced by it.

I've spent a lot of time staring at the code for that stuff, and my understanding of it is always evolving (I think I've described it vastly differently in two other ways). So sorry for the literalness of the above transliteration, I'm sticking with the code as closely as possible and trying to guess as little about the rest as I can. A lot of yadif documentation out there claims that what's added in slower mode is a "spatial" check. That's kind of misleading. It's still doing adjustments over time, looking between fields. However, it's mostly looking between fields in the same frame.

Currently I don't have a mode to skip yadif altogether and only use eedi2, although I did in early testing. I removed it because it's not very worthwhile. The only true timesink in yadif is the spatial interpolation's edge detection, YADIF_CHECK. That's the part we're skipping and replacing with EEDI2'S far slower interpolation that goes over a much larger search distance. In comparison, the temporal stuff takes very little time, and there's no reason not to use it, since it compensates for EEDI2's only weakness -- it only works off a single field of input.
Elan
Posts: 35
Joined: Wed Jan 30, 2008 6:32 am

Re: Combing Detection

Post by Elan »

Great!

I will give it a try... Mode 9 will be okay since 8 won't filter anything. ;)

I like the idea of a binary system like Unix permissions. This will give a lot of freedom to "power" users.

Thanks

EDIT : Jbrjake, could you check the patch (#498) because I'm unable to apply it on a clean checkout of SVN2254 and over.

Code: Select all

patching file /Users/admin/Documents/Handbrake/libhb/decomb.c
patch unexpectedly ends in middle of line
Hunk #16 succeeded at 1907 with fuzz 2.
I carefully made the changes by hand and I double check it but Handbrake is unstable (Mac GUI x86_64) :
- Using the mode 9:-1 for all frames result in unexpected quit of the app when I press srart.
- Unexpected quit when I stop a job during the encode using mode 9.

Maybe I should use the CLI... I didn't try with other modes but I will wait for a patch that apply correctly and then I will post activity/crash log if the issues are still there.
jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Re: Combing Detection

Post by jbrjake »

Err, that patch is applying. If it wasn't it'd say "Patch failed." I ran it on an up to date checkout here and got the same thing:

Code: Select all

jonathon-rubins-macbook-pro:build jonr$ patch -p0 < ../decomb-binary-modes.patch 
patching file ../libhb/decomb.c
patch unexpectedly ends in middle of line
Hunk #16 succeeded at 1907 with fuzz 2.
...and it's working in both GUI and CLI.
Elan
Posts: 35
Joined: Wed Jan 30, 2008 6:32 am

Re: Combing Detection

Post by Elan »

Sorry, my mistake. :?

Everything is working fine, forget all I add in the "Edit" of my last post.
jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Re: Combing Detection

Post by jbrjake »

Here's an updated mode patch, with some more of the weirder modes working:
http://handbrake.fr/pastebin/pastebin.php?show=502

Working combos:
1: Just yadif
2: Just blend
3: Switch between yadif and blend based on combing intensity
4: Just cubic interpolate
5: Cubic->yadif
6: Switch between cubic and blend based on combing intensity
7 (default): Switch between cubic->yadif and blend based on combing intensity
8: Just EEDI2 interpolate
9: EEDI2->yadif
10: Switch between EEDI2 and blend based on combing intensity
11: Switch between EEDI2->yadif and blend based on combing intensity
So that means you can see what a plain EEDI2 interpolation (--decomb=8:-1) looks like compared to the yadif-adjusted version (--decomb=9:-1). Really shows off yadif's strengths at temporal adjustment, which can be hard to notice when using its weak internal spatial predictions.

Running cubic on its own is just a lark -- without any edge direction it's rather useless. But adding the functionality was easy and it helped flesh out the mode list. Not quite working perfectly yet...overreading on the last couple of scan lines so there's a little green fuzz at the bottom of frames.

Modes 12-15, which would try to use cubic interpolation and eedi2 interpolation at once, which is impossible, will just use eedi2. 16+ aren't enabled yet because of mcdeint issues, but I'm getting there.
Elan
Posts: 35
Joined: Wed Jan 30, 2008 6:32 am

Re: Combing Detection

Post by Elan »

Jbrjake,

For now, mode 9 give me the best result I ever had with Decomb. Tomorow, I will test and compare with mode 8 to see the diferences between these modes.

After mcdeint do you plan to add other new mode like a NNEDI port? Some say it's superior in quality to the "older" EEDI2. What you though?

http://forum.doom9.org/showthread.php?t=129953
Elan
Posts: 35
Joined: Wed Jan 30, 2008 6:32 am

Mode 8 vs Mode 9

Post by Elan »

I did some compare between mode 8 vs mode 9 with the option :-1 to apply it on all frames. Here is an interesting situation where mode 9 seams to do nothing. See by yourself :

Original frame :
Image

Mode : 8:-1 (EEDI2 only)
Image

Code: Select all

HandBrake Activity Log for /Volumes/USERS/Madonna - Give It 2 Me Inter 8.mp4: 2009-03-24 20:22:57 -0400
Handbrake Version: svn2271 x86_64 (2009032301)

[20:22:57] macgui: Rip: Pending queue count is 0
[20:22:57] macgui: Rip: No pending jobs, so sending this one to doAddToQueue
[20:22:57] macgui: Rip: Going right to performNewQueueScan
[20:22:57] macgui: scanning specifically for title: 1
[20:22:57] macgui: performNewQueueScan currentQueueEncodeIndex is: 0
[20:22:57] hb_scan: path=/Volumes/DISK2/Madonna - Give It 2 Me Inter.mpg, title_index=1
[20:22:57] thread 20249000 started ("scan")
[20:22:57] scan: trying to open with libdvdread
[20:22:57] dvd: not a dvd - trying as a stream/file instead
[20:22:57] file is MPEG DVD Program Stream
[20:22:57] add_audio_to_title: added AC3 audio stream 0x80bd
[20:22:57] scan: decoding previews for title 1
[20:22:57] scan: preview 1
[20:22:57] scan: audio 0x80bd: AC-3, rate=48000Hz, bitrate=192000 Unknown (AC3) (Dolby Surround)
[20:22:57] scan: preview 2
[20:22:57] Interlacing detected in preview frame 2
[20:22:57] scan: preview 3
[20:22:57] Interlacing detected in preview frame 3
[20:22:57] scan: preview 4
[20:22:57] scan: preview 5
[20:22:57] Interlacing detected in preview frame 5
[20:22:57] scan: preview 6
[20:22:57] Interlacing detected in preview frame 6
[20:22:57] scan: preview 7
[20:22:57] Interlacing detected in preview frame 7
[20:22:57] scan: preview 8
[20:22:57] scan: preview 9
[20:22:57] scan: preview 10
[20:22:57] scan: preview 11
[20:22:57] Interlacing detected in preview frame 11
[20:22:57] scan: preview 12
[20:22:57] scan: preview 13
[20:22:57] scan: preview 14
[20:22:57] Interlacing detected in preview frame 14
[20:22:57] scan: preview 15
[20:22:57] scan: preview 16
[20:22:57] Interlacing detected in preview frame 16
[20:22:57] scan: preview 17
[20:22:57] scan: preview 18
[20:22:57] Interlacing detected in preview frame 18
[20:22:57] scan: preview 19
[20:22:57] Interlacing detected in preview frame 19
[20:22:57] scan: preview 20
[20:22:57] Interlacing detected in preview frame 20
[20:22:57] scan: preview 21
[20:22:57] scan: preview 22
[20:22:57] Interlacing detected in preview frame 22
[20:22:57] scan: preview 23
[20:22:57] scan: preview 24
[20:22:57] scan: preview 25
[20:22:57] Interlacing detected in preview frame 25
[20:22:57] scan: preview 26
[20:22:57] Interlacing detected in preview frame 26
[20:22:57] scan: preview 27
[20:22:57] Interlacing detected in preview frame 27
[20:22:57] scan: preview 28
[20:22:57] Interlacing detected in preview frame 28
[20:22:57] scan: preview 29
[20:22:57] scan: preview 30
[20:22:57] Interlacing detected in preview frame 30
[20:22:57] scan: 30 previews, 704x480, 29.970 fps, autocrop = 62/44/0/0, aspect 4:3, PAR 10:11
[20:22:57] Title is likely interlaced or telecined (17 out of 30 previews). You should do something about that.
[20:22:57] scan: title (0) job->width:624, job->height:368
[20:22:57] thread 20249000 exited ("scan")
[20:22:58] thread 20249000 joined ("scan")
[20:22:58] libhb: scan thread found 1 valid title(s)
[20:22:58] macgui: ScanDone state received from fQueueEncodeLibhb
[20:22:58] macgui: processNewQueueEncode title list is: 1
[20:22:58] macgui: Preset: New3
[20:22:58] macgui: processNewQueueEncode number of passes expected is: 1
[20:22:58] macgui: prepareJob exiting
[20:22:58] thread 20249000 started ("work")
[20:22:58] 1 job(s) to process
[20:22:58] starting job
[20:22:58] job configuration:
[20:22:58]  * source
[20:22:58]    + /Volumes/DISK2/Madonna - Give It 2 Me Inter.mpg
[20:22:58]    + title 1, chapter(s) 1 to 1
[20:22:58]  * destination
[20:22:58]    + /Volumes/USERS/Madonna - Give It 2 Me Inter 8.mp4
[20:22:58]    + container: MPEG-4 (.mp4 and .m4v)
[20:22:58]      + 64-bit formatting
[20:22:58]      + optimized for progressive web downloads
[20:22:58]  * video track
[20:22:58]    + decoder: mpeg2
[20:22:58]      + bitrate 15000 kbps
[20:22:58]    + frame rate: 29.970 fps -> variable fps
[20:22:58]    + dimensions: 704 * 480 -> 704 * 480, crop 0/0/0/0
[20:22:58]    + filters
[20:22:58]      + Detelecine (pullup) (default settings)
[20:22:58]      + Decomb (8:-1)
[20:22:58]    + encoder: x264
[20:22:58]      + options: analyse=all:ref=6:8x8dct=1:subq=7:bframes=6:direct=auto:merange=16:trellis=2:me=umh:b-adapt=2:mixed-refs=1:psy-rd=1.0,0.2:weightb=1
[20:22:58]      + quality: 20.00 (RF)
[20:22:58]  * audio track 0
[20:22:58]    + decoder: Unknown (AC3) (Dolby Surround) (track 1, id 80bd)
[20:22:58]      + bitrate: 192 kbps, samplerate: 48000 Hz
[20:22:58]    + mixdown: Dolby Surround
[20:22:58]    + dynamic range compression: 1.000000
[20:22:58]    + encoder: ca_aac
[20:22:58]      + bitrate: 192 kbps, samplerate: 48000 Hz
[20:22:58] thread 205a5000 started ("reader")
[20:22:58] dvd: not a dvd - trying as a stream/file instead
[20:22:58] thread 20aed000 started ("MPEG-2 decoder (libmpeg2)")
[20:22:58] reader: first SCR 194
[20:22:58] thread 2172a000 started ("yadif_filter_segment")
[20:22:58] yadif thread started for segment 0
[20:22:58] thread 217ad000 started ("yadif_filter_segment")
[20:22:58] yadif thread started for segment 1
[20:22:58] thread 21830000 started ("yadif_filter_segment")
[20:22:58] yadif thread started for segment 2
[20:22:58] thread 218b3000 started ("yadif_filter_segment")
[20:22:58] yadif thread started for segment 3
[20:22:58] thread 21936000 started ("yadif_filter_segment")
[20:22:58] yadif thread started for segment 4
[20:22:58] thread 219b9000 started ("yadif_filter_segment")
[20:22:58] yadif thread started for segment 5
[20:22:58] thread 21a3c000 started ("yadif_filter_segment")
[20:22:58] yadif thread started for segment 6
[20:22:58] thread 21abf000 started ("yadif_filter_segment")
[20:22:58] yadif thread started for segment 7
[20:22:58] thread 21b42000 started ("decomb_filter_segment")
[20:22:58] decomb thread started for segment 0
[20:22:58] thread 21bc5000 started ("decomb_filter_segment")
[20:22:58] decomb thread started for segment 1
[20:22:58] thread 21c48000 started ("decomb_filter_segment")
[20:22:58] decomb thread started for segment 2
[20:22:58] thread 21ccb000 started ("decomb_filter_segment")
[20:22:58] decomb thread started for segment 3
[20:22:58] thread 21d4e000 started ("decomb_filter_segment")
[20:22:58] decomb thread started for segment 4
[20:22:58] thread 21dd1000 started ("decomb_filter_segment")
[20:22:58] decomb thread started for segment 5
[20:22:58] thread 21e54000 started ("decomb_filter_segment")
[20:22:58] decomb thread started for segment 6
[20:22:58] thread 21ed7000 started ("decomb_filter_segment")
[20:22:58] decomb thread started for segment 7
[20:22:58] thread 21f5a000 started ("eedi2_filter_segment")
[20:22:58] eedi2 thread started for plane 0
[20:22:58] thread 21fdd000 started ("eedi2_filter_segment")
[20:22:58] eedi2 thread started for plane 1
[20:22:58] thread 22060000 started ("eedi2_filter_segment")
[20:22:58] eedi2 thread started for plane 2
[20:22:58] thread 220e3000 started ("Renderer")
[20:22:58] mpeg2: "" (1) at frame 0 time 3003
[20:22:58] encx264: keyint-min: 30, keyint-max: 300
[20:22:58] encx264: Encoding at constant RF 20.000000
[20:22:58] encx264: opening libx264 (pass 0)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 PHADD SSE4.1 Cache64
x264 [info]: profile High, level 3.0
[20:22:58] thread 25e88000 started ("H.264/AVC encoder (libx264)")
No accelerated IMDCT transform found
[20:22:58] thread 25f0b000 started ("AC3 decoder")
[20:22:58] thread 2698e000 started ("AAC encoder (Apple)")
[20:22:58] thread 26a11000 started ("muxer")
[20:22:58] sync: expecting 7480 video frames
[20:22:58] sync: first pts is 3003
[20:22:58] muxmp4: using 64-bit MP4 formatting.
[20:44:15] reader: done. 0 scr changes
[20:44:15] thread 205a5000 exited ("reader")
[20:44:43] sync: got 7541 frames, 7480 expected
[20:44:43] work: average encoding speed for job is 5.777175 fps
[20:44:55] muxmp4: optimizing file
[20:44:55] mux: file size, 57785520 bytes
[20:44:55] mux: track 0, 50454830 bytes, 1605.08 kbps
[20:44:55] mux: video bitrate error, +50454830 bytes
[20:44:55] mux: track 1, 7160762 bytes, 227.80 kbps
[20:44:55] mux: overhead, 9.39 bytes per frame
[20:44:55] thread 26a11000 exited ("muxer")
[20:44:55] thread 26a11000 joined ("muxer")
[20:44:55] thread 25f0b000 exited ("AC3 decoder")
[20:44:55] thread 220e3000 exited ("Renderer")
[20:44:55] thread 20aed000 exited ("MPEG-2 decoder (libmpeg2)")
[20:44:55] thread 2698e000 exited ("AAC encoder (Apple)")
[20:44:55] thread 25e88000 exited ("H.264/AVC encoder (libx264)")
[20:44:55] thread 20aed000 joined ("MPEG-2 decoder (libmpeg2)")
[20:44:55] mpeg2 done: 7542 frames
[20:44:55] thread 220e3000 joined ("Renderer")
[20:44:55] render: lost time: 3723720 (1240 frames)
[20:44:55] render: gained time: 3722970 (4927 frames) (750 not accounted for)
[20:44:55] render: average dropped frame duration: 3003
[20:44:55] fifo_close: trashing 1 buffer(s)
[20:44:55] fifo_close: trashing 3 buffer(s)
[20:44:55] thread 25e88000 joined ("H.264/AVC encoder (libx264)")
x264 [info]: slice I:89    Avg QP:16.09  size: 19586  PSNR Mean Y:48.28 U:53.39 V:54.44 Avg:49.42 Global:48.92
x264 [info]: slice P:2959  Avg QP:17.52  size: 10850  PSNR Mean Y:46.27 U:52.19 V:53.64 Avg:47.51 Global:46.71
x264 [info]: slice B:3249  Avg QP:21.19  size:  5111  PSNR Mean Y:44.18 U:50.84 V:52.68 Avg:45.51 Global:44.81
x264 [info]: consecutive B-frames: 13.1% 39.4% 38.6%  5.1%  3.1%  0.2%  0.5%
x264 [info]: mb I  I16..4: 20.7% 68.6% 10.7%
x264 [info]: mb P  I16..4:  6.9% 27.6%  3.1%  P16..4: 25.3%  9.4%  3.8%  0.2%  0.1%    skip:23.5%
x264 [info]: mb B  I16..4:  0.9%  3.9%  0.4%  B16..8: 35.1%  2.8%  2.8%  direct: 8.7%  skip:45.4%  L0:42.5% L1:46.3% BI:11.2%
x264 [info]: 8x8 transform  intra:73.2%  inter:74.7%
x264 [info]: direct mvs  spatial:99.7%  temporal:0.3%
x264 [info]: ref P L0  69.6% 14.4%  7.0%  3.8%  2.8%  2.5%
x264 [info]: ref B L0  80.0% 10.6%  4.9%  2.6%  2.0%
x264 [info]: SSIM Mean Y:0.9907053
x264 [info]: PSNR Mean Y:45.220 U:51.511 V:53.153 Avg:46.505 Global:45.644 kb/s:1921.05
[20:44:56] thread 25f0b000 joined ("AC3 decoder")
[20:44:56] thread 2698e000 joined ("AAC encoder (Apple)")
[20:44:56] thread 205a5000 joined ("reader")
[20:44:56] fifo_close: trashing 0 buffer(s)
[20:44:56] fifo_close: trashing 1 buffer(s)
[20:44:56] fifo_close: trashing 0 buffer(s)
[20:44:56] fifo_close: trashing 0 buffer(s)
[20:44:56] fifo_close: trashing 0 buffer(s)
[20:44:56] fifo_close: trashing 0 buffer(s)
[20:44:56] fifo_close: trashing 0 buffer(s)
[20:44:56] fifo_close: trashing 0 buffer(s)
[20:44:56] fifo_close: trashing 0 buffer(s)
[20:44:56] decomb: yadif+eedi2 deinterlaced 6300 | blend deinterlaced 0 | unfiltered 0 | total 6300
[20:44:56] thread 2172a000 exited ("yadif_filter_segment")
[20:44:56] thread 2172a000 joined ("yadif_filter_segment")
[20:44:56] thread 217ad000 exited ("yadif_filter_segment")
[20:44:56] thread 217ad000 joined ("yadif_filter_segment")
[20:44:56] thread 21830000 exited ("yadif_filter_segment")
[20:44:56] thread 21830000 joined ("yadif_filter_segment")
[20:44:56] thread 218b3000 exited ("yadif_filter_segment")
[20:44:56] thread 218b3000 joined ("yadif_filter_segment")
[20:44:56] thread 21936000 exited ("yadif_filter_segment")
[20:44:56] thread 21936000 joined ("yadif_filter_segment")
[20:44:56] thread 219b9000 exited ("yadif_filter_segment")
[20:44:56] thread 219b9000 joined ("yadif_filter_segment")
[20:44:56] thread 21a3c000 exited ("yadif_filter_segment")
[20:44:56] thread 21a3c000 joined ("yadif_filter_segment")
[20:44:56] thread 21abf000 exited ("yadif_filter_segment")
[20:44:56] thread 21abf000 joined ("yadif_filter_segment")
[20:44:56] thread 21b42000 exited ("decomb_filter_segment")
[20:44:56] thread 21b42000 joined ("decomb_filter_segment")
[20:44:56] thread 21bc5000 exited ("decomb_filter_segment")
[20:44:56] thread 21bc5000 joined ("decomb_filter_segment")
[20:44:56] thread 21c48000 exited ("decomb_filter_segment")
[20:44:56] thread 21c48000 joined ("decomb_filter_segment")
[20:44:56] thread 21ccb000 exited ("decomb_filter_segment")
[20:44:56] thread 21ccb000 joined ("decomb_filter_segment")
[20:44:56] thread 21d4e000 exited ("decomb_filter_segment")
[20:44:56] thread 21d4e000 joined ("decomb_filter_segment")
[20:44:56] thread 21dd1000 exited ("decomb_filter_segment")
[20:44:56] thread 21dd1000 joined ("decomb_filter_segment")
[20:44:56] thread 21e54000 exited ("decomb_filter_segment")
[20:44:56] thread 21e54000 joined ("decomb_filter_segment")
[20:44:56] thread 21ed7000 exited ("decomb_filter_segment")
[20:44:56] thread 21ed7000 joined ("decomb_filter_segment")
[20:44:56] thread 21f5a000 exited ("eedi2_filter_segment")
[20:44:56] thread 21f5a000 joined ("eedi2_filter_segment")
[20:44:56] thread 21fdd000 exited ("eedi2_filter_segment")
[20:44:56] thread 21fdd000 joined ("eedi2_filter_segment")
[20:44:56] thread 22060000 exited ("eedi2_filter_segment")
[20:44:56] thread 22060000 joined ("eedi2_filter_segment")
[20:44:56] Freed 3 buffers of size 1024
[20:44:56] Freed 32 buffers of size 2048
[20:44:56] Freed 32 buffers of size 16384
[20:44:56] Freed 32 buffers of size 524288
[20:44:56] Allocated 17894400 bytes of buffers on this pass and Freed 17370112 bytes, 524288 bytes leaked
[20:44:56] thread 20249000 exited ("work")
Mode : 9:-1 (EEDI2 -> Yadif)
Image

Code: Select all

HandBrake Activity Log for /Volumes/USERS/Madonna - Give It 2 Me Inter 9.mp4: 2009-03-24 20:44:56 -0400
Handbrake Version: svn2271 x86_64 (2009032301)

[20:44:56] thread 20249000 joined ("work")
[20:44:56] libhb: work result = 0
[20:44:56] macgui: incrementQueueItemDone currentQueueEncodeIndex is incremented to: 1
[20:44:56] macgui: incrementQueueItemDone currentQueueEncodeIndex is incremented to: 1
[20:44:56] macgui: scanning specifically for title: 1
[20:44:56] macgui: performNewQueueScan currentQueueEncodeIndex is: 1
[20:44:56] hb_scan: path=/Volumes/DISK2/Madonna - Give It 2 Me Inter.mpg, title_index=1
[20:44:56] thread 14781000 started ("scan")
[20:44:56] scan: trying to open with libdvdread
[20:44:56] dvd: not a dvd - trying as a stream/file instead
[20:44:56] file is MPEG DVD Program Stream
[20:44:56] add_audio_to_title: added AC3 audio stream 0x80bd
[20:44:56] scan: decoding previews for title 1
[20:44:56] scan: preview 1
[20:44:56] scan: audio 0x80bd: AC-3, rate=48000Hz, bitrate=192000 Unknown (AC3) (Dolby Surround)
[20:44:56] scan: preview 2
[20:44:56] Interlacing detected in preview frame 2
[20:44:56] scan: preview 3
[20:44:56] Interlacing detected in preview frame 3
[20:44:56] scan: preview 4
[20:44:56] scan: preview 5
[20:44:56] Interlacing detected in preview frame 5
[20:44:56] scan: preview 6
[20:44:56] Interlacing detected in preview frame 6
[20:44:56] scan: preview 7
[20:44:56] Interlacing detected in preview frame 7
[20:44:56] scan: preview 8
[20:44:56] scan: preview 9
[20:44:56] scan: preview 10
[20:44:56] scan: preview 11
[20:44:56] Interlacing detected in preview frame 11
[20:44:56] scan: preview 12
[20:44:56] scan: preview 13
[20:44:56] scan: preview 14
[20:44:56] Interlacing detected in preview frame 14
[20:44:56] scan: preview 15
[20:44:56] scan: preview 16
[20:44:56] Interlacing detected in preview frame 16
[20:44:56] scan: preview 17
[20:44:56] scan: preview 18
[20:44:56] Interlacing detected in preview frame 18
[20:44:56] scan: preview 19
[20:44:56] Interlacing detected in preview frame 19
[20:44:56] scan: preview 20
[20:44:56] Interlacing detected in preview frame 20
[20:44:56] scan: preview 21
[20:44:56] scan: preview 22
[20:44:56] Interlacing detected in preview frame 22
[20:44:56] scan: preview 23
[20:44:56] scan: preview 24
[20:44:56] scan: preview 25
[20:44:56] Interlacing detected in preview frame 25
[20:44:56] scan: preview 26
[20:44:56] Interlacing detected in preview frame 26
[20:44:56] scan: preview 27
[20:44:56] Interlacing detected in preview frame 27
[20:44:56] scan: preview 28
[20:44:56] Interlacing detected in preview frame 28
[20:44:56] scan: preview 29
[20:44:56] scan: preview 30
[20:44:56] Interlacing detected in preview frame 30
[20:44:56] scan: 30 previews, 704x480, 29.970 fps, autocrop = 62/44/0/0, aspect 4:3, PAR 10:11
[20:44:56] Title is likely interlaced or telecined (17 out of 30 previews). You should do something about that.
[20:44:56] scan: title (0) job->width:624, job->height:368
[20:44:56] thread 14781000 exited ("scan")
[20:44:56] thread 14781000 joined ("scan")
[20:44:56] libhb: scan thread found 1 valid title(s)
[20:44:56] macgui: currentScanCount received from fQueueEncodeLibhb
[20:44:56] macgui: ScanDone state received from fQueueEncodeLibhb
[20:44:56] macgui: processNewQueueEncode title list is: 1
[20:44:56] macgui: Preset: New3
[20:44:56] macgui: processNewQueueEncode number of passes expected is: 1
[20:44:56] macgui: prepareJob exiting
[20:44:56] thread 144d5000 started ("work")
[20:44:56] 1 job(s) to process
[20:44:56] starting job
[20:44:56] job configuration:
[20:44:56]  * source
[20:44:56]    + /Volumes/DISK2/Madonna - Give It 2 Me Inter.mpg
[20:44:56]    + title 1, chapter(s) 1 to 1
[20:44:56]  * destination
[20:44:56]    + /Volumes/USERS/Madonna - Give It 2 Me Inter 9.mp4
[20:44:56]    + container: MPEG-4 (.mp4 and .m4v)
[20:44:56]      + 64-bit formatting
[20:44:56]      + optimized for progressive web downloads
[20:44:56]  * video track
[20:44:56]    + decoder: mpeg2
[20:44:56]      + bitrate 15000 kbps
[20:44:56]    + frame rate: 29.970 fps -> variable fps
[20:44:56]    + dimensions: 704 * 480 -> 704 * 480, crop 0/0/0/0
[20:44:56]    + filters
[20:44:56]      + Detelecine (pullup) (default settings)
[20:44:56]      + Decomb (9:-1)
[20:44:56]    + encoder: x264
[20:44:56]      + options: analyse=all:ref=6:8x8dct=1:subq=7:bframes=6:direct=auto:merange=16:trellis=2:me=umh:b-adapt=2:mixed-refs=1:psy-rd=1.0,0.2:weightb=1
[20:44:56]      + quality: 20.00 (RF)
[20:44:56]  * audio track 0
[20:44:56]    + decoder: Unknown (AC3) (Dolby Surround) (track 1, id 80bd)
[20:44:56]      + bitrate: 192 kbps, samplerate: 48000 Hz
[20:44:56]    + mixdown: Dolby Surround
[20:44:56]    + dynamic range compression: 1.000000
[20:44:56]    + encoder: ca_aac
[20:44:56]      + bitrate: 192 kbps, samplerate: 48000 Hz
[20:44:56] thread 14781000 started ("reader")
[20:44:56] thread 1492f000 started ("MPEG-2 decoder (libmpeg2)")
[20:44:56] dvd: not a dvd - trying as a stream/file instead
[20:44:56] reader: first SCR 194
[20:44:56] thread 21381000 started ("yadif_filter_segment")
[20:44:56] yadif thread started for segment 0
[20:44:56] thread 21404000 started ("yadif_filter_segment")
[20:44:56] yadif thread started for segment 1
[20:44:56] thread 21487000 started ("yadif_filter_segment")
[20:44:56] yadif thread started for segment 2
[20:44:56] thread 2150a000 started ("yadif_filter_segment")
[20:44:56] yadif thread started for segment 3
[20:44:56] thread 2158d000 started ("yadif_filter_segment")
[20:44:56] yadif thread started for segment 4
[20:44:56] thread 21610000 started ("yadif_filter_segment")
[20:44:56] yadif thread started for segment 5
[20:44:56] thread 21693000 started ("yadif_filter_segment")
[20:44:56] yadif thread started for segment 6
[20:44:56] thread 21716000 started ("yadif_filter_segment")
[20:44:56] yadif thread started for segment 7
[20:44:56] thread 21799000 started ("decomb_filter_segment")
[20:44:56] decomb thread started for segment 0
[20:44:56] thread 2181c000 started ("decomb_filter_segment")
[20:44:56] decomb thread started for segment 1
[20:44:56] thread 2189f000 started ("decomb_filter_segment")
[20:44:56] decomb thread started for segment 2
[20:44:56] thread 21922000 started ("decomb_filter_segment")
[20:44:56] decomb thread started for segment 3
[20:44:56] thread 219a5000 started ("decomb_filter_segment")
[20:44:56] decomb thread started for segment 4
[20:44:56] thread 21a28000 started ("decomb_filter_segment")
[20:44:56] decomb thread started for segment 5
[20:44:56] thread 21aab000 started ("decomb_filter_segment")
[20:44:56] decomb thread started for segment 6
[20:44:56] thread 21b2e000 started ("decomb_filter_segment")
[20:44:56] decomb thread started for segment 7
[20:44:56] thread 21bb1000 started ("eedi2_filter_segment")
[20:44:56] eedi2 thread started for plane 0
[20:44:56] thread 21c34000 started ("eedi2_filter_segment")
[20:44:56] eedi2 thread started for plane 1
[20:44:56] thread 21cb7000 started ("eedi2_filter_segment")
[20:44:56] eedi2 thread started for plane 2
[20:44:56] thread 21d3a000 started ("Renderer")
[20:44:56] encx264: keyint-min: 30, keyint-max: 300
[20:44:56] encx264: Encoding at constant RF 20.000000
[20:44:56] encx264: opening libx264 (pass 0)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 PHADD SSE4.1 Cache64
x264 [info]: profile High, level 3.0
[20:44:56] thread 24fe4000 started ("H.264/AVC encoder (libx264)")
No accelerated IMDCT transform found
[20:44:56] thread 25067000 started ("AC3 decoder")
[20:44:56] thread 25261000 started ("AAC encoder (Apple)")
[20:44:56] thread 252e4000 started ("muxer")
[20:44:56] sync: expecting 7480 video frames
[20:44:56] muxmp4: using 64-bit MP4 formatting.
[20:44:56] mpeg2: "" (1) at frame 0 time 3003
[20:44:56] sync: first pts is 3003
[21:06:08] reader: done. 0 scr changes
[21:06:08] thread 14781000 exited ("reader")
[21:06:36] sync: got 7541 frames, 7480 expected
[21:06:36] work: average encoding speed for job is 5.801202 fps
[21:06:48] muxmp4: optimizing file
[21:06:49] mux: file size, 57799687 bytes
[21:06:49] mux: track 0, 50468981 bytes, 1605.53 kbps
[21:06:49] mux: video bitrate error, +50468981 bytes
[21:06:49] mux: track 1, 7160762 bytes, 227.80 kbps
[21:06:49] mux: overhead, 9.39 bytes per frame
[21:06:49] thread 252e4000 exited ("muxer")
[21:06:49] thread 252e4000 joined ("muxer")
[21:06:49] thread 25067000 exited ("AC3 decoder")
[21:06:49] thread 1492f000 exited ("MPEG-2 decoder (libmpeg2)")
[21:06:49] thread 21d3a000 exited ("Renderer")
[21:06:49] thread 25261000 exited ("AAC encoder (Apple)")
[21:06:49] thread 1492f000 joined ("MPEG-2 decoder (libmpeg2)")
[21:06:49] mpeg2 done: 7542 frames
[21:06:49] thread 21d3a000 joined ("Renderer")
[21:06:49] render: lost time: 3723720 (1240 frames)
[21:06:49] render: gained time: 3722970 (4927 frames) (750 not accounted for)
[21:06:49] render: average dropped frame duration: 3003
[21:06:49] fifo_close: trashing 1 buffer(s)
[21:06:49] fifo_close: trashing 3 buffer(s)
[21:06:49] thread 24fe4000 exited ("H.264/AVC encoder (libx264)")
[21:06:49] thread 24fe4000 joined ("H.264/AVC encoder (libx264)")
x264 [info]: slice I:88    Avg QP:16.04  size: 19067  PSNR Mean Y:48.28 U:53.50 V:54.61 Avg:49.43 Global:48.96
x264 [info]: slice P:2956  Avg QP:17.62  size: 10898  PSNR Mean Y:46.15 U:52.18 V:53.65 Avg:47.41 Global:46.61
x264 [info]: slice B:3253  Avg QP:21.46  size:  5095  PSNR Mean Y:44.05 U:50.85 V:52.69 Avg:45.39 Global:44.69
x264 [info]: consecutive B-frames: 13.2% 39.0% 38.4%  5.0%  3.4%  0.4%  0.6%
x264 [info]: mb I  I16..4: 20.5% 68.6% 10.9%
x264 [info]: mb P  I16..4:  7.0% 27.6%  3.2%  P16..4: 25.6%  9.4%  3.9%  0.2%  0.1%    skip:23.0%
x264 [info]: mb B  I16..4:  1.0%  3.8%  0.4%  B16..8: 34.4%  2.8%  2.8%  direct: 8.8%  skip:46.0%  L0:42.4% L1:45.9% BI:11.7%
x264 [info]: 8x8 transform  intra:72.8%  inter:74.1%
x264 [info]: direct mvs  spatial:99.5%  temporal:0.5%
x264 [info]: ref P L0  69.9% 14.2%  7.0%  3.7%  2.8%  2.5%
x264 [info]: ref B L0  80.2% 10.4%  4.8%  2.5%  2.0%
x264 [info]: SSIM Mean Y:0.9905715
x264 [info]: PSNR Mean Y:45.095 U:51.511 V:53.167 Avg:46.391 Global:45.534 kb/s:1921.59
[21:06:49] thread 25067000 joined ("AC3 decoder")
[21:06:49] thread 25261000 joined ("AAC encoder (Apple)")
[21:06:49] thread 14781000 joined ("reader")
[21:06:49] fifo_close: trashing 0 buffer(s)
[21:06:49] fifo_close: trashing 1 buffer(s)
[21:06:49] fifo_close: trashing 0 buffer(s)
[21:06:49] fifo_close: trashing 0 buffer(s)
[21:06:49] fifo_close: trashing 0 buffer(s)
[21:06:49] fifo_close: trashing 0 buffer(s)
[21:06:49] fifo_close: trashing 0 buffer(s)
[21:06:49] fifo_close: trashing 0 buffer(s)
[21:06:49] fifo_close: trashing 0 buffer(s)
[21:06:49] decomb: yadif+eedi2 deinterlaced 6300 | blend deinterlaced 0 | unfiltered 0 | total 6300
[21:06:49] thread 21381000 exited ("yadif_filter_segment")
[21:06:49] thread 21381000 joined ("yadif_filter_segment")
[21:06:49] thread 21404000 exited ("yadif_filter_segment")
[21:06:49] thread 21404000 joined ("yadif_filter_segment")
[21:06:49] thread 21487000 exited ("yadif_filter_segment")
[21:06:49] thread 21487000 joined ("yadif_filter_segment")
[21:06:49] thread 2150a000 exited ("yadif_filter_segment")
[21:06:49] thread 2150a000 joined ("yadif_filter_segment")
[21:06:49] thread 2158d000 exited ("yadif_filter_segment")
[21:06:49] thread 2158d000 joined ("yadif_filter_segment")
[21:06:49] thread 21610000 exited ("yadif_filter_segment")
[21:06:49] thread 21610000 joined ("yadif_filter_segment")
[21:06:49] thread 21693000 exited ("yadif_filter_segment")
[21:06:49] thread 21693000 joined ("yadif_filter_segment")
[21:06:49] thread 21716000 exited ("yadif_filter_segment")
[21:06:49] thread 21716000 joined ("yadif_filter_segment")
[21:06:49] thread 21799000 exited ("decomb_filter_segment")
[21:06:49] thread 21799000 joined ("decomb_filter_segment")
[21:06:49] thread 2181c000 exited ("decomb_filter_segment")
[21:06:49] thread 2181c000 joined ("decomb_filter_segment")
[21:06:49] thread 2189f000 exited ("decomb_filter_segment")
[21:06:49] thread 2189f000 joined ("decomb_filter_segment")
[21:06:49] thread 21922000 exited ("decomb_filter_segment")
[21:06:49] thread 21922000 joined ("decomb_filter_segment")
[21:06:49] thread 219a5000 exited ("decomb_filter_segment")
[21:06:49] thread 219a5000 joined ("decomb_filter_segment")
[21:06:49] thread 21a28000 exited ("decomb_filter_segment")
[21:06:49] thread 21a28000 joined ("decomb_filter_segment")
[21:06:49] thread 21aab000 exited ("decomb_filter_segment")
[21:06:49] thread 21aab000 joined ("decomb_filter_segment")
[21:06:49] thread 21b2e000 exited ("decomb_filter_segment")
[21:06:49] thread 21b2e000 joined ("decomb_filter_segment")
[21:06:49] thread 21bb1000 exited ("eedi2_filter_segment")
[21:06:49] thread 21bb1000 joined ("eedi2_filter_segment")
[21:06:49] thread 21c34000 exited ("eedi2_filter_segment")
[21:06:49] thread 21c34000 joined ("eedi2_filter_segment")
[21:06:49] thread 21cb7000 exited ("eedi2_filter_segment")
[21:06:49] thread 21cb7000 joined ("eedi2_filter_segment")
[21:06:49] Freed 3 buffers of size 1024
[21:06:49] Freed 32 buffers of size 2048
[21:06:49] Freed 32 buffers of size 16384
[21:06:49] Freed 32 buffers of size 524288
[21:06:49] Allocated 17632256 bytes of buffers on this pass and Freed 17370112 bytes, 262144 bytes leaked
[21:06:49] thread 144d5000 exited ("work")
[21:06:49] thread 144d5000 joined ("work")
[21:06:49] libhb: work result = 0
jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Re: Combing Detection

Post by jbrjake »

Elan wrote:After mcdeint do you plan to add other new mode like a NNEDI port? Some say it's superior in quality to the "older" EEDI2.
Uh...how?

To port a filter I need source code...
Elan
Posts: 35
Joined: Wed Jan 30, 2008 6:32 am

Re: Combing Detection

Post by Elan »

Good point, no link to the source file in the entire thread. Only unuseful tiny code sections. :(

Maybe I can ask for a source file if it's on an open licence but I guess no...
jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Re: Combing Detection

Post by jbrjake »

No, It's closed source. I never would have bothered porting EEDI2 if I could have ported NNEDI. :(

At some point in the future I'll try to fold TDeint/TIVTC in, though, to replace yadif and pullup. But EEDI2 is as good as spatial interpolating for deinterlacing is going to get for awhile, in the GPL world. I think. Would be happy to be told otherwise.
jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Re: Combing Detection

Post by jbrjake »

- Fixed the green lines at the bottom of frames filtered solely with cubic interpolation.

- Added a new mode, 32, for debugging. It displays the full-resolution bitmap the combing detector generates. It paints in white every pixel the filter thinks is combed.

- I got mcdeint working again, it seems, partially thanks to elmopitt's patch ( http://forum.handbrake.fr/viewtopic.php?f=4&t=8090 ). I've also finally got a handle on what mcdeint does, I think. It gives me a number of ideas on how to speed it up....for example, MCDEINT_CHECK is rerunning the same edge direction detection that yadif/eedi2 has already run. If I recorded the offset those filters already found, mcdeint should be able to do its temporal diffs without an iterative search. I'm also not completely sold on the utility of using snow as the codec, which seems to come down to Michael Niedermayer's understandable preferenced for his overlapping block-based motion estimation. I can't help but wonder if, as superdump suggested to him once, using x264 with UMH might work almost as well with a lot more speed.

Patch:
http://handbrake.fr/pastebin/pastebin.php?show=506

Modes:
1 = Use yadif
2 = Use blend
4 = Use cubic interpolation
8 = Use EEDI2 interpolation
16 = Use mcdeint
32 = Display combing mask

Working combos:
1: Just yadif
2: Just blend
3: Switch between yadif and blend
4: Just cubic interpolate
5: Cubic->yadif
6: Switch between cubic and blend
7: Switch between cubic->yadif and blend
8: Just EEDI2 interpolate
9: EEDI2->yadif
10: Switch between EEDI2 and blend
11: Switch between EEDI2->yadif and blend
17: Yadif->mcdeint
18: Blend->mcdeint
19: Switch between blending and yadif -> mcdeint
20: Cubic->mdeint
21: Cubic->yadif->mcdeint
22: Cubic or blend -> mcdeint
23: Cubic->yadif or blend -> mcdeint
24: EEDI2->mcdeint
25: EEDI2->yadif->mcdeint
...okay I'm getting bored now listing all these different modes
32: Passes through the combing mask for every combed frame (white for combed pixels, otherwise black)
33+: Overlay the combing mask for every combed frame on top of the filtered output (white for combed pixels)

12-15: EEDI2 will override cubic interpolation
16: DOES NOT WORK BY ITSELF-- mcdeint needs to be fed by another deinterlacer

I an pleased to say that EEDI2->yadif->mcdeint produces the finest output I've yet seen from decomb. As well as the slowest ;>

Here's a visual comparison:

Mode 1: Yadif
Image
Mode 2: Blend
Image
Mode 4: Cubic
Image
Mode 5: Cubic->yadif
Image
Mode 8: EEDI2
Image
Mode 9: EEDI2->yadif
Image
Mode 21: Cubic->yadif->mcdeint
Image
Mode 25: EEDI2->yadif->mcdeint
Image
Mode 32: Combing mask
Image
Mode 57: Combing mask overlaid on EEDI2->yadif->mcdeint
Image
eddyg
Veteran User
Posts: 798
Joined: Mon Apr 23, 2007 3:34 am

Re: Combing Detection

Post by eddyg »

25 looks amazing, very cool.
jzietman
Enlightened
Posts: 146
Joined: Mon Feb 04, 2008 3:29 pm

Re: Combing Detection

Post by jzietman »

Agreed; it's almost perfect!
jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Re: Combing Detection

Post by jbrjake »

Here's an updated patch. I finally got rid of those nagging compiler warnings for decomb.c, and informs mcdeint of eedi2's edge direction decisions when both are enabled (mode 25):

http://handbrake.fr/pastebin/pastebin.php?show=509

There's one weird part about this:

Both of these edge direction detectors work off pixel offsets between adjoining lines. But mcdeint only checks for horizontal offsets of +- 2 pixels. EEDI2, with its default maxd of 24, searches for offsets of +- 12 pixels. If I feed mcdeint these EEDI2 offsets directly, they look terrible, far worse than the internal direction estimate. But if I cap them to +- 2, in my limited testing, they *seem* to look *better* than the internal direction estimate.

This...does not make sense to me. I'd expect that when looking for a most common 3x1 pixel block in the lines above and below, there would be cases where the local minimum for a range of 4 might be entirely different -- maybe even a different sign -- than the minimum for a range of 24...Like, if the edge direction across 12 pixels was -12, it seems hasty to me to say -2 would be the best if constraining to a 4 pixel range. Maybe with that limited data set, a +1 offset would be a better match. Yet, that seems to look better, so in this patch, that's what's happening.

Oh yeah, it does speed mcdeint up a bit, but unfortunately, the timesink there really is waiting for avcodec_encode_video() to return from snow. And we aren't even using the iterative ME that makes snow useful for this...enabling it causes crashiness on my machine. I tried plugging regular MPEG-4 in as a codec instead, which did boost the speed, but the quality suffered, even at a quality of 1....trying to use libx264 through libav* in mcdeint just gave me segfaults, although I bet someone else more familiar with using x264 through ffmpeg could get it to work.

I think I'll check this stuff in soon, and start looking at ways to postprocess the combing mask to reduce noise, so we can raise the sensitivity and overlap the combing block check without too many false positives.
Starhawk
Experienced
Posts: 90
Joined: Sun Feb 24, 2008 8:27 pm

Re: Combing Detection

Post by Starhawk »

Thanks for this patch, jbrjake.

I finally installed it a couple of months ago, but hadnt used it on any fully interlaced material until recently. I was excited to try out EEDI2's improvements, and I knew it was going to be a little slower, but damn is it slow! :) Im getting about 2-3 fps on a 2.33 Intel Core 2 Duo using Mode 9. On quick inspection, I can't tell too much of a difference between Default (which I think it Mode 7?) and Mode 9, but Im using it on some lower quality material so it might not have been the best source to test. Because most of the stuff I was de-interlacing was lower quality TV shows and such, I figured Default was good enough for that, especially for the upside of getting ~18fps.

I havent tried any animation yet, but Im going to give it a shot. It sounds like I'd be able to see EEDI2's improvements easier with that material.

Also tried mcdeint with Mode 25 just to see and was getting slightly above 1fps :o
jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Re: Combing Detection

Post by jbrjake »

Heh...actually, Starhawk, that copy of the patch is pretty outdated. I've been tinkering with it, on and off, ever since March. I found a number of problems with different things I was doing, like the way I was protecting the yadif and mcdeint check macros from striding outside their memory.

Here's a current version:
http://handbrake.fr/pastebin/pastebin.php?show=719

If it was going in as an SVN commit already, the message would be:

- Introduces a new mode structure for decomb, to make more flexible combinations of options possible. A full explanation of the new modes can be found at the top of libhb/decomb.c. The new default parameter string is: 7:2:6:9:80:16:16:10:20:20:4:2:50:24:1
- Cleans up some compiler warnings for decomb
- Partially fixes mcdeint in decomb. I would not suggest using it yet, but it's getting piped the right data, it no longer outstrides its memory, and lavc finally fixed their Snow bug that broke iterative ME). Seems to work well on fully interlaced material, with the spatial metric (2nd paramter) set to -1. Output with hybrid material and when actually decombing is...poor, but these are, hopefully, surmountable problems.

(There's also some commented-out code intended for bobbing, which is coming whenever this gets checked in.)
Starhawk wrote:I was excited to try out EEDI2's improvements, and I knew it was going to be a little slower, but damn is it slow! :) Im getting about 2-3 fps on a 2.33 Intel Core 2 Duo using Mode 9. On quick inspection, I can't tell too much of a difference between Default (which I think it Mode 7?) and Mode 9, but Im using it on some lower quality material so it might not have been the best source to test.
<snip>
I havent tried any animation yet, but Im going to give it a shot. It sounds like I'd be able to see EEDI2's improvements easier with that material.
Yeah, EEDI2 doesn't necessarily help with low quality stuff. Rhester, actually, feels that it can produce worse output than yadif on live action material. But the benefits with animation are unquestionable, imo.
Also tried mcdeint with Mode 25 just to see and was getting slightly above 1fps :o
Well, it's even slower with this new version of the patch. I spent a lot more time looking at mcdeint. Since we bumped ffmpeg a couple months ago, Snow's iterative motion estimation works again (was fixed this past spring). I re-enabled it, which brings mcdeint down to well under 1fps on my 2.4ghz unibody MBP.

This version of the patch also removes my dumb attempts to inform mcdeint of eedi2's edge detection decisions, since I never gained any confidence that it was improving quality overall.
jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Decomb v3

Post by jbrjake »

Some decomb changes...

Gamma scaling

Dark_Shikari mentioned offhandedly the other month that:
Jason wrote:[17:20] [Dark_Shikari] make all filters gamma-aware
[17:20] [Dark_Shikari] most filters completely ignore gamma
[17:20] [Dark_Shikari] which can give surprisingly bad results sometimes
<snip>
[17:22] [Dark_Shikari] "gamma-aware" means that the filter understands that the difference between "30 and 31" luma is not the same as between "250 and 251"
http://handbrake.fr/irclogs/handbrake/h ... 5_pg3.html

...which was a real "oh, duh" moment for me.

Decomb *does* assume the difference between 250 and 251 is the same as the difference between 30 and 31. But the eye is more sensitive to some brightness than to others. The actual intensity of light as a percentage isn't 30/255 (11.76%), it's (30/255)^2.2 (0.9%). So the difference between 30 and 31 is:

Code: Select all

(30/255) ^ 2.2 -  (31/255) ^ 2.2
==
0.0090214919 - 0.0096963287
==
-0.0006748368
Which is to say, there is hardly any perceptible difference.

Whereas 250-251 is:

Code: Select all

(250/255) ^ 2.2 - (251-255) ^ 2.2
==
0.957369576 -  0.965814654
==
-0.008445078
So a difference of 1 out of 255, which naively would always be a difference of 0.39%, can actually range from 0.07% to 0.84%

I modified decomb to scale values to the gamma when a new mode is applied, 128 (64 is reserved for bobbing). I quickly found that this lowered decomb's sensitivity. Because it is now looking for only changes in high-intensity parts of the image, it does not get as many hits. An extra from the Serenity DVD demonstrates this nicely.

To make what's going on a little clearer, I've added a new function that gets called in maskmode (32), which draws a box on the output. This box surrounds the area of the frame that made decomb decide the frame was combed. The box has dimensions equivalent to the block_width and block_height parameters of the filter, and the placement corresponds to the first block of pixels in the image where the number of combed pixels exceeds the block_threshold (except when a frame is blended, then it's the location of the *last* block of pixels in the image that exceeds *half* the block_threshold). If no pixels are combed, the box rests in the upper left corner of the frame.

Here is the raw frame:
Image

Here are, in white, the pixels decomb currently thinks are combed (filtered with decomb options 295:2:6:9:1):
Image
Here are, in white, the pixels decomb thinks are combed after scaling luminosity (filtered with decomb options 423:2:6:9:1):
Image
As you see, the scaled one ignores combing in shadowed parts of the frame that are harder to see, while still catching the parts that are bright and easy to see.

Now, because it is being more discerning, I was able to turn up the sensitivity. I'm now using spatial and motion thresholds of 3 instead of 9 and 6 respectively, and a block threshold of 40 instead of 80.

With the gamma scaling and increased sensitivity, the output looks like this (filtered with decomb options 423:2:3:3:1:40):
Image

Note that it looks like the unscaled output at current thresholds, but with less noise. However, lower motion and spatial thresholds do mean there is a chance for noise slipping through, potentially tripping the lower block threshold.

Filtering

So the second change I've made is to add filtering of the combing mask. When decomb analyzes a frame, it builds a black and white bitmap image the size of the picture frame, with white pixels to denote each combed pixel's location, and black to indicate uncombed.

If you've ever worked with Photoshop's selection tools, you probably know about a little trick to improve their efficacy. Say you want to select every pixel of a particular color. However, once you do so, you end up with a bunch of selections separated by small gaps where the colors are slightly different, as well as some small oddball selections you didn't want just because one pixel is the right color. The solution is to contract the selection by several pixels, then expand the selection by several pixels more, then contract it again. Contracting removes small selection areas, expanding recovers them if they are near bigger selection areas, and contracting again returns to the original selection shape...which is hopefully now more uniform.

I stole this erosion+dilation technique from EEDI2, which uses it to clean up its edge mask.

Decomb was, previously, doing something a little like this. It would filter out any combed pixels that did not have combed pixels to the immediate left or right of themselves. This was actually a bug -- I'd meant to filter out combed pixels that didn't have other combed pixels immediately above or below themselves -- but worked well enough. Now, in the initial erosion phase, it's filtering out any that don't have more than 2 pixels combed on any sides, which is a little more flexible.

Applying the new filter to the gamma-scaled combing mask produces this (filtered with decomb options 423:2:3:3:2:40):
Image

Putting it all together

Decomb is now more sensitive to combing that people can easily see and less sensitive to combing that people can't easily see. This allows much lower thresholds for what constitutes combing, making it less likely to give false negatives on interlaced frames. Better filtering of the results means the filter should be more resilient to noise and less likely to give false positives on progressive frames.

In order to make these comparisons more clear, I've added a mode 512 to decomb to aid in debugging. It's to be used with mask mode (32) to composite the combing mask onto the output frame. The mask shows every pixel it thinks is combed in white, and every pixel it thinks is not combed in black. Compositing shows the combed pixels in white and the uncombed pixels as they would be output normally.

Fewer false negatives:

Example of an interlaced frame current decomb lets through and new decomb catches:
Image

Here is what old decomb would do (--decomb=807:2:6:9:1):
Image
The filter sees some pixels are combed, but doesn't see it as sufficiently combed to deinterlace it.

Here is what new decomb will do (--decomb=935:2:3:3:2:40):
Image
It latches on to the combing on the side of Joss Whedon's nose and deinterlaces the frame.

On the hybrid clip that frame is from...
Old decomb stats are: deinterlaced 460 | blended 107 | unfiltered 752 | total 1319
New decomb stats are: deinterlaced 617 | blended 79 | unfiltered 623 | total 1319

Fewer false positives

One kind of progressive frame that can trip up decomb is vegetation. Put a bunch of tree branches in front of a camera, and all the criss-crossed lines can look like interlacing artifacts to the filter. For example, at the beginning of the TV series Lost, Jack Shephard opens his eyes on the jungle floor and looks up at a bamboo forest, swaying in the breeze.

Image

The mask old decomb produced (--decomb=295:2:6:9:1):
Image

New combing mask of the frame (--decomb=423:2:3:3:2:40):
Image

Patch:
http://handbrake.fr/pastebin/pastebin.php?show=1414

Code: Select all

Index: libhb/decomb.c
===================================================================
--- libhb/decomb.c	(revision 3295)
+++ libhb/decomb.c	(working copy)
@@ -11,8 +11,8 @@
 
 /*****
 Parameters:
-    Mode : Spatial metric : Motion thresh : Spatial thresh : Block thresh :
-    Block width : Block height
+    Mode : Spatial metric : Motion thresh : Spatial thresh : Mask Filter Mode :
+    Block thresh : Block width : Block height
 
 Appended for EEDI2:
     Magnitude thresh : Variance thresh : Laplacian thresh : Dilation thresh :
@@ -22,7 +22,7 @@
     Parity
     
 Defaults:
-    7:2:6:9:80:16:16:10:20:20:4:2:50:24:1:-1
+    7:2:6:9:1:80:16:16:10:20:20:4:2:50:24:1:-1
 *****/
 
 #define MODE_YADIF       1 // Use yadif
@@ -31,7 +31,13 @@
 #define MODE_EEDI2       8 // Use EEDI2 interpolation
 #define MODE_MCDEINT    16 // Post-process with mcdeint
 #define MODE_MASK       32 // Output combing masks instead of pictures
+#define MODE_GAMMA      128 // Scale gamma when decombing
+#define MODE_FILTER     256 // Filter combing mask
+#define MODE_COMPOSITE  512 // Overlay combing mask onto picture
 
+#define FILTER_CLASSIC 1 
+#define FILTER_ERODE_DILATE 2
+
 /***** 
 These modes can be layered. For example, Yadif (1) + EEDI2 (8) = 9,
 which will feed EEDI2 interpolations to yadif.
@@ -136,6 +142,7 @@
 
     // Decomb parameters
     int              mode;
+    int              filter_mode;
     int              spatial_metric;
     int              motion_threshold;
     int              spatial_threshold;
@@ -143,6 +150,8 @@
     int              block_width;
     int              block_height;
     
+    float            gamma_lut[256];
+    
     // EEDI2 parameters
     int              magnitude_threshold;
     int              variance_threshold;
@@ -174,9 +183,15 @@
     uint8_t        * ref[4][3];
     int              ref_stride[3];
 
-    /* Make a buffer to store a comb mask. */
+    /* Make buffers to store a comb masks. */
     uint8_t        * mask[3];
+    uint8_t        * mask_filtered[3];
+    uint8_t        * mask_temp[3];
+    int              mask_box_x;
+    int              mask_box_y;
+    uint8_t          mask_box_color;
 
+
     uint8_t        * eedi_half[4][3];
     uint8_t        * eedi_full[5][3];
     int            * cx2;
@@ -311,6 +326,31 @@
     }
 }
 
+void draw_mask_box( hb_filter_private_t * pv )
+{
+    int x = pv->mask_box_x;
+    int y = pv->mask_box_y;
+    int box_width = pv->block_width;
+    int box_height = pv->block_height;
+    int stride = pv->ref_stride[0];
+    uint8_t * mskp = ( pv->mode & MODE_FILTER ) ? pv->mask_filtered[0] : pv->mask[0];
+
+    
+    int block_x, block_y;
+    for( block_x = 0; block_x < box_width; block_x++)
+    {
+        mskp[y*stride+x+block_x] = 128;
+        mskp[(y+box_height)*stride+x+block_x] = 128;
+    }
+    
+    for( block_y = 0; block_y < box_height; block_y++)
+    {
+        mskp[stride*(y+block_y)+x] = 128;
+        mskp[stride*(y+block_y) + x + box_width] = 128;
+    }
+    
+}
+
 void apply_mask_line( uint8_t * srcp,
                       uint8_t * mskp,
                       int width )
@@ -323,25 +363,34 @@
         {
             srcp[x] = 255;
         }
+        if( mskp[x] == 128 )
+        {
+            srcp[x] = 128;
+        }
     }
 }
 
 void apply_mask( hb_filter_private_t * pv )
 {
+    
+    /* draw_boxes */
+    draw_mask_box( pv );
+    
     int plane, height;
     
     for( plane = 0; plane < 3; plane++ )
     {
         uint8_t * srcp = ( pv->mode & MODE_MCDEINT ) ? pv->pic_in.data[plane] : pv->pic_out.data[plane];
-        uint8_t * mskp = pv->mask[plane];
+        uint8_t * mskp = ( pv->mode & MODE_FILTER ) ? pv->mask_filtered[plane] : pv->mask[plane];
         
         for( height = 0; height < pv->height[plane]; height++ )
         {
-            if( pv->mode == MODE_MASK && plane == 0 )
+            
+            if( !(pv->mode & MODE_COMPOSITE) && plane == 0 )
             {
                 memcpy( srcp, mskp, pv->width[plane] );
             }
-            else if( pv->mode == MODE_MASK )
+            else if( !(pv->mode & MODE_COMPOSITE) )
             {
                 memset( srcp, 128, pv->width[plane] );
             }
@@ -349,11 +398,32 @@
             {
                 apply_mask_line( srcp, mskp, pv->width[plane] );
             }
+            
+//            if( ( pv->mode == MODE_MASK ||
+//                    ( pv->mode == ( MODE_MASK + MODE_GAMMA + MODE_FILTER) ) || 
+//                    ( pv->mode == ( MODE_MASK + MODE_FILTER ) ) || 
+//                    ( pv->mode == ( MODE_MASK + MODE_GAMMA ) ) )
+//                  && plane == 0 )
+//            {
+//                memcpy( srcp, mskp, pv->width[plane] );
+//            }
+//            else if( pv->mode == MODE_MASK || 
+//                   ( pv->mode == ( MODE_MASK + MODE_GAMMA + MODE_FILTER ) ) || 
+//                   ( pv->mode == ( MODE_MASK + MODE_FILTER ) ) ||
+//                   ( pv->mode == ( MODE_MASK + MODE_GAMMA ) ) )
+//            {
+//                memset( srcp, 128, pv->width[plane] );
+//            }
+//            else if( plane == 0 )
+//            {
+//                apply_mask_line( srcp, mskp, pv->width[plane] );
+//            }
 
             srcp += pv->pic_out.linesize[plane];
             mskp += pv->ref_stride[plane];
         }
     }
+    
 }
 
 static void store_ref( const uint8_t ** pic,
@@ -483,6 +553,307 @@
     }
 }
 
+void erode_combing_mask( hb_filter_private_t * pv )
+{
+    //Take in mask, output to mask_temp
+    int x, y, k;
+    
+    uint8_t * cur;
+    uint8_t * dst;
+    
+    int count;
+    
+    int erosion_threshold = 2;
+    
+    int ref;
+    for( k = 0; k < 1; k++ )
+    {
+        ref = pv->ref_stride[k];
+        
+        for( y = 1; y < pv->height[k] -1; y++ )
+        {
+            cur = &pv->mask_temp[k][y*ref];
+            dst = &pv->mask_filtered[k][y*ref];
+            
+            for( x = 1; x < pv->width[k]-1; x++ )
+            {
+                if( cur[0] == 0 )
+                {
+                    dst[0] = 0;
+                    cur++;
+                    dst++;
+                    continue;
+                }
+                    
+                count = 0;
+                if( cur[-ref-1] == 255 )
+                    ++count;
+                if( cur[-ref] == 255 )
+                    ++count;
+                if( cur[-ref+1] == 255 )
+                    ++count;
+                if( cur[-1] == 255 )
+                    ++count;
+                if( cur[+1] == 255 )
+                    ++count;
+                if( cur[ref-1] == 255 )
+                    ++count;
+                if( cur[ref] == 255 )
+                    ++count;
+                if( cur[ref+1] == 255 )
+                    ++count;
+
+                if( count < erosion_threshold )
+                    dst[0] = 0;
+                else
+                    dst[0] = 255;
+                    
+                cur++;
+                dst++;
+            }
+        }
+    }
+}
+
+void dilate_combing_mask( hb_filter_private_t * pv )
+{
+    //Take in mask_temp, output to mask
+    int x, y, k;
+    
+    uint8_t * curp;
+    uint8_t * cur;
+    uint8_t * curn;
+    uint8_t * dst;
+    
+    int count;
+    
+    int dilation_threshold = 4;
+    
+    int ref;
+    for( k = 0; k < 1; k++ )
+    {
+        ref = pv->ref_stride[k];
+        
+        for( y = 1; y < pv->height[k] -1; y++ )
+        {
+            curp = &pv->mask_filtered[k][(y-1)*ref];
+            cur  = &pv->mask_filtered[k][y*ref];
+            curn = &pv->mask_filtered[k][(y+1)*ref];
+            dst = &pv->mask_temp[k][y*ref];
+            
+            for( x = 1; x < pv->width[k]-1; x++ )
+            {
+                if( cur[0] == 255 )
+                {
+                    dst[0] = 255;
+                    curp++;
+                    cur++;
+                    curn++;
+                    dst++;
+                    continue;
+                }
+                    
+                count = 0;
+                if( curp[-1] == 255 )
+                    ++count;
+                if( curp[0] == 255 )
+                    ++count;
+                if( curp[+1] == 255 )
+                    ++count;
+                if( cur[-1] == 255 )
+                    ++count;
+                if( cur[+1] == 255 )
+                    ++count;
+                if( curn[-1] == 255 )
+                    ++count;
+                if( curn[0] == 255 )
+                    ++count;
+                if( curn[+1] == 255 )
+                    ++count;
+
+                if( count >= dilation_threshold )
+                    dst[0] = 255;
+                else
+                    dst[0] = 0;
+                    
+                curp++;
+                cur++;
+                curn++;
+                dst++;
+            }
+        }
+    }
+}
+
+
+void filter_combing_mask( hb_filter_private_t * pv )
+{
+    int x, y, k;
+    
+    uint8_t * curp;
+    uint8_t * cur;
+    uint8_t * curn;
+    uint8_t * dst;
+    
+    int h_count, v_count;
+    
+    int ref;
+    for( k = 0; k < 1; k++ )
+    {
+        ref = pv->ref_stride[k];
+        
+        for( y = 0; y < pv->height[k] -1; y++ )
+        {
+            curp = &pv->mask[k][(y-1)*ref];
+            cur = &pv->mask[k][y*ref];
+            curn = &pv->mask[k][(y+1)*ref];
+            dst = (pv->filter_mode == FILTER_CLASSIC ) ? &pv->mask_filtered[k][y*ref] : &pv->mask_temp[k][y*ref] ;
+            
+            for( x = 0; x < pv->width[k]-1; x++ )
+            {
+                
+                h_count = v_count = 0;
+                if( x == 0 )
+                {
+                    if( cur[0] == 255 && cur[1] == 255 )
+                        h_count++;
+                }
+                else if( x == pv->width[k]-1 )
+                {
+                    if( cur[-1] == 255 && cur[0] == 255 )
+                        h_count++;
+                }
+                else
+                {
+                    if(cur[-1] == 255 && cur[0] == 255 && cur[1] == 255 )
+                        h_count++;
+                }
+                
+                if( y == 0 )
+                {
+                    if( cur[0] == 255 && curn[0] == 255 )
+                        v_count++;
+                }
+                else if( y == pv->height[k]-1 )
+                {
+                    if( curp[0] == 255 && cur[0] == 255 )
+                        v_count++;
+                }
+                else
+                {
+                    if( curp[0] == 255 && cur[0] == 255 && curn[0] == 255 )
+                        v_count++;
+                }
+                
+                if( h_count & pv->filter_mode == FILTER_CLASSIC )
+                    dst[0] = 255;
+                else if( pv->filter_mode == FILTER_CLASSIC )
+                    dst[0] = 0;
+                else if( h_count && v_count )
+                    dst[0] = 255;
+                else
+                    dst[0] = 0;
+                    
+                curp++;
+                cur++;
+                curn++;
+                dst++;
+            }
+        }
+    }
+}
+
+int check_filtered_combing_mask( hb_filter_private_t * pv )
+{
+    /* Go through the mask in X*Y blocks. If any of these windows
+       have threshold or more combed pixels, consider the whole
+       frame to be combed and send it on to be deinterlaced.     */
+
+    /* Block mask threshold -- The number of pixels
+       in a block_width * block_height window of
+       he mask that need to show combing for the
+       whole frame to be seen as such.            */
+    int threshold       = pv->block_threshold;
+    int block_width     = pv->block_width;
+    int block_height    = pv->block_height;
+    int block_x, block_y;
+    int block_score = 0; int send_to_blend = 0;
+    uint8_t * mask_p;
+    int x, y, k;
+
+    for( k = 0; k < 1; k++ )
+    {
+        int ref_stride = pv->ref_stride[k];
+        pv->mask_box_x = -1;
+        pv->mask_box_y = -1;
+        pv->mask_box_color = 0;
+        
+        for( y = 0; y < ( pv->height[k] - block_height ); y = y + block_height )
+        {
+            for( x = 0; x < ( pv->width[k] - block_width ); x = x + block_width )
+            {
+                block_score = 0;
+                
+                for( block_y = 0; block_y < block_height; block_y++ )
+                {
+                    int mask_y = y + block_y;
+                    mask_p = &pv->mask_filtered[k][mask_y*ref_stride + x];
+                    
+                    for( block_x = 0; block_x < block_width; block_x++ )
+                    {
+
+                        if( mask_p[ 0 ] == 255 )
+                            block_score++;
+                        mask_p++;
+                    }
+                }
+
+                if( block_score >= ( threshold / 2 ) )
+                {
+#if 0
+                    hb_log("decomb: frame %i | score %i | type %s", pv->deinterlaced_frames + pv->blended_frames +  pv->unfiltered_frames + 1, block_score, pv->buf_settings->flags & 16 ? "Film" : "Video");
+#endif
+                    pv->mask_box_x = x;
+                    pv->mask_box_y = y;
+
+                   if ( block_score <= threshold && !( pv->buf_settings->flags & 16) )
+                    {
+                        /* Blend video content that scores between
+                           ( threshold / 2 ) and threshold.        */
+                        send_to_blend = 1;
+                        pv->mask_box_color = 2;
+                    }
+                    else if( block_score > threshold )
+                    {
+                        if( pv->buf_settings->flags & 16 )
+                        {
+                            /* Blend progressive content above the threshold.*/
+                            pv->mask_box_color = 2;
+                            return 2;
+                        }
+                        else
+                        {
+                            /* Yadif deinterlace video content above the threshold. */
+                            pv->mask_box_color = 1;
+                            return 1;
+                        }
+                    }
+                }
+            }
+        } 
+    }
+    
+    if( send_to_blend )
+    {
+        return 2;
+    }
+    else
+    {
+        /* Consider this frame to be uncombed. */
+        return 0;
+    }
+}
+
 int check_combing_mask( hb_filter_private_t * pv )
 {
     /* Go through the mask in X*Y blocks. If any of these windows
@@ -539,6 +910,27 @@
                                 mask_p[  1 ] == 255 )
                                     block_score++;
                         }
+
+//                           if( (y + block_y) == 0 )
+//                           {
+//                               if( mask_p[ 0 ] == 255 &&
+//                                   mask_p[ ref_stride ] == 255 )
+//                                       block_score++;
+//                           }
+//                           else if( (y + block_y) == (pv->height[k] -1) )
+//                           {
+//                               if( mask_p[ -ref_stride ] == 255 &&
+//                                   mask_p[  0 ] == 255 )
+//                                       block_score++;
+//                           }
+//                           else
+//                           {
+//                               if( mask_p[ -ref_stride ] == 255 &&
+//                                   mask_p[  0 ] == 255 &&
+//                                   mask_p[  +ref_stride ] == 255 )
+//                                       block_score++;
+//                           }
+//
                         mask_p++;
                     }
                 }
@@ -548,22 +940,29 @@
 #if 0
                     hb_log("decomb: frame %i | score %i | type %s", pv->deinterlaced_frames + pv->blended_frames +  pv->unfiltered_frames + 1, block_score, pv->buf_settings->flags & 16 ? "Film" : "Video");
 #endif
+
+                    pv->mask_box_x = x;
+                    pv->mask_box_y = y;
+
                     if ( block_score <= threshold && !( pv->buf_settings->flags & 16) )
                     {
                         /* Blend video content that scores between
                            ( threshold / 2 ) and threshold.        */
                         send_to_blend = 1;
+                        pv->mask_box_color = 2;
                     }
                     else if( block_score > threshold )
                     {
                         if( pv->buf_settings->flags & 16 )
                         {
                             /* Blend progressive content above the threshold.*/
+                            pv->mask_box_color = 2;
                             return 2;
                         }
                         else
                         {
                             /* Yadif deinterlace video content above the threshold. */
+                            pv->mask_box_color = 1;
                             return 1;
                         }
                     }
@@ -583,6 +982,144 @@
     }
 }
 
+void build_gamma_lut( hb_filter_private_t * pv )
+{
+    int i;
+    for( i = 0; i < 256; i++ )
+    {
+        pv->gamma_lut[i] = pow( ( (float)i / (float)255 ), 2.2f );
+    }
+}
+
+float scale_gamma( int pixel, hb_filter_private_t * pv )
+{
+    return pv->gamma_lut[pixel];
+}
+
+void detect_gamma_combed_segment( hb_filter_private_t * pv, int segment_start, int segment_stop )
+{
+    /* A mish-mash of various comb detection tricks
+       picked up from neuron2's Decomb plugin for
+       AviSynth and tritical's IsCombedT and
+       IsCombedTIVTC plugins.                       */
+       
+    int x, y, k, width, height;
+    
+    /* Comb scoring algorithm */
+    float spatial_metric  = (float)pv->spatial_metric / (float)255;
+    /* Motion threshold */
+    float mthresh         = (float)pv->motion_threshold / (float)255;
+    /* Spatial threshold */
+    float athresh         = (float)pv->spatial_threshold / (float)255;
+    float athresh_squared = athresh * athresh;
+    float athresh6        = 6 *athresh;
+
+    /* One pas for Y, one pass for U, one pass for V */    
+    for( k = 0; k < 1; k++ )
+    {
+        int ref_stride  = pv->ref_stride[k];
+        width           = pv->width[k];
+        height          = pv->height[k];
+        
+        /* Comb detection has to start at y = 2 and end at
+           y = height - 2, because it needs to examine
+           2 pixels above and 2 below the current pixel.      */
+        if( segment_start < 2 )
+            segment_start = 2;
+        if( segment_stop > height - 2 )
+            segment_stop = height - 2;
+            
+        for( y =  segment_start; y < segment_stop; y++ )
+        {
+            /* These are just to make the buffer locations easier to read. */
+            int up_2    = -2*ref_stride ;
+            int up_1    = -1*ref_stride;
+            int down_1 = ref_stride;
+            int down_2 = 2*ref_stride;
+            
+            /* We need to examine a column of 5 pixels
+               in the prev, cur, and next frames.      */
+            uint8_t * cur = &pv->ref[1][k][y*ref_stride];
+            uint8_t * prev = &pv->ref[0][k][y*ref_stride];
+            uint8_t * next = &pv->ref[2][k][y*ref_stride];
+            uint8_t * mask = &pv->mask[k][y*ref_stride];
+            
+            for( x = 0; x < width; x++ )
+            {
+                float up_diff   = pv->gamma_lut[cur[0]] - pv->gamma_lut[cur[up_1]];
+                float down_diff = pv->gamma_lut[cur[0]] - pv->gamma_lut[cur[down_1]];
+                
+                if( ( up_diff >  athresh && down_diff >  athresh ) ||
+                    ( up_diff < -athresh && down_diff < -athresh ) )
+                {
+                    /* The pixel above and below are different,
+                       and they change in the same "direction" too.*/
+                    int motion = 0;
+                    if( mthresh > 0 )
+                    {
+                        /* Make sure there's sufficient motion between frame t-1 to frame t+1. */
+                        if( fabs( pv->gamma_lut[prev[0]]      - pv->gamma_lut[cur[0]] ) > mthresh &&
+                            fabs( pv->gamma_lut[cur[up_1]]    - pv->gamma_lut[next[up_1]]    ) > mthresh &&
+                            fabs( pv->gamma_lut[cur[down_1]]  - pv->gamma_lut[next[down_1]]    ) > mthresh )
+                                motion++;
+                        if( fabs( pv->gamma_lut[next[0]]      - pv->gamma_lut[cur[0]] ) > mthresh &&
+                            fabs( pv->gamma_lut[prev[up_1]]   - pv->gamma_lut[cur[up_1]] ) > mthresh &&
+                            fabs( pv->gamma_lut[prev[down_1]] - pv->gamma_lut[cur[down_1]] ) > mthresh )
+                                motion++;
+                                
+//                            hb_log("prev->cur motion: %f, mthresh: %f", fabs( scale_gamma( prev[0] ) - scale_gamma( cur[0] ) ), mthresh);
+                    }
+                    else
+                    {
+                        /* User doesn't want to check for motion,
+                           so move on to the spatial check.       */
+                        motion = 1;
+                    }
+                           
+                    if( motion || ( pv->deinterlaced_frames==0 && pv->blended_frames==0 && pv->unfiltered_frames==0) )
+                    {
+
+                        /* Tritical's noise-resistant combing scorer.
+                           The check is done on a bob+blur convolution. */
+                        float combing = fabs( pv->gamma_lut[cur[up_2]]
+                                         + ( 4 * pv->gamma_lut[cur[0]] )
+                                         + pv->gamma_lut[cur[down_2]]
+                                         - ( 3 * ( pv->gamma_lut[cur[up_1]]
+                                                 + pv->gamma_lut[cur[down_1]] ) ) );
+                        /* If the frame is sufficiently combed,
+                           then mark it down on the mask as 255. */
+                           
+ //                              hb_log("combing: %f, athresh6: %f", combing, athresh6);
+                        if( combing > athresh6 )
+                        {
+                            mask[0] = 255;
+                        }
+                        else
+                        {
+                            mask[0] = 0;
+                        }
+
+                    }
+                    else
+                    {
+                        mask[0] = 0;
+                    }
+                }
+                else
+                {
+                    mask[0] = 0;
+                }
+                
+                cur++;
+                prev++;
+                next++;
+                mask++;
+            }
+        }
+    }
+}
+
+
 void detect_combed_segment( hb_filter_private_t * pv, int segment_start, int segment_stop )
 {
     /* A mish-mash of various comb detection tricks
@@ -950,7 +1487,14 @@
                 segment_stop = ( h / pv->cpu_count ) * ( segment + 1 );
             }
             
-            detect_combed_segment( pv, segment_start, segment_stop );
+            if( pv->mode & MODE_GAMMA )
+            {
+                detect_gamma_combed_segment( pv, segment_start, segment_stop );
+            }
+            else
+            {
+                detect_combed_segment( pv, segment_start, segment_stop );
+            }
         }
         /*
          * Finished this segment, let everyone know.
@@ -988,7 +1532,21 @@
         hb_unlock( pv->decomb_complete_lock[segment] );
     }
     
-    return check_combing_mask( pv );
+    if( pv->mode & MODE_FILTER )
+    {
+        filter_combing_mask( pv );
+        if( pv->filter_mode == FILTER_ERODE_DILATE )
+        {
+            erode_combing_mask( pv );
+            dilate_combing_mask( pv );
+            erode_combing_mask( pv );
+        }
+        return check_filtered_combing_mask( pv );
+    }
+    else
+    {
+        return check_combing_mask( pv );
+    }
 }
 
 static void yadif_filter_line( uint8_t *dst,
@@ -1458,7 +2016,7 @@
     
     if( pv->mode & MODE_MASK && pv->spatial_metric >= 0 )
     {
-        if( pv->mode == MODE_MASK || is_combed )
+        if( pv->mode == MODE_MASK || (pv->mode & MODE_MASK && pv->mode & MODE_GAMMA) || is_combed || (pv->mode & MODE_MASK && pv->mode & MODE_FILTER ))
         apply_mask( pv );
     }
 }
@@ -1606,6 +2164,8 @@
     pv->width[1]  = pv->width[2]  = width >> 1;
     pv->height[1] = pv->height[2] = height >> 1;
 
+    build_gamma_lut( pv );
+    
     pv->buf_out[0] = hb_video_buffer_init( width, height );
     pv->buf_out[1] = hb_video_buffer_init( width, height );
     pv->buf_settings = hb_buffer_init( 0 );
@@ -1617,6 +2177,7 @@
     pv->yadif_ready    = 0;
 
     pv->mode     = MODE_YADIF | MODE_BLEND | MODE_CUBIC;
+    pv->filter_mode = FILTER_ERODE_DILATE;
     pv->spatial_metric = 2;
     pv->motion_threshold = 6;
     pv->spatial_threshold = 9;
@@ -1640,11 +2201,12 @@
 
     if( settings )
     {
-        sscanf( settings, "%d:%d:%d:%d:%d:%d:%d:%d:%d:%d:%d:%d:%d:%d:%d:%d",
+        sscanf( settings, "%d:%d:%d:%d:%d:%d:%d:%d:%d:%d:%d:%d:%d:%d:%d:%d:%d",
                 &pv->mode,
                 &pv->spatial_metric,
                 &pv->motion_threshold,
                 &pv->spatial_threshold,
+                &pv->filter_mode,
                 &pv->block_threshold,
                 &pv->block_width,
                 &pv->block_height,
@@ -1683,7 +2245,7 @@
         }
     }
 
-    /* Allocate a buffer to store a comb mask. */
+    /* Allocate buffers to store comb masks. */
     for( i = 0; i < 3; i++ )
     {
         int is_chroma = !!i;
@@ -1691,6 +2253,8 @@
         int h = ((pv->height[0]+6+ 31) & (~31))>>is_chroma;
 
         pv->mask[i] = calloc( 1, w*h*sizeof(uint8_t) ) + 3*w;
+        pv->mask_filtered[i] = calloc( 1, w*h*sizeof(uint8_t) ) + 3*w;
+        pv->mask_temp[i] = calloc( 1, w*h*sizeof(uint8_t) ) + 3*w;
     }
     
     if( pv->mode & MODE_EEDI2 )
@@ -1950,15 +2514,28 @@
         }
     }
     
-    /* Cleanup combing mask. */
+    /* Cleanup combing masks. */
     for( i = 0; i<3*3; i++ )
     {
         uint8_t **p = &pv->mask[i/3];
+        uint8_t **p2 = &pv->mask_filtered[i/3];
+        uint8_t **p3 = &pv->mask_temp[i/3];
+
         if (*p)
         {
             free( *p - 3*pv->ref_stride[i/3] );
             *p = NULL;
         }
+        if (*p2)
+        {
+            free( *p2 - 3*pv->ref_stride[i/3] );
+            *p2 = NULL;
+        }
+        if (*p3)
+        {
+            free( *p3 - 3*pv->ref_stride[i/3] );
+            *p3 = NULL;
+        }
     }
     
     if( pv->mode & MODE_EEDI2 )
Deleted User 11865

Re: Combing Detection

Post by Deleted User 11865 »

Not read the whole post yet, but yay! :D
eddyg
Veteran User
Posts: 798
Joined: Mon Apr 23, 2007 3:34 am

Re: Combing Detection

Post by eddyg »

Impressive Jon. Congrats.
Post Reply