Combing Detection

Archive of historical development discussions
Discussions / Development has moved to GitHub
Forum rules
*******************************
Please be aware we are now using GitHub for issue tracking and feature requests.
- This section of the forum is now closed to new topics.

*******************************
jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Progress

Post by jbrjake »

PROGRESS REPORT

Well, I've made a lot of changes since I last posted in this thread.

I've moved the logic out of deinterlace.c, and added a new --decomb filter located in libhb/decomb.c. This is a highly modified version of deinterlace.c.

Comb detection can only get so good -- it is going to end up being called on progressive frames by mistake. This will happen especially on thin detail lines in animation. And when yadif jumps into action on stuff when it shouldn't, it's *obvious*.

INTERPOLATION

Deinterlacing is largely concerned with interpolation. You've got Line A, you've got line C. Line B is from another time and can't help you, you have to ignore it. You've got to interpolate between A and C to guess what B *should* be.

Now, if you've got 2 points, and you need to guess a point exactly in between, the answer seems obvious -- add them together, and divide by 2 to get the average. And so that's what yadif does. To decide on each pixel for line B, it does a bunch of checks and chooses, in the end, a pixel on Line A and a pixel on Line C it thinks are the closest. It averages them together, and that's the pixel for line B. Sometimes it's the ones directly above and below, sometimes they're off to the sides, sometimes they're from the previous and next frames...whatever. It's always the average of two pixels.

The downside of this is jaggies.

What makes yadif special is its intelligent spatio-temporal checks instead of always using the pixels directly above and below. However, there are plenty of other deinterlacers out there. And jaggies are problems that even the simple linear filters face. So it's worth looking at them.

FIR TREES

If you read through the MPlayer documentation you'll see stuff like:
fd/ffmpegdeint

FFmpeg deinterlacing filter that deinterlaces the given block by filtering every second line with a (−1 4 2 4 −1) filter.
Which is um...cryptic, no?

Well it turns out that kind of filter is called a Finite Impulse Response, or FIR filter. These used to be really popular in hardware filters, because they had limited memory and calculation time. There's a bunch of math behind it, involving words like "sinc" and "polyphasic" and "coefficient," which I don't really care enough about to figure out, because in the end it all works pretty simple for this kind of thing. They're weighted averages of pixels above and below.

That one above is the ffmpeg deinterlacer (what we call "Fast" and MPlayer/Transcode calls "pp=fd"). It's not, in fact, as I had long though, just a simple line-doubler. Let's say you're currently interpolating Line D. It would weigh things like so:

B * -1
C * 4
D * 2
E * 4
F * -1

Those are added up, and then divided by 8. These predictions are used to replace half the lines in each frame. Note how it actually gives some weight to the line it's replacing, but nowhere near as much as is given to the ones above and below. And it also includes the lines from the alternate field it should be ignoring, with negative weight. This all can, at times, create a smoother picture. Most of the time, it just looks worse, in my opinion.

BLENDING

There are, of course, many options besides spatial interpolation for deinterlacing. One simple option is blending. You can just blend the two fields together instead of dropping one. Then, you don't have to guess what those lines should be. It does away with the dancing pixels and stair-stepping and jaggies that are so distracting with linear interpolation.

The best blender I could find is lowpass5. This is a FIR filter that runs on *every* line instead of half of them, and looks like this, running on Line C.

A * -1
B * 2
C * 6
D * 2
E * -1

What's everyone's opinion of blending? I'm tempted to use it for progressive sequences, since that way when the comb detection gets a false positive, it's barely noticeable. I'm toying with the notion of using blend when job->interlaced is false ( when most of the 10 preview scans don't show combing). Thoughts?

CUBIC INTERPOLATION

And then there is bicubic interpolation. If you have 2 points, you can draw a straight line. But if you have 4 points, you can generate an equation that plots a curve over all of them and get a better result:

Code: Select all

int CubicInterpolate(
   int y0,int y1,
   int y2,int y3,
   double mu)
{
   double a0,a1,a2,a3,mu2;
   
   mu2 = mu*mu;
   a0 = y3 - y2 - y0 + y1;
   a1 = y0 - y1 - a0;
   a2 = y2 - y0;
   a3 = y1;

   double result = (a0*mu*mu2+a1*mu2+a2*mu+a3);
   if( result > 255)
   {
       result = 255;
   }
   if( result < 0 )
   {
       result = 0;
   }
   
   return (int)result;
}
That's the "right" way to do it but instead I just found the correct weights to use from a link (see Resources at the bottom):

Code: Select all

int EfficientCubicInterpolate( int y0, int y1, int y2, int y3 )
{
    int result = ( y0 * -3 ) + ( y1 * 23 ) + ( y2 * 23 ) + ( y3 * -3 );
    result /= 40;
    
    if( result > 255 )
    {
        result = 255;
    }
    else if( result < 0 )
    {
        result = 0;
    }
    
    return result;
}
Cubic filtering gets rid of a lot of the jaggies associated with simple linear interpolation. Wouldn't it be nice if we could pair that with yadif?

YADIF AND CUBIC INTERPOLATION

How do we find 4 points to feed that cubic interpolator? Well, we want to only look at pixels on the "good" half the lines we're keeping intact from the source. So in addition to the pixels directly above and below, we also skip up past the next "bad" line to the pixel 3 above and down to the pixel 3 below. Easy. But what about if yadif doesn't stick with its initial vertical prediction? What if it senses less difference in another direction and chooses pixels on a diagonal?

Well, we've got two pixels from yadif for sure. We can look at that as line. If we continue to draw these lines out, pixel choices for all of yadif's possible spatial decisions become apparent. We just look for the next pixels the lines hit on "good" fields.

Code: Select all

case -1:\
    spatial_pred = EfficientCubicInterpolate(cur[-3 * refs - 3], cur[-refs -1], cur[+refs + 1], cur[3* refs + 3] );\
break;\
case -2:\
    spatial_pred = EfficientCubicInterpolate(cur[-3 * refs -5], cur[-refs -2], cur[+refs + 2], cur[3* refs + 5] );\
break;\
case 1:\
    spatial_pred = EfficientCubicInterpolate(cur[-3 * refs +3], cur[-refs +1], cur[+refs - 1], cur[3* refs -3] );\
break;\
case 2:\
    spatial_pred = EfficientCubicInterpolate(cur[-3 * refs +5], cur[-refs +2], cur[+refs - 2], cur[3*refs - 5] );\
break;\
That sets yadif to use cubic interps for all its spatial predictions. It will still use an average for the temporal check (averaging the current pixel's intensity in the frame before and after).

This improves curves somewhat.

PATCH

So here's the current patch, with selective deinterlacing based on comb detection, using yadif tweaked to do cubic interpolation:

http://handbrake.pastebin.ca/1007122

Code: Select all

Index: test/test.c
===================================================================
--- test/test.c	(revision 1452)
+++ test/test.c	(working copy)
@@ -40,6 +40,8 @@
 static char * denoise_opt           = 0;
 static int    detelecine            = 0;
 static char * detelecine_opt        = 0;
+static int    decomb                = 0;
+static char * decomb_opt            = 0;
 static int    grayscale   = 0;
 static int    vcodec      = HB_VCODEC_FFMPEG;
 static int    h264_13     = 0;
@@ -826,7 +828,12 @@
                 hb_filter_denoise.settings = denoise_opt;
                 hb_list_add( job->filters, &hb_filter_denoise );
             }
-
+            if( decomb )
+            {
+                hb_filter_decomb.settings = decomb_opt;
+                hb_list_add( job->filters, &hb_filter_decomb );
+            }
+            
             if( width && height )
             {
                 job->width  = width;
@@ -1455,6 +1462,8 @@
      "          <weak/medium/strong>\n"
      "    -9, --detelecine        Detelecine video with pullup filter\n"
      "          <L:R:T:B:SB:MP>   (default 1:1:4:4:0:0)\n"
+     "    -5, --decomb           Selectively deinterlaces when it detects combing\n"
+     "          <YM:FD:EQ:DF:TR>  (default: 0:-1:10:30:9)\n"
     "    -g, --grayscale         Grayscale encoding\n"
     "    -p, --pixelratio        Store pixel aspect ratio in video stream\n"
     "    -P, --loosePixelratio   Store pixel aspect ratio with specified width\n"
@@ -1592,6 +1601,7 @@
             { "deblock",     optional_argument, NULL,    '7' },
             { "denoise",     optional_argument, NULL,    '8' },
             { "detelecine",  optional_argument, NULL,    '9' },
+            { "decomb",      optional_argument, NULL,    '5' },            
             { "grayscale",   no_argument,       NULL,    'g' },
             { "pixelratio",  no_argument,       NULL,    'p' },
             { "loosePixelratio", optional_argument,   NULL,    'P' },
@@ -1621,7 +1631,7 @@
         int c;
 
 		c = getopt_long( argc, argv,
-						 "hvuC:f:4i:Io:t:Lc:m::a:6:s:UFN:e:E:2dD:789gpOP::w:l:n:b:q:S:B:r:R:Qx:TY:X:VZ:z",
+						 "hvuC:f:4i:Io:t:Lc:m::a:6:s:UFN:e:E:2dD:7895gpOP::w:l:n:b:q:S:B:r:R:Qx:TY:X:VZ:z",
                          long_options, &option_index );
         if( c < 0 )
         {
@@ -1854,6 +1864,13 @@
                 }
                 detelecine = 1;
                 break;
+            case '5':
+                if( optarg != NULL )
+                {
+                    decomb_opt = strdup( optarg );
+                }
+                decomb = 1;
+                break;
             case 'g':
                 grayscale = 1;
                 break;
Index: libhb/Jamfile
===================================================================
--- libhb/Jamfile	(revision 1452)
+++ libhb/Jamfile	(working copy)
@@ -11,7 +11,7 @@
 demuxmpeg.c fifo.c render.c reader.c muxcommon.c muxmp4.c sync.c stream.c
 decsub.c deca52.c decdca.c encfaac.c declpcm.c encx264.c decavcodec.c encxvid.c
 muxavi.c enclame.c muxogm.c encvorbis.c dvd.c muxmkv.c deblock.c deinterlace.c 
-denoise.c detelecine.c lang.c enctheora.c ;
+denoise.c detelecine.c decomb.c lang.c enctheora.c ;
 
 Library libhb : $(LIBHB_SRC) ;
 
Index: libhb/hb.c
===================================================================
--- libhb/hb.c	(revision 1452)
+++ libhb/hb.c	(working copy)
@@ -510,30 +510,48 @@
         }
 
         // compare results
-        /* The final metric seems to be doing some kind of bits per pixel style calculation
-           to decide whether or not enough lines showed alternating colors for the frame size. */
+        /*  The final cc score for a plane is the percentage of combed pixels it contains.
+            Because sensitivity goes down to hundreths of a percent, multiply by 1000
+            so it will be easy to compare against the threhold value which is an integer. */
         cc[k] = (int)( ( cc_1 + cc_2 ) * 1000.0 / ( width * height ) );
 
-        /* If the plane's cc score meets the threshold, flag it as combed. */
+        /* If the plane's cc score surpasses the threshold (by default 0.09%), flag it as combed.
+           That means, with default settings, a frame is flagged if 1% or more of it is combed. */
         flag[k] = 0;
         if ( cc[k] > threshold )
         {
+//            hb_log("Threhold %i: %i/%i | PTS: %lld (%fs)", k, cc[k], threshold, buf->start, (float)buf->start / 90000 );
             flag[k] = 1;
         }
     }
 
 #if 0
 /* Debugging info */
-//    if(flag)
+    if(flag[0] || flag[1] || flag[2])
         hb_log("flags: %i/%i/%i | cc0: %i | cc1: %i | cc2: %i", flag[0], flag[1], flag[2], cc[0], cc[1], cc[2]);
 #endif
 
-    /* When more than one plane shows combing, tell the caller. */
-    if (flag[0] || flag[1] || flag[2] )
+    if( buf->flags & 16 )
     {
-        return 1;
+        /* It's a progressive frame.*/
+        if( flag[0] && ( flag[1] || flag[2] ) )
+        {
+            /* So only mark as combed if luma and one chroma show combing. */
+            return 1;
+        }
     }
-
+    else
+    {
+        /* It's not a progressive frame */
+        if( flag[0] || flag[1] || flag[2] )
+        {
+            /* So mark as framed if any plane shows combing. */
+            return 1;
+            
+        }
+    }
+    
+    /* Reaching this point means no combing detected. */
     return 0;
 }
 
Index: libhb/Makefile
===================================================================
--- libhb/Makefile	(revision 1452)
+++ libhb/Makefile	(working copy)
@@ -25,7 +25,7 @@
 	   update.c demuxmpeg.c fifo.c render.c reader.c muxcommon.c stream.c \
 	   muxmp4.c sync.c decsub.c deca52.c decdca.c encfaac.c declpcm.c encx264.c \
 	   decavcodec.c encxvid.c muxmkv.c muxavi.c enclame.c muxogm.c encvorbis.c \
-	   dvd.c  ipodutil.cpp deblock.c deinterlace.c denoise.c detelecine.c lang.c
+	   dvd.c  ipodutil.cpp deblock.c deinterlace.c denoise.c detelecine.c decomb.c lang.c
 OTMP = $(SRCS:%.c=%.o) 
 OBJS = $(OTMP:%.cpp=%.o)
 
Index: libhb/internal.h
===================================================================
--- libhb/internal.h	(revision 1452)
+++ libhb/internal.h	(working copy)
@@ -224,7 +224,8 @@
     FILTER_DEINTERLACE = 1,
     FILTER_DEBLOCK,
     FILTER_DENOISE,
-    FILTER_DETELECINE
+    FILTER_DETELECINE,
+    FILTER_DECOMB
 };
 
 extern hb_work_object_t * hb_objects;
Index: libhb/decomb.c
===================================================================
--- libhb/decomb.c	(revision 0)
+++ libhb/decomb.c	(revision 0)
@@ -0,0 +1,770 @@
+/* $Id: decomb.c,v 1.14 2008/04/25 5:00:00 jbrjake Exp $
+
+   This file is part of the HandBrake source code.
+   Homepage: <http://handbrake.fr/>.
+   It may be used under the terms of the GNU General Public License. 
+   
+   The yadif algorithm was created by Michael Niedermayer. */
+#include "hb.h"
+#include "ffmpeg/avcodec.h"
+#include "mpeg2dec/mpeg2.h"
+
+#define SUPPRESS_AV_LOG
+
+#define MODE_DEFAULT     0
+#define PARITY_DEFAULT   -1
+
+#define MCDEINT_MODE_DEFAULT   -1
+#define MCDEINT_QP_DEFAULT      1
+
+#define ABS(a) ((a) > 0 ? (a) : (-(a)))
+#define MIN3(a,b,c) MIN(MIN(a,b),c)
+#define MAX3(a,b,c) MAX(MAX(a,b),c)
+
+struct hb_filter_private_s
+{
+    int              pix_fmt;
+    int              width[3];
+    int              height[3];
+
+    int              mode;
+    int              parity;
+    
+    int              yadif_ready;
+
+    int              mcdeint_mode;
+    int              mcdeint_qp;
+
+    int              mcdeint_outbuf_size;
+    uint8_t        * mcdeint_outbuf;
+    AVCodecContext * mcdeint_avctx_enc;
+    AVFrame        * mcdeint_frame;
+    AVFrame        * mcdeint_frame_dec;
+
+    int              comb;
+    int              color_equal;
+    int              color_diff;
+    int              threshold;
+    int              deinterlaced_frames;
+    int              passed_frames;
+
+    uint8_t        * ref[4][3];
+    int              ref_stride[3];
+
+    AVPicture        pic_in;
+    AVPicture        pic_out;
+    hb_buffer_t *    buf_out[2];
+    hb_buffer_t *    buf_settings;
+};
+
+hb_filter_private_t * hb_decomb_init( int pix_fmt,
+                                           int width,
+                                           int height,
+                                           char * settings );
+
+int hb_decomb_work( hb_buffer_t * buf_in,
+                         hb_buffer_t ** buf_out,
+                         int pix_fmt,
+                         int width,
+                         int height,
+                         hb_filter_private_t * pv );
+
+void hb_decomb_close( hb_filter_private_t * pv );
+
+hb_filter_object_t hb_filter_decomb =
+{
+    FILTER_DECOMB,
+    "Decombs selectively with (ffmpeg or yadif/mcdeint)",
+    NULL,
+    hb_decomb_init,
+    hb_decomb_work,
+    hb_decomb_close,
+};
+
+int EfficientCubicInterpolate( int y0, int y1, int y2, int y3 )
+{
+    int result = ( y0 * -3 ) + ( y1 * 23 ) + ( y2 * 23 ) + ( y3 * -3 );
+    result /= 40;
+    
+    if( result > 255 )
+    {
+        result = 255;
+    }
+    else if( result < 0 )
+    {
+        result = 0;
+    }
+    
+    return result;
+}
+
+static void store_ref( const uint8_t ** pic,
+                             hb_filter_private_t * pv )
+{
+    memcpy( pv->ref[3],
+            pv->ref[0],
+            sizeof(uint8_t *)*3 );
+
+    memmove( pv->ref[0],
+             pv->ref[1],
+             sizeof(uint8_t *)*3*3 );
+
+    int i;
+    for( i = 0; i < 3; i++ )
+    {
+        const uint8_t * src = pic[i];
+        uint8_t * ref = pv->ref[2][i];
+
+        int w = pv->width[i];
+        int h = pv->height[i];
+        int ref_stride = pv->ref_stride[i];
+
+        int y;
+        for( y = 0; y < pv->height[i]; y++ )
+        {
+            memcpy(ref, src, w);
+            src = (uint8_t*)src + w;
+            ref = (uint8_t*)ref + ref_stride;
+        }
+    }
+}
+
+static void get_ref( uint8_t ** pic, hb_filter_private_t * pv, int frm )
+{
+    int i;
+    for( i = 0; i < 3; i++ )
+    {
+        uint8_t * dst = pic[i];
+        const uint8_t * ref = pv->ref[frm][i];
+        int w = pv->width[i];
+        int ref_stride = pv->ref_stride[i];
+        
+        int y;
+        for( y = 0; y < pv->height[i]; y++ )
+        {
+            memcpy(dst, ref, w);
+            dst += w;
+            ref += ref_stride;
+        }
+    }
+}
+
+static void blend_filter_line( uint8_t *dst,
+                               uint8_t *prev,
+                               uint8_t *cur,
+                               uint8_t *next,
+                               int plane,
+                               int parity,
+                               int y,
+                               hb_filter_private_t * pv )
+{
+    uint8_t *prev2 = parity ? prev : cur ;
+    uint8_t *next2 = parity ? cur  : next;
+
+    int w = pv->width[plane];
+    int refs = pv->ref_stride[plane];
+    int x;
+
+    for( x = 0; x < w; x++)
+    {
+
+        int result = 0;
+        result += -cur[-2*refs];
+        result += cur[-refs] * 2;
+        result += cur[0] * 6;
+        result += cur[+refs] * 2;
+        result += -cur[2*refs];
+        result /= 8;
+        
+        if( result > 255 )
+        {
+            result = 255;
+        }
+        if( result < 0 )
+        {
+            result = 0;
+        }
+        
+        dst[0] = result;
+
+        dst++;
+        cur++;
+        prev++;
+        next++;
+        prev2++;
+        next2++;
+    }
+}
+
+static void yadif_filter_line( uint8_t *dst,
+                               uint8_t *prev,
+                               uint8_t *cur,
+                               uint8_t *next,
+                               int plane,
+                               int parity,
+                               int y,
+                               hb_filter_private_t * pv )
+{
+    uint8_t *prev2 = parity ? prev : cur ;
+    uint8_t *next2 = parity ? cur  : next;
+
+    int w = pv->width[plane];
+    int refs = pv->ref_stride[plane];
+    int x;
+        
+    for( x = 0; x < w; x++)
+    {        
+        int c              = cur[-refs];
+        int d              = (prev2[0] + next2[0])>>1;
+        int e              = cur[+refs];
+        int temporal_diff0 = ABS(prev2[0] - next2[0]);
+        int temporal_diff1 = ( ABS(prev[-refs] - cur[-refs]) + ABS(prev[+refs] - cur[+refs]) ) >> 1;
+        int temporal_diff2 = ( ABS(next[-refs] - cur[-refs]) + ABS(next[+refs] - cur[+refs]) ) >> 1;
+        int diff           = MAX3(temporal_diff0>>1, temporal_diff1, temporal_diff2);
+        
+        int spatial_score  = ABS(cur[-refs-1] - cur[+refs-1]) + ABS(cur[-refs]-cur[+refs]) +
+                                     ABS(cur[-refs+1] - cur[+refs+1]) - 1;         
+        int spatial_pred;
+         
+        if( pv->mode >= 4  )
+        {
+            spatial_pred = EfficientCubicInterpolate( cur[-3*refs], cur[-refs], cur[+refs], cur[3*refs] );
+        }
+        else
+        {
+            spatial_pred = (c+e)>>1;
+        }
+
+#define YADIF_CHECK(j)\
+        {   int score = ABS(cur[-refs-1+j] - cur[+refs-1-j])\
+                      + ABS(cur[-refs  +j] - cur[+refs  -j])\
+                      + ABS(cur[-refs+1+j] - cur[+refs+1-j]);\
+            if( score < spatial_score ){\
+                spatial_score = score;\
+                if( pv->mode >= 4)\
+                {\
+                    switch(j)\
+                    {\
+                        case -1:\
+                            spatial_pred = EfficientCubicInterpolate(cur[-3 * refs - 3], cur[-refs -1], cur[+refs + 1], cur[3* refs + 3] );\
+                        break;\
+                        case -2:\
+                            spatial_pred = EfficientCubicInterpolate(cur[-3 * refs -5], cur[-refs -2], cur[+refs + 2], cur[3* refs + 5] );\
+                        break;\
+                        case 1:\
+                            spatial_pred = EfficientCubicInterpolate(cur[-3 * refs +3], cur[-refs +1], cur[+refs - 1], cur[3* refs -3] );\
+                        break;\
+                        case 2:\
+                            spatial_pred = EfficientCubicInterpolate(cur[-3 * refs +5], cur[-refs +2], cur[+refs - 2], cur[3*refs - 5] );\
+                        break;\
+                    }\
+                }\
+                else\
+                {\
+                    spatial_pred = (cur[-refs  +j] + cur[+refs  -j])>>1;\
+                }\
+                
+                YADIF_CHECK(-1) YADIF_CHECK(-2) }} }}
+                YADIF_CHECK( 1) YADIF_CHECK( 2) }} }}
+                                
+        /* Temporally adjust the spatial prediction by comparing against the
+           alternate (associated) fields in the previous and next frames. */
+        int b = (prev2[-2*refs] + next2[-2*refs])>>1;
+        int f = (prev2[+2*refs] + next2[+2*refs])>>1;
+        
+        int max = MAX3(d-e, d-c, MIN(b-c, f-e));
+        int min = MIN3(d-e, d-c, MAX(b-c, f-e));
+        
+        diff = MAX3( diff, min, -max );
+        
+        if( spatial_pred > d + diff )
+        {
+            spatial_pred = d + diff;
+        }
+        else if( spatial_pred < d - diff )
+        {
+            spatial_pred = d - diff;
+        }
+        
+        dst[0] = spatial_pred;
+                        
+end_of_yadif_filter_pixel:
+        dst++;
+        cur++;
+        prev++;
+        next++;
+        prev2++;
+        next2++;
+    }
+}
+
+static void yadif_filter( uint8_t ** dst,
+                          int parity,
+                          int tff,
+                          hb_filter_private_t * pv )
+{
+    int i;
+    for( i = 0; i < 3; i++ )
+    {
+        int w = pv->width[i];
+        int h = pv->height[i];
+        int ref_stride = pv->ref_stride[i];
+
+        int y;
+        for( y = 0; y < h; y++ )
+        {
+            if( pv->mode == 3)
+            {
+                uint8_t *prev = &pv->ref[0][i][y*ref_stride];
+                uint8_t *cur  = &pv->ref[1][i][y*ref_stride];
+                uint8_t *next = &pv->ref[2][i][y*ref_stride];
+                uint8_t *dst2 = &dst[i][y*w];
+
+                blend_filter_line( dst2, prev, cur, next, i, parity ^ tff, y, pv );
+                
+            }
+            else if( (y ^ parity) &  1 )
+            {
+                uint8_t *prev = &pv->ref[0][i][y*ref_stride];
+                uint8_t *cur  = &pv->ref[1][i][y*ref_stride];
+                uint8_t *next = &pv->ref[2][i][y*ref_stride];
+                uint8_t *dst2 = &dst[i][y*w];
+
+                yadif_filter_line( dst2, prev, cur, next, i, parity ^ tff, y, pv );
+            }
+            else
+            {
+                memcpy( &dst[i][y*w],
+                        &pv->ref[1][i][y*ref_stride],
+                        w * sizeof(uint8_t) );              
+            }
+        }
+    }
+}
+
+static void mcdeint_filter( uint8_t ** dst,
+                            uint8_t ** src,
+                            int parity,
+                            hb_filter_private_t * pv )
+{
+    int x, y, i;
+    int out_size;
+
+#ifdef SUPPRESS_AV_LOG
+    /* TODO: temporarily change log level to suppress obnoxious debug output */
+    int loglevel = av_log_get_level();
+    av_log_set_level( AV_LOG_QUIET );
+#endif
+
+    for( i=0; i<3; i++ )
+    {
+        pv->mcdeint_frame->data[i] = src[i];
+        pv->mcdeint_frame->linesize[i] = pv->width[i];
+    }
+    pv->mcdeint_avctx_enc->me_cmp     = FF_CMP_SAD;
+    pv->mcdeint_avctx_enc->me_sub_cmp = FF_CMP_SAD;
+    pv->mcdeint_frame->quality        = pv->mcdeint_qp * FF_QP2LAMBDA;
+
+    out_size = avcodec_encode_video( pv->mcdeint_avctx_enc,
+                                     pv->mcdeint_outbuf,
+                                     pv->mcdeint_outbuf_size,
+                                     pv->mcdeint_frame );
+
+    pv->mcdeint_frame_dec = pv->mcdeint_avctx_enc->coded_frame;
+
+    for( i = 0; i < 3; i++ )
+    {
+        int w    = pv->width[i];
+        int h    = pv->height[i];
+        int fils = pv->mcdeint_frame_dec->linesize[i];
+        int srcs = pv->width[i];
+
+        for( y = 0; y < h; y++ )
+        {
+            if( (y ^ parity) & 1 )
+            {
+                for( x = 0; x < w; x++ )
+                {
+                    if( (x-2)+(y-1)*w >= 0 && (x+2)+(y+1)*w < w*h )
+                    {
+                        uint8_t * filp =
+                            &pv->mcdeint_frame_dec->data[i][x + y*fils];
+                        uint8_t * srcp = &src[i][x + y*srcs];
+
+                        int diff0 = filp[-fils] - srcp[-srcs];
+                        int diff1 = filp[+fils] - srcp[+srcs];
+
+                        int spatial_score =
+                              ABS(srcp[-srcs-1] - srcp[+srcs-1])
+                            + ABS(srcp[-srcs  ] - srcp[+srcs  ])
+                            + ABS(srcp[-srcs+1] - srcp[+srcs+1]) - 1;
+
+                        int temp = filp[0];
+
+#define MCDEINT_CHECK(j)\
+                        {   int score = ABS(srcp[-srcs-1+j] - srcp[+srcs-1-j])\
+                                      + ABS(srcp[-srcs  +j] - srcp[+srcs  -j])\
+                                      + ABS(srcp[-srcs+1+j] - srcp[+srcs+1-j]);\
+                            if( score < spatial_score ) {\
+                                spatial_score = score;\
+                                diff0 = filp[-fils+j] - srcp[-srcs+j];\
+                                diff1 = filp[+fils-j] - srcp[+srcs-j];
+
+                        MCDEINT_CHECK(-1) MCDEINT_CHECK(-2) }} }}
+                        MCDEINT_CHECK( 1) MCDEINT_CHECK( 2) }} }}
+
+                        if(diff0 + diff1 > 0)
+                        {
+                            temp -= (diff0 + diff1 -
+                                     ABS( ABS(diff0) - ABS(diff1) ) / 2) / 2;
+                        }
+                        else
+                        {
+                            temp -= (diff0 + diff1 +
+                                     ABS( ABS(diff0) - ABS(diff1) ) / 2) / 2;
+                        }
+
+                        filp[0] = dst[i][x + y*w] =
+                            temp > 255U ? ~(temp>>31) : temp;
+                    }
+                    else
+                    {
+                        dst[i][x + y*w] =
+                            pv->mcdeint_frame_dec->data[i][x + y*fils];
+                    }
+                }
+            }
+        }
+
+        for( y = 0; y < h; y++ )
+        {
+            if( !((y ^ parity) & 1) )
+            {
+                for( x = 0; x < w; x++ )
+                {
+                    pv->mcdeint_frame_dec->data[i][x + y*fils] =
+                        dst[i][x + y*w]= src[i][x + y*srcs];
+                }
+            }
+        }
+    }
+
+#ifdef SUPPRESS_AV_LOG
+    /* TODO: restore previous log level */
+    av_log_set_level(loglevel);
+#endif
+}
+
+hb_filter_private_t * hb_decomb_init( int pix_fmt,
+                                           int width,
+                                           int height,
+                                           char * settings )
+{
+    if( pix_fmt != PIX_FMT_YUV420P )
+    {
+        return 0;
+    }
+
+    hb_filter_private_t * pv = calloc( 1, sizeof(struct hb_filter_private_s) );
+
+    pv->pix_fmt = pix_fmt;
+
+    pv->width[0]  = width;
+    pv->height[0] = height;
+    pv->width[1]  = pv->width[2]  = width >> 1;
+    pv->height[1] = pv->height[2] = height >> 1;
+
+    int buf_size = 3 * width * height / 2;
+    pv->buf_out[0] = hb_buffer_init( buf_size );
+    pv->buf_out[1] = hb_buffer_init( buf_size );
+    pv->buf_settings = hb_buffer_init( 0 );
+
+    pv->deinterlaced_frames = 0;
+    pv->passed_frames = 0;
+    pv->color_equal = 10;
+    pv->color_diff = 30;
+    pv->threshold = 9;
+
+    pv->yadif_ready    = 0;
+    pv->mode     = MODE_DEFAULT;
+    pv->parity   = PARITY_DEFAULT;
+
+    pv->mcdeint_mode   = MCDEINT_MODE_DEFAULT;
+    pv->mcdeint_qp     = MCDEINT_QP_DEFAULT;
+
+    if( settings )
+    {
+        sscanf( settings, "%d:%d:%d:%d",
+                &pv->mode,
+                &pv->color_equal,
+                &pv->color_diff,
+                &pv->threshold );
+    }
+    
+    if( pv->mode == 2 || pv->mode == 5 )
+    {
+        pv->mcdeint_mode = 0;
+    }
+    
+    /* Allocate yadif specific buffers */
+    if( pv->mode > 0 )
+    {
+        int i, j;
+        for( i = 0; i < 3; i++ )
+        {
+            int is_chroma = !!i;
+            int w = ((width   + 31) & (~31))>>is_chroma;
+            int h = ((height+6+ 31) & (~31))>>is_chroma;
+
+            pv->ref_stride[i] = w;
+
+            for( j = 0; j < 3; j++ )
+            {
+                pv->ref[j][i] = malloc( w*h*sizeof(uint8_t) ) + 3*w;
+            }
+        }
+    }
+
+    /* Allocate mcdeint specific buffers */
+    if( pv->mcdeint_mode >= 0 )
+    {
+        avcodec_init();
+        avcodec_register_all();
+
+        AVCodec * enc = avcodec_find_encoder( CODEC_ID_SNOW );
+
+        int i;
+        for (i = 0; i < 3; i++ )
+        {
+            AVCodecContext * avctx_enc;
+
+            avctx_enc = pv->mcdeint_avctx_enc = avcodec_alloc_context();
+
+            avctx_enc->width                    = width;
+            avctx_enc->height                   = height;
+            avctx_enc->time_base                = (AVRational){1,25};  // meaningless
+            avctx_enc->gop_size                 = 300;
+            avctx_enc->max_b_frames             = 0;
+            avctx_enc->pix_fmt                  = PIX_FMT_YUV420P;
+            avctx_enc->flags                    = CODEC_FLAG_QSCALE | CODEC_FLAG_LOW_DELAY;
+            avctx_enc->strict_std_compliance    = FF_COMPLIANCE_EXPERIMENTAL;
+            avctx_enc->global_quality           = 1;
+            avctx_enc->flags2                   = CODEC_FLAG2_MEMC_ONLY;
+            avctx_enc->me_cmp                   = FF_CMP_SAD; //SSE;
+            avctx_enc->me_sub_cmp               = FF_CMP_SAD; //SSE;
+            avctx_enc->mb_cmp                   = FF_CMP_SSE;
+
+            switch( pv->mcdeint_mode )
+            {
+                case 3:
+                    avctx_enc->refs = 3;
+                case 2:
+                    avctx_enc->me_method = ME_UMH;
+                case 1:
+                    avctx_enc->flags |= CODEC_FLAG_4MV;
+                    avctx_enc->dia_size =2;
+                case 0:
+                    avctx_enc->flags |= CODEC_FLAG_QPEL;
+            }
+
+            avcodec_open(avctx_enc, enc);
+        }
+
+        pv->mcdeint_frame       = avcodec_alloc_frame();
+        pv->mcdeint_outbuf_size = width * height * 10;
+        pv->mcdeint_outbuf      = malloc( pv->mcdeint_outbuf_size );
+    }
+
+    return pv;
+}
+
+void hb_decomb_close( hb_filter_private_t * pv )
+{
+    if( !pv )
+    {
+        return;
+    }
+
+    hb_log("deinterlacer: filtered %i | unfiltered %i | total %i", pv->deinterlaced_frames, pv->passed_frames, pv->deinterlaced_frames + pv->passed_frames);
+
+    /* Cleanup frame buffers */
+    if( pv->buf_out[0] )
+    {
+        hb_buffer_close( &pv->buf_out[0] );
+    }
+    if( pv->buf_out[1] )
+    {
+        hb_buffer_close( &pv->buf_out[1] );
+    }
+    if (pv->buf_settings )
+    {
+        hb_buffer_close( &pv->buf_settings );
+    }
+
+    /* Cleanup yadif specific buffers */
+    if( pv->mode > 0 )
+    {
+        int i;
+        for( i = 0; i<3*3; i++ )
+        {
+            uint8_t **p = &pv->ref[i%3][i/3];
+            if (*p)
+            {
+                free( *p - 3*pv->ref_stride[i/3] );
+                *p = NULL;
+            }
+        }
+    }
+
+    /* Cleanup mcdeint specific buffers */
+    if( pv->mcdeint_mode >= 0 )
+    {
+        if( pv->mcdeint_avctx_enc )
+        {
+            avcodec_close( pv->mcdeint_avctx_enc );
+            av_freep( &pv->mcdeint_avctx_enc );
+        }
+        if( pv->mcdeint_outbuf )
+        {
+            free( pv->mcdeint_outbuf );
+        }
+    }
+
+    free( pv );
+}
+
+int hb_decomb_work( hb_buffer_t * buf_in,
+                         hb_buffer_t ** buf_out,
+                         int pix_fmt,
+                         int width,
+                         int height,
+                         hb_filter_private_t * pv )
+{
+    if( !pv ||
+        pix_fmt != pv->pix_fmt ||
+        width   != pv->width[0] ||
+        height  != pv->height[0] )
+    {
+        return FILTER_FAILED;
+    }
+
+    avpicture_fill( &pv->pic_in, buf_in->data,
+                    pix_fmt, width, height );
+
+    /* Use libavcodec deinterlace if mode < 0 */
+    if( pv->mode == 0 )
+    {
+        avpicture_fill( &pv->pic_out, pv->buf_out[0]->data,
+                        pix_fmt, width, height );
+
+        /* Check for combing on the input frame */
+        int interlaced =  hb_detect_comb(buf_in, width, height, pv->color_equal, pv->color_diff, pv->threshold);
+        
+        if(interlaced)
+        {
+            avpicture_deinterlace( &pv->pic_out, &pv->pic_in,
+                                   pix_fmt, width, height );
+
+            pv->deinterlaced_frames++;
+            //hb_log("Frame %i is combed (Progressive: %s )", pv->deinterlaced_frames + pv->passed_frames, (buf_in->flags & 16) ? "Y" : "N");
+            
+            hb_buffer_copy_settings( pv->buf_out[0], buf_in );
+            *buf_out = pv->buf_out[0];            
+        }
+        else
+        {
+            /* No combing detected, pass input frame through unmolested.*/
+            
+            pv->passed_frames++;
+            
+            hb_buffer_copy_settings( pv->buf_out[0], buf_in );
+            *buf_out = buf_in;
+            
+        }
+
+        return FILTER_OK;
+    }
+    
+    /* Determine if top-field first layout */
+    int tff;
+    if( pv->parity < 0 )
+    {
+        tff = !!(buf_in->flags & PIC_FLAG_TOP_FIELD_FIRST);
+    }
+    else
+    {
+        tff = (pv->parity & 1) ^ 1;
+    }
+
+    /* Store current frame in yadif cache */
+    store_ref( (const uint8_t**)pv->pic_in.data, pv );
+
+    /* Note down if the input frame is combed */
+    pv->comb = (pv->comb << 1) | hb_detect_comb(buf_in, width, height, pv->color_equal, pv->color_diff, pv->threshold);
+
+    /* If yadif is not ready, store another ref and return FILTER_DELAY */
+    if( pv->yadif_ready == 0 )
+    {
+        store_ref( (const uint8_t**)pv->pic_in.data, pv );
+
+        hb_buffer_copy_settings( pv->buf_settings, buf_in );
+
+        /* don't let 'work_loop' send a chapter mark upstream */
+        buf_in->new_chap  = 0;
+
+        pv->yadif_ready = 1;
+
+        return FILTER_DELAY;
+    }
+
+    /* yadif works one frame behind so if the previous frame
+     * had combing, deinterlace it otherwise just output it. */
+    if(  (pv->comb & 2 ) == 0 )
+    {
+        /* previous frame not interlaced - copy cached input frame to buf_out */
+        
+        pv->passed_frames++;
+        
+        avpicture_fill( &pv->pic_out,  pv->buf_out[0]->data, pix_fmt, width, height );
+        get_ref( (uint8_t**)pv->pic_out.data, pv, 1 );
+        *buf_out = pv->buf_out[0];
+    }
+    else
+    {
+        /* Perform yadif filtering */
+        
+        pv->deinterlaced_frames++;
+        int frame;
+        for( frame = 0; frame <= ( ( pv->mode == 2 || pv->mode == 5 )? 1 : 0 ) ; frame++ )
+        {
+            int parity = frame ^ tff ^ 1;
+
+            avpicture_fill( &pv->pic_out, pv->buf_out[!(frame^1)]->data,
+                            pix_fmt, width, height );
+
+            yadif_filter( pv->pic_out.data, parity, tff, pv );
+
+            if( pv->mcdeint_mode >= 0 )
+            {
+                /* Perform mcdeint filtering */
+                avpicture_fill( &pv->pic_in,  pv->buf_out[(frame^1)]->data,
+                                pix_fmt, width, height );
+
+                mcdeint_filter( pv->pic_in.data, pv->pic_out.data, parity, pv );
+            }
+
+            *buf_out = pv->buf_out[!(frame^1)];
+        }
+    }
+
+    /* Copy buffered settings to output buffer settings */
+    hb_buffer_copy_settings( *buf_out, pv->buf_settings );
+
+    /* Replace buffered settings with input buffer settings */
+    hb_buffer_copy_settings( pv->buf_settings, buf_in );
+
+    /* don't let 'work_loop' send a chapter mark upstream */
+    buf_in->new_chap  = 0;
+
+    return FILTER_OK;
+}
Index: libhb/common.h
===================================================================
--- libhb/common.h	(revision 1452)
+++ libhb/common.h	(working copy)
@@ -574,6 +574,7 @@
 extern hb_filter_object_t hb_filter_deinterlace;
 extern hb_filter_object_t hb_filter_deblock;
 extern hb_filter_object_t hb_filter_denoise;
+extern hb_filter_object_t hb_filter_decomb;
 
 typedef void hb_error_handler_t( const char *errmsg );
 
Index: macosx/HandBrake.xcodeproj/project.pbxproj
===================================================================
--- macosx/HandBrake.xcodeproj/project.pbxproj	(revision 1452)
+++ macosx/HandBrake.xcodeproj/project.pbxproj	(working copy)
@@ -90,6 +90,8 @@
 		593034EC0BBA39A100172349 /* ChapterTitles.m in Sources */ = {isa = PBXBuildFile; fileRef = 593034EA0BBA39A100172349 /* ChapterTitles.m */; };
 		59CBD2370BBB44DA004A3BE3 /* parsecsv.c in Sources */ = {isa = PBXBuildFile; fileRef = 59CBD2360BBB44DA004A3BE3 /* parsecsv.c */; };
 		59CBD2650BBB4D1B004A3BE3 /* ChapterTitles.m in Sources */ = {isa = PBXBuildFile; fileRef = 593034EA0BBA39A100172349 /* ChapterTitles.m */; };
+		7497010F0DC281BB009200D8 /* decomb.c in Sources */ = {isa = PBXBuildFile; fileRef = 7497010E0DC281BB009200D8 /* decomb.c */; };
+		749701100DC281BB009200D8 /* decomb.c in Sources */ = {isa = PBXBuildFile; fileRef = 7497010E0DC281BB009200D8 /* decomb.c */; };
 		A22C85EC0D05D35000C10E36 /* HBPresets.h in Headers */ = {isa = PBXBuildFile; fileRef = A22C85EA0D05D35000C10E36 /* HBPresets.h */; };
 		A22C85ED0D05D35100C10E36 /* HBPresets.m in Sources */ = {isa = PBXBuildFile; fileRef = A22C85EB0D05D35000C10E36 /* HBPresets.m */; };
 		A25289E60D87A27D00461D5B /* enctheora.c in Sources */ = {isa = PBXBuildFile; fileRef = A25289E50D87A27D00461D5B /* enctheora.c */; };
@@ -258,6 +260,7 @@
 		593034E90BBA39A100172349 /* ChapterTitles.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ChapterTitles.h; sourceTree = "<group>"; };
 		593034EA0BBA39A100172349 /* ChapterTitles.m */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.objc; path = ChapterTitles.m; sourceTree = "<group>"; };
 		59CBD2360BBB44DA004A3BE3 /* parsecsv.c */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.c; name = parsecsv.c; path = ../test/parsecsv.c; sourceTree = SOURCE_ROOT; };
+		7497010E0DC281BB009200D8 /* decomb.c */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.c; name = decomb.c; path = ../libhb/decomb.c; sourceTree = SOURCE_ROOT; };
 		A22C85EA0D05D35000C10E36 /* HBPresets.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = HBPresets.h; sourceTree = "<group>"; };
 		A22C85EB0D05D35000C10E36 /* HBPresets.m */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.objc; path = HBPresets.m; sourceTree = "<group>"; };
 		A25289E50D87A27D00461D5B /* enctheora.c */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.c; name = enctheora.c; path = ../libhb/enctheora.c; sourceTree = SOURCE_ROOT; };
@@ -435,6 +438,7 @@
 			isa = PBXGroup;
 			children = (
 				A25289E50D87A27D00461D5B /* enctheora.c */,
+				7497010E0DC281BB009200D8 /* decomb.c */,
 				B48359A70C82960500E04440 /* lang.c */,
 				EAA526920C3B25D200944FF2 /* stream.c */,
 				0DFA5C7E0B8DD3B60020BC09 /* declpcm.c */,
@@ -787,6 +791,7 @@
 				FC8519550C59A02C0073812C /* deinterlace.c in Sources */,
 				FC8519560C59A02C0073812C /* deblock.c in Sources */,
 				FC8519570C59A02C0073812C /* detelecine.c in Sources */,
+				749701100DC281BB009200D8 /* decomb.c in Sources */,
 			);
 			runOnlyForDeploymentPostprocessing = 0;
 		};
@@ -832,6 +837,7 @@
 				FC8519530C59A02C0073812C /* detelecine.c in Sources */,
 				B48359A80C82960500E04440 /* lang.c in Sources */,
 				A25289E60D87A27D00461D5B /* enctheora.c in Sources */,
+				7497010F0DC281BB009200D8 /* decomb.c in Sources */,
 			);
 			runOnlyForDeploymentPostprocessing = 0;
 		};
It's called with --decomb. The options are mode, color equal, color diff, threshold.

Mode is different than the yadif_mode in --deinterlace:

Mode 0: uses "Fast" (ffmpeg, the traditional deinterlacer in HandBrake).
Mode 1: Default, uses yadif, mode 0 (I don't bother yadif mode 2), traditional 2-point interpolation
Mode 2: uses yadif and mcdeint (probably not worth it)
Mode 3: does a lowpass5 blend instead of interpolating
Mode 4: uses yadif, mode 0, with the new 4-point interpolation
Mode 5: yadif with the new 4-point interpolation and mcdeint

The last three options are the same as I've explained in earlier posts -- they're the comb detection parameters. I'm sticking with Transcode's values for them for now, but might increase the sensitivity....oh, and the threshold? Figured out how that works. It's the hundredth of a percent of a plane's pixels that need to show combing for the plane to show combing -- by default just under 1%, (greater than 0.09%).

Default is:
1:10:30:9

Know bug:
It's overreading a bit on the bottom of frames, so there's half a garbage line sometimes on the bottom. I think I can fix that, and I'm not sure it doesn't always happens with yadif in HB (I've read people complaining about it before).

RESOURCES

The page I got the CubicInterpolation() function from:
http://local.wasp.uwa.edu.au/~pbourke/o ... rpolation/

Efficient Cubic Interpolation:
http://www.neuron2.net/library/cubicinterp.html

Gerard de Haan is a video engineer with Philips who writes easy-to-grasp papers explaining deinterlacing algorithms he's studied, which he is gracious enough to put on the web for free as PDFs:
http://www.es.ele.tue.nl/~dehaan/papers.html

FIR FAQ:
http://www.dspguru.com/info/faqs/firfaq.htm

Also, if you wikipedia any of the keywords in this post, you'll find some useful articles.
jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Re: Combing Detection

Post by jbrjake »

Ooops, found a stupid bug in hb_detect_comb. Here's an updated patch: http://handbrake.pastebin.ca/1008863
sdm
Bright Spark User
Posts: 194
Joined: Mon Feb 19, 2007 4:53 pm

Re: Combing Detection

Post by sdm »

Any chance of of a patch to access the different modes and and comb detect options in the MacGui? Dynaflash? I'm curious if there is an elegant way...

As for frame blending, I've generally always avoided it - although I know it has its benefits like jbrjake mentions as well as smooth motion.

I'd be curious to try it out though.

--sdm.
cvk_b
Veteran User
Posts: 527
Joined: Sun Mar 18, 2007 2:11 am

Re: Combing Detection

Post by cvk_b »

jbrjake wrote:Ooops, found a stupid bug in hb_detect_comb. Here's an updated patch: http://handbrake.pastebin.ca/1008863
Is this still the latest?
jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Re: Combing Detection

Post by jbrjake »

cvk_b wrote:
jbrjake wrote:Ooops, found a stupid bug in hb_detect_comb. Here's an updated patch: http://handbrake.pastebin.ca/1008863
Is this still the latest?
Yeah. Got some more stuff in store, but before I can code it, it's going to require a trip to Office Depot for graphing paper, a compass, and a protractor....
rhester
Veteran User
Posts: 2888
Joined: Tue Apr 18, 2006 10:24 pm

Re: Combing Detection

Post by rhester »

jbrjake wrote:Yeah. Got some more stuff in store, but before I can code it, it's going to require a trip to Office Depot for graphing paper, a compass, and a protractor....
North is that way. ^

Rodney
dynaflash
Veteran User
Posts: 3820
Joined: Thu Nov 02, 2006 8:19 pm

Re: Combing Detection

Post by dynaflash »

sdm wrote:Any chance of of a patch to access the different modes and and comb detect options in the MacGui? Dynaflash? I'm curious if there is an elegant way...
Well, as I am notoriously cli-phobic I do have a local version here that has a Decomb option in the gui, but it just sets jbjakes currently recommended setting. Nothing that allows you to set each value individually. I suspect more will be coming from him so I don't want to add more yet until the core feature gets the nod from him.
jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Re: Combing Detection

Post by jbrjake »

Well, the stuff I was looking into that required a compass and protractor has kicked my ass. I forgot that I suck at math, and concentric circle modeling requires trig and a little calc. So now I'm working backwards. Picked up a copy of Hamming's Digital Filters, which is full of more trig and calc I don't understand, but at least it explains the variable names ;-)

I *think* this is pretty much finalized. I'd sort of like to get it into the next snapshot/0.9.3, but that'll depend on how much feedback it gets.

I've made some significant changes to the comb detection, away from how Transcode does it. Before, it was being doubly sensitive in chroma planes and weighing them equally against the luma. I don't think that makes sense, since we're dealing with YUV420 video where the chroma plane is very low rez. I don't see how it would show combing very well. I've changed it so the sensitivity doesn't get doubled for them. Then, instead of just saying "Okay, if any of the 3 planes show combing, it's combed" I'm doing a weighted average of all 3 planes and comparing the average against the threshold value. The weights strongly favor the luma.

I've also changed the default parameters, which was necessary because they don't do the same things anymore, after the above changes. The old ones were 10:30:9, now they're 10:15:9. This makes it easier to see consecutive lines as "different." It's strong enough to get rid of most of the artifacts sdm was getting, like the white fade-in in the Oasis video, and the quick cut at the beginning of the Lemonheads' video.

But what about progressive stuff? After all, we don't want to munge the opening credits to A Night at the Museum. So for progressive frames, I've added more parameters. It's another triplet that just gets appended to the end of the settings string. I'm using 10:35:9 for those, a value I arrived at by encoding chapter 1 of A Night at the Museum over and over. It also clears up the opening credits to Angel.

Here's what I'd *like* to do at a later date: treat ivtc'd sequences the same as progressive, by using the weaker progressive parameters. Doing this means setting render.c to mark detelecined frames as progressive as well. This makes the comb detection more flexible. It will deinterlace most every 30fps video frame, and only deinterlace parts of progressive or ivtc'd material if they show more obvious combing. But doing this is a pain. See, decomb runs in the regular filter chain, and detelecine does too. But VFR only extends the duration of detelecined frames after a 3-frame delay buffer. This is because when detelecine drops a frame, it's going to be two frames into a telecine sequence. Render.c needs to go back and rewrite the durations for those earlier frames, to make up the lost time. It's only when the duration is rewritten -- after that buffer -- that those frames can be flagged as progressive...and that's long after decomb has run. So the regular filter loop needs to be set not to run decomb. Then I'd add a new filter loop at the bottom of render, after the delay queue, solely to run decomb. At this point I realized that wouldn't work. Why? Because decomb needs to run before the frame has subtitles applied or is scaled or cropped. That means this requires rearranging render.c. VFR will do its delay queue thing before rendering of the frames has finished. It'll caches and extends before subs or scaling or cropping are applied. I haven't made these changes to render.c. I can't get it to work right, and will need the aid of a Real Programmer. But when they are made at some future date, decomb will benefit immediately.

So. Here's a current patch. It's everything but what I discussed in the preceding paragraph:

http://handbrake.pastebin.ca/1027041

Code: Select all

Index: test/test.c
===================================================================
--- test/test.c	(revision 1469)
+++ test/test.c	(working copy)
@@ -40,6 +40,8 @@
 static char * denoise_opt           = 0;
 static int    detelecine            = 0;
 static char * detelecine_opt        = 0;
+static int    decomb                = 0;
+static char * decomb_opt            = 0;
 static int    grayscale   = 0;
 static int    vcodec      = HB_VCODEC_FFMPEG;
 static int    h264_13     = 0;
@@ -819,6 +821,11 @@
                 hb_filter_detelecine.settings = detelecine_opt;
                 hb_list_add( job->filters, &hb_filter_detelecine );
             }
+            if( decomb )
+            {
+                hb_filter_decomb.settings = decomb_opt;
+                hb_list_add( job->filters, &hb_filter_decomb );
+            }
             if( deinterlace )
             {
                 hb_filter_deinterlace.settings = deinterlace_opt;
@@ -1474,6 +1481,8 @@
      "          <weak/medium/strong>\n"
      "    -9, --detelecine        Detelecine video with pullup filter\n"
      "          <L:R:T:B:SB:MP>   (default 1:1:4:4:0:0)\n"
+     "    -5, --decomb           Selectively deinterlaces when it detects combing\n"
+     "          <M:EQ:DF:TR:EQ:DF:TR>     (default: 4:10:15:9:10:35:9)\n"
     "    -g, --grayscale         Grayscale encoding\n"
     "    -p, --pixelratio        Store pixel aspect ratio in video stream\n"
     "    -P, --loosePixelratio   Store pixel aspect ratio with specified width\n"
@@ -1612,6 +1621,7 @@
             { "deblock",     optional_argument, NULL,    '7' },
             { "denoise",     optional_argument, NULL,    '8' },
             { "detelecine",  optional_argument, NULL,    '9' },
+            { "decomb",      optional_argument, NULL,    '5' },            
             { "grayscale",   no_argument,       NULL,    'g' },
             { "pixelratio",  no_argument,       NULL,    'p' },
             { "loosePixelratio", optional_argument,   NULL,    'P' },
@@ -1641,7 +1651,7 @@
         int c;
 
 		c = getopt_long( argc, argv,
-						 "hvuC:f:4i:Io:t:Lc:m::a:6:s:UFN:e:E:2dD:789gpOP::w:l:n:b:q:S:B:r:R:Qx:TY:X:VZ:z",
+						 "hvuC:f:4i:Io:t:Lc:m::a:6:s:UFN:e:E:2dD:7895gpOP::w:l:n:b:q:S:B:r:R:Qx:TY:X:VZ:z",
                          long_options, &option_index );
         if( c < 0 )
         {
@@ -1874,6 +1884,13 @@
                 }
                 detelecine = 1;
                 break;
+            case '5':
+                if( optarg != NULL )
+                {
+                    decomb_opt = strdup( optarg );
+                }
+                decomb = 1;
+                break;
             case 'g':
                 grayscale = 1;
                 break;
Index: libhb/Jamfile
===================================================================
--- libhb/Jamfile	(revision 1469)
+++ libhb/Jamfile	(working copy)
@@ -11,7 +11,7 @@
 demuxmpeg.c fifo.c render.c reader.c muxcommon.c muxmp4.c sync.c stream.c
 decsub.c deca52.c decdca.c encfaac.c declpcm.c encx264.c decavcodec.c encxvid.c
 muxavi.c enclame.c muxogm.c encvorbis.c dvd.c muxmkv.c deblock.c deinterlace.c 
-denoise.c detelecine.c lang.c enctheora.c ;
+denoise.c detelecine.c decomb.c lang.c enctheora.c ;
 
 Library libhb : $(LIBHB_SRC) ;
 
Index: libhb/hb.c
===================================================================
--- libhb/hb.c	(revision 1469)
+++ libhb/hb.c	(working copy)
@@ -446,18 +446,25 @@
  * @param color_diff  Sensitivity for detecting different colors
  * @param threshold   Sensitivity for flagging planes as combed
  */
-int hb_detect_comb( hb_buffer_t * buf, int width, int height, int color_equal, int color_diff, int threshold )
+int hb_detect_comb( hb_buffer_t * buf, int width, int height, int color_equal, int color_diff, int threshold, int prog_equal, int prog_diff, int prog_threshold )
 {
-    int j, k, n, off, block, cc_1, cc_2, cc[3], flag[3];
+    int j, k, n, off, cc_1, cc_2, cc[3], flag[3], plane_color_equal, plane_color_diff;
     uint16_t s1, s2, s3, s4;
     cc_1 = 0; cc_2 = 0;
 
     int offset = 0;
+    
+    if ( buf->flags & 16 )
+    {
+        /* Frame is progressive, be more discerning. */
+        color_diff = prog_diff;
+        color_equal = prog_equal;
+        threshold = prog_threshold;
+    }
 
+    /* One pas for Y, one pass for Cb, one pass for Cr */    
     for( k = 0; k < 3; k++ )
     {
-        /* One pas for Y, one pass for Cb, one pass for Cr */
-
         if( k == 1 )
         {
             /* Y has already been checked, now offset by Y's dimensions
@@ -466,9 +473,6 @@
             offset = width * height;
             width >>= 1;
             height >>= 1;
-            threshold >>= 1;
-            color_equal >>= 1;
-            color_diff >>= 1;
         }
         else if ( k == 2 )
         {
@@ -477,10 +481,7 @@
             offset *= 5/4;
         }
 
-        /* Look at one horizontal line at a time */
-        block = width;
-
-        for( j = 0; j < block; ++j )
+        for( j = 0; j < width; ++j )
         {
             off = 0;
 
@@ -488,9 +489,9 @@
             {
                 /* Look at groups of 4 sequential horizontal lines */
                 s1 = ( ( buf->data + offset )[ off + j             ] & 0xff );
-                s2 = ( ( buf->data + offset )[ off + j + block     ] & 0xff );
-                s3 = ( ( buf->data + offset )[ off + j + 2 * block ] & 0xff );
-                s4 = ( ( buf->data + offset )[ off + j + 3 * block ] & 0xff );
+                s2 = ( ( buf->data + offset )[ off + j + width     ] & 0xff );
+                s3 = ( ( buf->data + offset )[ off + j + 2 * width ] & 0xff );
+                s4 = ( ( buf->data + offset )[ off + j + 3 * width ] & 0xff );
 
                 /* Note if the 1st and 2nd lines are more different in
                    color than the 1st and 3rd lines are similar in color.*/
@@ -505,39 +506,38 @@
                         ++cc_2;
 
                 /* Now move down 2 horizontal lines before starting over.*/
-                off += 2 * block;
+                off += 2 * width;
             }
         }
 
         // compare results
-        /* The final metric seems to be doing some kind of bits per pixel style calculation
-           to decide whether or not enough lines showed alternating colors for the frame size. */
+        /*  The final cc score for a plane is the percentage of combed pixels it contains.
+            Because sensitivity goes down to hundreths of a percent, multiply by 1000
+            so it will be easy to compare against the threhold value which is an integer. */
         cc[k] = (int)( ( cc_1 + cc_2 ) * 1000.0 / ( width * height ) );
-
-        /* If the plane's cc score meets the threshold, flag it as combed. */
-        flag[k] = 0;
-        if ( cc[k] > threshold )
-        {
-            flag[k] = 1;
-        }
     }
 
+
+    /* HandBrake is all yuv420, so weight the average percentage of all 3 planes accordingly.*/
+    int average_cc = ( 2 * cc[0] + ( cc[1] / 2 ) + ( cc[2] / 2 ) ) / 3;
+    
+    /* Now see if that average percentage of combed pixels surpasses the threshold percentage given by the user.*/
+    if( average_cc > threshold )
+    {
 #if 0
-/* Debugging info */
-//    if(flag)
-        hb_log("flags: %i/%i/%i | cc0: %i | cc1: %i | cc2: %i", flag[0], flag[1], flag[2], cc[0], cc[1], cc[2]);
+            hb_log("Average %i combed (Threshold %i) %i/%i/%i | PTS: %lld (%fs) %s", average_cc, threshold, cc[0], cc[1], cc[2], buf->start, (float)buf->start / 90000, (buf->flags & 16) ? "Film" : "Video" );
 #endif
-
-    /* When more than one plane shows combing, tell the caller. */
-    if (flag[0] || flag[1] || flag[2] )
-    {
         return 1;
     }
 
+#if 0
+    hb_log("SKIPPED Average %i combed (Threshold %i) %i/%i/%i | PTS: %lld (%fs) %s", average_cc, threshold, cc[0], cc[1], cc[2], buf->start, (float)buf->start / 90000, (buf->flags & 16) ? "Film" : "Video" );
+#endif
+
+    /* Reaching this point means no combing detected. */
     return 0;
 }
 
-
 /**
  * Calculates job width and height for anamorphic content,
  *
Index: libhb/Makefile
===================================================================
--- libhb/Makefile	(revision 1469)
+++ libhb/Makefile	(working copy)
@@ -25,7 +25,7 @@
 	   update.c demuxmpeg.c fifo.c render.c reader.c muxcommon.c stream.c \
 	   muxmp4.c sync.c decsub.c deca52.c decdca.c encfaac.c declpcm.c encx264.c \
 	   decavcodec.c encxvid.c muxmkv.c muxavi.c enclame.c muxogm.c encvorbis.c \
-	   dvd.c  ipodutil.cpp deblock.c deinterlace.c denoise.c detelecine.c lang.c
+	   dvd.c  ipodutil.cpp deblock.c deinterlace.c denoise.c detelecine.c decomb.c lang.c
 OTMP = $(SRCS:%.c=%.o) 
 OBJS = $(OTMP:%.cpp=%.o)
 
Index: libhb/hb.h
===================================================================
--- libhb/hb.h	(revision 1469)
+++ libhb/hb.h	(working copy)
@@ -79,7 +79,7 @@
 /* hb_detect_comb()
    Analyze a frame for interlacing artifacts, returns true if they're found.
    Taken from Thomas Oestreich's 32detect filter in the Transcode project.  */
-int hb_detect_comb( hb_buffer_t * buf, int width, int height, int color_equal, int color_diff, int threshold );
+int hb_detect_comb( hb_buffer_t * buf, int width, int height, int color_equal, int color_diff, int threshold, int prog_equal, int prog_diff, int prog_threshold );
 
 void          hb_get_preview( hb_handle_t *, hb_title_t *, int,
                               uint8_t * );
Index: libhb/internal.h
===================================================================
--- libhb/internal.h	(revision 1469)
+++ libhb/internal.h	(working copy)
@@ -224,7 +224,8 @@
     FILTER_DEINTERLACE = 1,
     FILTER_DEBLOCK,
     FILTER_DENOISE,
-    FILTER_DETELECINE
+    FILTER_DETELECINE,
+    FILTER_DECOMB
 };
 
 extern hb_work_object_t * hb_objects;
Index: libhb/scan.c
===================================================================
--- libhb/scan.c	(revision 1469)
+++ libhb/scan.c	(working copy)
@@ -432,7 +432,7 @@
         buf_raw = hb_list_item( list_raw, 0 );
 
         /* Check preview for interlacing artifacts */
-        if( hb_detect_comb( buf_raw, title->width, title->height, 10, 30, 9 ) )
+        if( hb_detect_comb( buf_raw, title->width, title->height, 10, 30, 9, 10, 30, 9 ) )
         {
             hb_log("Interlacing detected in preview frame %i", i);
             interlaced_preview_count++;
Index: libhb/decomb.c
===================================================================
--- libhb/decomb.c	(revision 0)
+++ libhb/decomb.c	(revision 0)
@@ -0,0 +1,800 @@
+/* $Id: decomb.c,v 1.14 2008/04/25 5:00:00 jbrjake Exp $
+
+   This file is part of the HandBrake source code.
+   Homepage: <http://handbrake.fr/>.
+   It may be used under the terms of the GNU General Public License. 
+   
+   The yadif algorithm was created by Michael Niedermayer. */
+#include "hb.h"
+#include "ffmpeg/avcodec.h"
+#include "mpeg2dec/mpeg2.h"
+
+#define SUPPRESS_AV_LOG
+
+#define MODE_DEFAULT     4
+#define PARITY_DEFAULT   -1
+
+#define MCDEINT_MODE_DEFAULT   -1
+#define MCDEINT_QP_DEFAULT      1
+
+#define ABS(a) ((a) > 0 ? (a) : (-(a)))
+#define MIN3(a,b,c) MIN(MIN(a,b),c)
+#define MAX3(a,b,c) MAX(MAX(a,b),c)
+
+struct hb_filter_private_s
+{
+    int              pix_fmt;
+    int              width[3];
+    int              height[3];
+
+    int              mode;
+    int              parity;
+    
+    int              yadif_ready;
+
+    int              mcdeint_mode;
+    int              mcdeint_qp;
+
+    int              mcdeint_outbuf_size;
+    uint8_t        * mcdeint_outbuf;
+    AVCodecContext * mcdeint_avctx_enc;
+    AVFrame        * mcdeint_frame;
+    AVFrame        * mcdeint_frame_dec;
+
+    int              comb;
+    int              color_equal;
+    int              color_diff;
+    int              threshold;
+    int              prog_equal;
+    int              prog_diff;
+    int              prog_threshold;
+    int              deinterlaced_frames;
+    int              passed_frames;
+
+    uint8_t        * ref[4][3];
+    int              ref_stride[3];
+
+    AVPicture        pic_in;
+    AVPicture        pic_out;
+    hb_buffer_t *    buf_out[2];
+    hb_buffer_t *    buf_settings;
+};
+
+hb_filter_private_t * hb_decomb_init( int pix_fmt,
+                                           int width,
+                                           int height,
+                                           char * settings );
+
+int hb_decomb_work( hb_buffer_t * buf_in,
+                         hb_buffer_t ** buf_out,
+                         int pix_fmt,
+                         int width,
+                         int height,
+                         hb_filter_private_t * pv );
+
+void hb_decomb_close( hb_filter_private_t * pv );
+
+hb_filter_object_t hb_filter_decomb =
+{
+    FILTER_DECOMB,
+    "Decombs selectively with (ffmpeg or yadif/mcdeint or blending)",
+    NULL,
+    hb_decomb_init,
+    hb_decomb_work,
+    hb_decomb_close,
+};
+
+int cubic_interpolate( int y0, int y1, int y2, int y3 )
+{
+    /* From http://www.neuron2.net/library/cubicinterp.html */
+    int result = ( y0 * -3 ) + ( y1 * 23 ) + ( y2 * 23 ) + ( y3 * -3 );
+    result /= 40;
+    
+    if( result > 255 )
+    {
+        result = 255;
+    }
+    else if( result < 0 )
+    {
+        result = 0;
+    }
+    
+    return result;
+}
+
+static void store_ref( const uint8_t ** pic,
+                             hb_filter_private_t * pv )
+{
+    memcpy( pv->ref[3],
+            pv->ref[0],
+            sizeof(uint8_t *)*3 );
+
+    memmove( pv->ref[0],
+             pv->ref[1],
+             sizeof(uint8_t *)*3*3 );
+
+    int i;
+    for( i = 0; i < 3; i++ )
+    {
+        const uint8_t * src = pic[i];
+        uint8_t * ref = pv->ref[2][i];
+
+        int w = pv->width[i];
+        int h = pv->height[i];
+        int ref_stride = pv->ref_stride[i];
+
+        int y;
+        for( y = 0; y < pv->height[i]; y++ )
+        {
+            memcpy(ref, src, w);
+            src = (uint8_t*)src + w;
+            ref = (uint8_t*)ref + ref_stride;
+        }
+    }
+}
+
+static void get_ref( uint8_t ** pic, hb_filter_private_t * pv, int frm )
+{
+    int i;
+    for( i = 0; i < 3; i++ )
+    {
+        uint8_t * dst = pic[i];
+        const uint8_t * ref = pv->ref[frm][i];
+        int w = pv->width[i];
+        int ref_stride = pv->ref_stride[i];
+        
+        int y;
+        for( y = 0; y < pv->height[i]; y++ )
+        {
+            memcpy(dst, ref, w);
+            dst += w;
+            ref += ref_stride;
+        }
+    }
+}
+
+int blend_filter_pixel( int up2, int up1, int current, int down1, int down2 )
+{
+    /* Low-pass 5-tap filter */
+    int result = 0;
+    result += -up2;
+    result += up1 * 2;
+    result += current * 6;
+    result += down1 *2;
+    result += -down2;
+    result /= 8;
+
+    if( result > 255 )
+    {
+        result = 255;
+    }
+    if( result < 0 )
+    {
+        result = 0;
+    }
+    
+    return result;
+}
+
+static void blend_filter_line( uint8_t *dst,
+                               uint8_t *cur,
+                               int plane,
+                               int y,
+                               hb_filter_private_t * pv )
+{
+    int w = pv->width[plane];
+    int refs = pv->ref_stride[plane];
+    int x;
+
+    for( x = 0; x < w; x++)
+    {
+        
+        dst[0] = blend_filter_pixel( cur[-2*refs], cur[-refs], cur[0], cur[+refs], cur[2*refs] );
+
+        dst++;
+        cur++;
+    }
+}
+
+static void yadif_filter_line( uint8_t *dst,
+                               uint8_t *prev,
+                               uint8_t *cur,
+                               uint8_t *next,
+                               int plane,
+                               int parity,
+                               int y,
+                               hb_filter_private_t * pv )
+{
+    uint8_t *prev2 = parity ? prev : cur ;
+    uint8_t *next2 = parity ? cur  : next;
+
+    int w = pv->width[plane];
+    int refs = pv->ref_stride[plane];
+    int x;
+        
+    for( x = 0; x < w; x++)
+    {        
+
+#if 0
+        if( !detect_combed_pixel(cur[0], cur[-refs], cur[+refs], cur[2*refs], prev[0], next[0] ) )
+        {
+            dst[0] = cur[0];
+            goto end_of_yadif_filter_pixel;
+        }
+#endif
+        
+        /* Pixel above*/
+        int c              = cur[-refs];
+        /* Temporal average -- the current pixel location in the previous and next fields */
+        int d              = (prev2[0] + next2[0])>>1;
+        /* Pixel below */
+        int e              = cur[+refs];
+        
+        /* How the current pixel changes from the field before to the field after */
+        int temporal_diff0 = ABS(prev2[0] - next2[0]);
+        /* The average of how much the pixels above and below change from the field before to now. */
+        int temporal_diff1 = ( ABS(prev[-refs] - cur[-refs]) + ABS(prev[+refs] - cur[+refs]) ) >> 1;
+        /* The average of how much the pixels above and below change from now to the next field. */
+        int temporal_diff2 = ( ABS(next[-refs] - cur[-refs]) + ABS(next[+refs] - cur[+refs]) ) >> 1;
+        /* For the actual difference, use the largest of the previous average diffs. */
+        int diff           = MAX3(temporal_diff0>>1, temporal_diff1, temporal_diff2);
+        
+        /* SAD of how the pixel-1, the pixel, and the pixel+1 change from the line above to below. */ 
+        int spatial_score  = ABS(cur[-refs-1] - cur[+refs-1]) + ABS(cur[-refs]-cur[+refs]) +
+                                     ABS(cur[-refs+1] - cur[+refs+1]) - 1;         
+        int spatial_pred;
+         
+        /* Spatial pred is either a bilinear or cubic vertical interpolation. */
+        if( pv->mode >= 4  )
+        {
+            spatial_pred = cubic_interpolate( cur[-3*refs], cur[-refs], cur[+refs], cur[3*refs] );
+        }
+        else
+        {
+            spatial_pred = (c+e)>>1;
+        }
+
+/* EDDI: Edge Directed Deinterlacing Interpolation
+   Uses the Martinez-Lim Line Shift Parametric Modeling algorithm...I think.
+   Checks 4 different slopes to see if there is more similarity along a diagonal
+   than there was vertically. If a diagonal is more similar, then it indicates
+   an edge, so interpolate along that instead of a vertical line, using either
+   linear or cubic interpolation depending on mode. */
+#define YADIF_CHECK(j)\
+        {   int score = ABS(cur[-refs-1+j] - cur[+refs-1-j])\
+                      + ABS(cur[-refs  +j] - cur[+refs  -j])\
+                      + ABS(cur[-refs+1+j] - cur[+refs+1-j]);\
+            if( score < spatial_score ){\
+                spatial_score = score;\
+                if( pv->mode >= 4)\
+                {\
+                    switch(j)\
+                    {\
+                        case -1:\
+                            spatial_pred = cubic_interpolate(cur[-3 * refs - 3], cur[-refs -1], cur[+refs + 1], cur[3* refs + 3] );\
+                        break;\
+                        case -2:\
+                            spatial_pred = cubic_interpolate( ( ( cur[-3*refs - 4] + cur[-refs - 4] ) / 2 ) , cur[-refs -2], cur[+refs + 2], ( ( cur[3*refs + 4] + cur[refs + 4] ) / 2 ) );\
+                        break;\
+                        case 1:\
+                            spatial_pred = cubic_interpolate(cur[-3 * refs +3], cur[-refs +1], cur[+refs - 1], cur[3* refs -3] );\
+                        break;\
+                        case 2:\
+                            spatial_pred = cubic_interpolate(( ( cur[-3*refs + 4] + cur[-refs + 4] ) / 2 ), cur[-refs +2], cur[+refs - 2], ( ( cur[3*refs - 4] + cur[refs - 4] ) / 2 ) );\
+                        break;\
+                    }\
+                }\
+                else\
+                {\
+                    spatial_pred = ( cur[-refs +j] + cur[+refs -j] ) >>1;\
+                }\
+                
+                YADIF_CHECK(-1) YADIF_CHECK(-2) }} }}
+                YADIF_CHECK( 1) YADIF_CHECK( 2) }} }}
+                                
+        /* Temporally adjust the spatial prediction by comparing against the
+           alternate (associated) fields in the previous and next frames. */
+        int b = (prev2[-2*refs] + next2[-2*refs])>>1;
+        int f = (prev2[+2*refs] + next2[+2*refs])>>1;
+        
+        /* Find the median value */
+        int max = MAX3(d-e, d-c, MIN(b-c, f-e));
+        int min = MIN3(d-e, d-c, MAX(b-c, f-e));
+        diff = MAX3( diff, min, -max );
+        
+        if( spatial_pred > d + diff )
+        {
+            spatial_pred = d + diff;
+        }
+        else if( spatial_pred < d - diff )
+        {
+            spatial_pred = d - diff;
+        }
+        
+        dst[0] = spatial_pred;
+                        
+end_of_yadif_filter_pixel:
+        dst++;
+        cur++;
+        prev++;
+        next++;
+        prev2++;
+        next2++;
+    }
+}
+
+static void yadif_filter( uint8_t ** dst,
+                          int parity,
+                          int tff,
+                          hb_filter_private_t * pv )
+{
+    int i;
+    for( i = 0; i < 3; i++ )
+    {
+        int w = pv->width[i];
+        int h = pv->height[i];
+        int ref_stride = pv->ref_stride[i];
+
+        int y;
+        for( y = 0; y < h; y++ )
+        {
+            if( pv->mode == 3)
+            {
+                uint8_t *prev = &pv->ref[0][i][y*ref_stride];
+                uint8_t *cur  = &pv->ref[1][i][y*ref_stride];
+                uint8_t *next = &pv->ref[2][i][y*ref_stride];
+                uint8_t *dst2 = &dst[i][y*w];
+
+                blend_filter_line( dst2, cur, i, y, pv );
+            }
+            else if( (y ^ parity) &  1 )
+            {
+                uint8_t *prev = &pv->ref[0][i][y*ref_stride];
+                uint8_t *cur  = &pv->ref[1][i][y*ref_stride];
+                uint8_t *next = &pv->ref[2][i][y*ref_stride];
+                uint8_t *dst2 = &dst[i][y*w];
+
+                yadif_filter_line( dst2, prev, cur, next, i, parity ^ tff, y, pv );
+            }
+            else
+            {
+                memcpy( &dst[i][y*w],
+                        &pv->ref[1][i][y*ref_stride],
+                        w * sizeof(uint8_t) );              
+            }
+        }
+    }
+}
+
+static void mcdeint_filter( uint8_t ** dst,
+                            uint8_t ** src,
+                            int parity,
+                            hb_filter_private_t * pv )
+{
+    int x, y, i;
+    int out_size;
+
+#ifdef SUPPRESS_AV_LOG
+    /* TODO: temporarily change log level to suppress obnoxious debug output */
+    int loglevel = av_log_get_level();
+    av_log_set_level( AV_LOG_QUIET );
+#endif
+
+    for( i=0; i<3; i++ )
+    {
+        pv->mcdeint_frame->data[i] = src[i];
+        pv->mcdeint_frame->linesize[i] = pv->width[i];
+    }
+    pv->mcdeint_avctx_enc->me_cmp     = FF_CMP_SAD;
+    pv->mcdeint_avctx_enc->me_sub_cmp = FF_CMP_SAD;
+    pv->mcdeint_frame->quality        = pv->mcdeint_qp * FF_QP2LAMBDA;
+
+    out_size = avcodec_encode_video( pv->mcdeint_avctx_enc,
+                                     pv->mcdeint_outbuf,
+                                     pv->mcdeint_outbuf_size,
+                                     pv->mcdeint_frame );
+
+    pv->mcdeint_frame_dec = pv->mcdeint_avctx_enc->coded_frame;
+
+    for( i = 0; i < 3; i++ )
+    {
+        int w    = pv->width[i];
+        int h    = pv->height[i];
+        int fils = pv->mcdeint_frame_dec->linesize[i];
+        int srcs = pv->width[i];
+
+        for( y = 0; y < h; y++ )
+        {
+            if( (y ^ parity) & 1 )
+            {
+                for( x = 0; x < w; x++ )
+                {
+                    if( (x-2)+(y-1)*w >= 0 && (x+2)+(y+1)*w < w*h )
+                    {
+                        uint8_t * filp =
+                            &pv->mcdeint_frame_dec->data[i][x + y*fils];
+                        uint8_t * srcp = &src[i][x + y*srcs];
+
+                        int diff0 = filp[-fils] - srcp[-srcs];
+                        int diff1 = filp[+fils] - srcp[+srcs];
+
+                        int spatial_score =
+                              ABS(srcp[-srcs-1] - srcp[+srcs-1])
+                            + ABS(srcp[-srcs  ] - srcp[+srcs  ])
+                            + ABS(srcp[-srcs+1] - srcp[+srcs+1]) - 1;
+
+                        int temp = filp[0];
+
+#define MCDEINT_CHECK(j)\
+                        {   int score = ABS(srcp[-srcs-1+j] - srcp[+srcs-1-j])\
+                                      + ABS(srcp[-srcs  +j] - srcp[+srcs  -j])\
+                                      + ABS(srcp[-srcs+1+j] - srcp[+srcs+1-j]);\
+                            if( score < spatial_score ) {\
+                                spatial_score = score;\
+                                diff0 = filp[-fils+j] - srcp[-srcs+j];\
+                                diff1 = filp[+fils-j] - srcp[+srcs-j];
+
+                        MCDEINT_CHECK(-1) MCDEINT_CHECK(-2) }} }}
+                        MCDEINT_CHECK( 1) MCDEINT_CHECK( 2) }} }}
+
+                        if(diff0 + diff1 > 0)
+                        {
+                            temp -= (diff0 + diff1 -
+                                     ABS( ABS(diff0) - ABS(diff1) ) / 2) / 2;
+                        }
+                        else
+                        {
+                            temp -= (diff0 + diff1 +
+                                     ABS( ABS(diff0) - ABS(diff1) ) / 2) / 2;
+                        }
+
+                        filp[0] = dst[i][x + y*w] =
+                            temp > 255U ? ~(temp>>31) : temp;
+                    }
+                    else
+                    {
+                        dst[i][x + y*w] =
+                            pv->mcdeint_frame_dec->data[i][x + y*fils];
+                    }
+                }
+            }
+        }
+
+        for( y = 0; y < h; y++ )
+        {
+            if( !((y ^ parity) & 1) )
+            {
+                for( x = 0; x < w; x++ )
+                {
+                    pv->mcdeint_frame_dec->data[i][x + y*fils] =
+                        dst[i][x + y*w]= src[i][x + y*srcs];
+                }
+            }
+        }
+    }
+
+#ifdef SUPPRESS_AV_LOG
+    /* TODO: restore previous log level */
+    av_log_set_level(loglevel);
+#endif
+}
+
+hb_filter_private_t * hb_decomb_init( int pix_fmt,
+                                           int width,
+                                           int height,
+                                           char * settings )
+{
+    if( pix_fmt != PIX_FMT_YUV420P )
+    {
+        return 0;
+    }
+
+    hb_filter_private_t * pv = calloc( 1, sizeof(struct hb_filter_private_s) );
+
+    pv->pix_fmt = pix_fmt;
+
+    pv->width[0]  = width;
+    pv->height[0] = height;
+    pv->width[1]  = pv->width[2]  = width >> 1;
+    pv->height[1] = pv->height[2] = height >> 1;
+
+    int buf_size = 3 * width * height / 2;
+    pv->buf_out[0] = hb_buffer_init( buf_size );
+    pv->buf_out[1] = hb_buffer_init( buf_size );
+    pv->buf_settings = hb_buffer_init( 0 );
+
+    pv->deinterlaced_frames = 0;
+    pv->passed_frames = 0;
+    pv->color_equal = 10;
+    pv->color_diff = 15;
+    pv->threshold = 9;
+    pv->prog_equal = 10;
+    pv->prog_diff = 35;
+    pv->prog_threshold = 9;
+    
+    pv->yadif_ready    = 0;
+    pv->mode     = MODE_DEFAULT;
+    pv->parity   = PARITY_DEFAULT;
+
+    pv->mcdeint_mode   = MCDEINT_MODE_DEFAULT;
+    pv->mcdeint_qp     = MCDEINT_QP_DEFAULT;
+
+    if( settings )
+    {
+        sscanf( settings, "%d:%d:%d:%d:%d:%d:%d",
+                &pv->mode,
+                &pv->color_equal,
+                &pv->color_diff,
+                &pv->threshold,
+                &pv->prog_equal,
+                &pv->prog_diff,
+                &pv->prog_threshold );
+    }
+    
+    if( pv->mode == 2 || pv->mode == 5 )
+    {
+        pv->mcdeint_mode = 0;
+    }
+    
+    /* Allocate yadif specific buffers */
+    if( pv->mode > 0 )
+    {
+        int i, j;
+        for( i = 0; i < 3; i++ )
+        {
+            int is_chroma = !!i;
+            int w = ((width   + 31) & (~31))>>is_chroma;
+            int h = ((height+6+ 31) & (~31))>>is_chroma;
+
+            pv->ref_stride[i] = w;
+
+            for( j = 0; j < 3; j++ )
+            {
+                pv->ref[j][i] = malloc( w*h*sizeof(uint8_t) ) + 3*w;
+            }
+        }
+    }
+
+    /* Allocate mcdeint specific buffers */
+    if( pv->mcdeint_mode >= 0 )
+    {
+        avcodec_init();
+        avcodec_register_all();
+
+        AVCodec * enc = avcodec_find_encoder( CODEC_ID_SNOW );
+
+        int i;
+        for (i = 0; i < 3; i++ )
+        {
+            AVCodecContext * avctx_enc;
+
+            avctx_enc = pv->mcdeint_avctx_enc = avcodec_alloc_context();
+
+            avctx_enc->width                    = width;
+            avctx_enc->height                   = height;
+            avctx_enc->time_base                = (AVRational){1,25};  // meaningless
+            avctx_enc->gop_size                 = 300;
+            avctx_enc->max_b_frames             = 0;
+            avctx_enc->pix_fmt                  = PIX_FMT_YUV420P;
+            avctx_enc->flags                    = CODEC_FLAG_QSCALE | CODEC_FLAG_LOW_DELAY;
+            avctx_enc->strict_std_compliance    = FF_COMPLIANCE_EXPERIMENTAL;
+            avctx_enc->global_quality           = 1;
+            avctx_enc->flags2                   = CODEC_FLAG2_MEMC_ONLY;
+            avctx_enc->me_cmp                   = FF_CMP_SAD; //SSE;
+            avctx_enc->me_sub_cmp               = FF_CMP_SAD; //SSE;
+            avctx_enc->mb_cmp                   = FF_CMP_SSE;
+
+            switch( pv->mcdeint_mode )
+            {
+                case 3:
+                    avctx_enc->refs = 3;
+                case 2:
+                    avctx_enc->me_method = ME_UMH;
+                case 1:
+                    avctx_enc->flags |= CODEC_FLAG_4MV;
+                    avctx_enc->dia_size =2;
+                case 0:
+                    avctx_enc->flags |= CODEC_FLAG_QPEL;
+            }
+
+            avcodec_open(avctx_enc, enc);
+        }
+
+        pv->mcdeint_frame       = avcodec_alloc_frame();
+        pv->mcdeint_outbuf_size = width * height * 10;
+        pv->mcdeint_outbuf      = malloc( pv->mcdeint_outbuf_size );
+    }
+
+    return pv;
+}
+
+void hb_decomb_close( hb_filter_private_t * pv )
+{
+    if( !pv )
+    {
+        return;
+    }
+
+    hb_log("decomb: deinterlaced %i | unfiltered %i | total %i", pv->deinterlaced_frames, pv->passed_frames, pv->deinterlaced_frames + pv->passed_frames);
+
+    /* Cleanup frame buffers */
+    if( pv->buf_out[0] )
+    {
+        hb_buffer_close( &pv->buf_out[0] );
+    }
+    if( pv->buf_out[1] )
+    {
+        hb_buffer_close( &pv->buf_out[1] );
+    }
+    if (pv->buf_settings )
+    {
+        hb_buffer_close( &pv->buf_settings );
+    }
+
+    /* Cleanup yadif specific buffers */
+    if( pv->mode > 0 )
+    {
+        int i;
+        for( i = 0; i<3*3; i++ )
+        {
+            uint8_t **p = &pv->ref[i%3][i/3];
+            if (*p)
+            {
+                free( *p - 3*pv->ref_stride[i/3] );
+                *p = NULL;
+            }
+        }
+    }
+
+    /* Cleanup mcdeint specific buffers */
+    if( pv->mcdeint_mode >= 0 )
+    {
+        if( pv->mcdeint_avctx_enc )
+        {
+            avcodec_close( pv->mcdeint_avctx_enc );
+            av_freep( &pv->mcdeint_avctx_enc );
+        }
+        if( pv->mcdeint_outbuf )
+        {
+            free( pv->mcdeint_outbuf );
+        }
+    }
+
+    free( pv );
+}
+
+int hb_decomb_work( hb_buffer_t * buf_in,
+                         hb_buffer_t ** buf_out,
+                         int pix_fmt,
+                         int width,
+                         int height,
+                         hb_filter_private_t * pv )
+{
+    if( !pv ||
+        pix_fmt != pv->pix_fmt ||
+        width   != pv->width[0] ||
+        height  != pv->height[0] )
+    {
+        return FILTER_FAILED;
+    }
+
+    avpicture_fill( &pv->pic_in, buf_in->data,
+                    pix_fmt, width, height );
+
+    /* Use libavcodec deinterlace if mode == 0 */
+    if( pv->mode == 0 )
+    {
+        avpicture_fill( &pv->pic_out, pv->buf_out[0]->data,
+                        pix_fmt, width, height );
+
+        /* Check for combing on the input frame */
+        int interlaced =  hb_detect_comb(buf_in, width, height, pv->color_equal, pv->color_diff, pv->threshold, pv->prog_equal, pv->prog_diff, pv->prog_threshold);
+        
+        if(interlaced)
+        {
+            avpicture_deinterlace( &pv->pic_out, &pv->pic_in,
+                                   pix_fmt, width, height );
+
+            pv->deinterlaced_frames++;
+            //hb_log("Frame %i is combed (Progressive: %s )", pv->deinterlaced_frames + pv->passed_frames, (buf_in->flags & 16) ? "Y" : "N");
+            
+            hb_buffer_copy_settings( pv->buf_out[0], buf_in );
+            *buf_out = pv->buf_out[0];            
+        }
+        else
+        {
+            /* No combing detected, pass input frame through unmolested.*/
+            
+            pv->passed_frames++;
+            
+            hb_buffer_copy_settings( pv->buf_out[0], buf_in );
+            *buf_out = buf_in;
+            
+        }
+
+        return FILTER_OK;
+    }
+    
+    /* Determine if top-field first layout */
+    int tff;
+    if( pv->parity < 0 )
+    {
+        tff = !!(buf_in->flags & PIC_FLAG_TOP_FIELD_FIRST);
+    }
+    else
+    {
+        tff = (pv->parity & 1) ^ 1;
+    }
+
+    /* Store current frame in yadif cache */
+    store_ref( (const uint8_t**)pv->pic_in.data, pv );
+
+    /* Note down if the input frame is combed */
+    pv->comb = (pv->comb << 1) | hb_detect_comb(buf_in, width, height, pv->color_equal, pv->color_diff, pv->threshold, pv->prog_equal, pv->prog_diff, pv->prog_threshold);
+
+    /* If yadif is not ready, store another ref and return FILTER_DELAY */
+    if( pv->yadif_ready == 0 )
+    {
+        store_ref( (const uint8_t**)pv->pic_in.data, pv );
+
+        hb_buffer_copy_settings( pv->buf_settings, buf_in );
+
+        /* don't let 'work_loop' send a chapter mark upstream */
+        buf_in->new_chap  = 0;
+
+        pv->yadif_ready = 1;
+
+        return FILTER_DELAY;
+    }
+
+    /* yadif works one frame behind so if the previous frame
+     * had combing, deinterlace it otherwise just output it. */
+    if(  (pv->comb & 2 ) == 0 )
+    {
+        /* previous frame not interlaced - copy cached input frame to buf_out */
+        
+        pv->passed_frames++;
+        
+        avpicture_fill( &pv->pic_out,  pv->buf_out[0]->data, pix_fmt, width, height );
+        get_ref( (uint8_t**)pv->pic_out.data, pv, 1 );
+        *buf_out = pv->buf_out[0];
+    }
+    else
+    {
+        /* Perform yadif filtering */
+        
+        pv->deinterlaced_frames++;
+        int frame;
+        for( frame = 0; frame <= ( ( pv->mode == 2 || pv->mode == 5 )? 1 : 0 ) ; frame++ )
+        {
+            int parity = frame ^ tff ^ 1;
+
+            avpicture_fill( &pv->pic_out, pv->buf_out[!(frame^1)]->data,
+                            pix_fmt, width, height );
+
+            yadif_filter( pv->pic_out.data, parity, tff, pv );
+
+            if( pv->mcdeint_mode >= 0 )
+            {
+                /* Perform mcdeint filtering */
+                avpicture_fill( &pv->pic_in,  pv->buf_out[(frame^1)]->data,
+                                pix_fmt, width, height );
+
+                mcdeint_filter( pv->pic_in.data, pv->pic_out.data, parity, pv );
+            }
+
+            *buf_out = pv->buf_out[!(frame^1)];
+        }
+    }
+
+    /* Copy buffered settings to output buffer settings */
+    hb_buffer_copy_settings( *buf_out, pv->buf_settings );
+
+    /* Replace buffered settings with input buffer settings */
+    hb_buffer_copy_settings( pv->buf_settings, buf_in );
+
+    /* don't let 'work_loop' send a chapter mark upstream */
+    buf_in->new_chap  = 0;
+
+    return FILTER_OK;
+}
Index: libhb/common.h
===================================================================
--- libhb/common.h	(revision 1469)
+++ libhb/common.h	(working copy)
@@ -574,6 +574,7 @@
 extern hb_filter_object_t hb_filter_deinterlace;
 extern hb_filter_object_t hb_filter_deblock;
 extern hb_filter_object_t hb_filter_denoise;
+extern hb_filter_object_t hb_filter_decomb;
 
 typedef void hb_error_handler_t( const char *errmsg );
 
Index: macosx/HandBrake.xcodeproj/project.pbxproj
===================================================================
--- macosx/HandBrake.xcodeproj/project.pbxproj	(revision 1469)
+++ macosx/HandBrake.xcodeproj/project.pbxproj	(working copy)
@@ -89,6 +89,8 @@
 		593034EC0BBA39A100172349 /* ChapterTitles.m in Sources */ = {isa = PBXBuildFile; fileRef = 593034EA0BBA39A100172349 /* ChapterTitles.m */; };
 		59CBD2370BBB44DA004A3BE3 /* parsecsv.c in Sources */ = {isa = PBXBuildFile; fileRef = 59CBD2360BBB44DA004A3BE3 /* parsecsv.c */; };
 		59CBD2650BBB4D1B004A3BE3 /* ChapterTitles.m in Sources */ = {isa = PBXBuildFile; fileRef = 593034EA0BBA39A100172349 /* ChapterTitles.m */; };
+		7497010F0DC281BB009200D8 /* decomb.c in Sources */ = {isa = PBXBuildFile; fileRef = 7497010E0DC281BB009200D8 /* decomb.c */; };
+		749701100DC281BB009200D8 /* decomb.c in Sources */ = {isa = PBXBuildFile; fileRef = 7497010E0DC281BB009200D8 /* decomb.c */; };
 		A22C85EC0D05D35000C10E36 /* HBPresets.h in Headers */ = {isa = PBXBuildFile; fileRef = A22C85EA0D05D35000C10E36 /* HBPresets.h */; };
 		A22C85ED0D05D35100C10E36 /* HBPresets.m in Sources */ = {isa = PBXBuildFile; fileRef = A22C85EB0D05D35000C10E36 /* HBPresets.m */; };
 		A25289E60D87A27D00461D5B /* enctheora.c in Sources */ = {isa = PBXBuildFile; fileRef = A25289E50D87A27D00461D5B /* enctheora.c */; };
@@ -257,6 +259,7 @@
 		593034E90BBA39A100172349 /* ChapterTitles.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = ChapterTitles.h; sourceTree = "<group>"; };
 		593034EA0BBA39A100172349 /* ChapterTitles.m */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.objc; path = ChapterTitles.m; sourceTree = "<group>"; };
 		59CBD2360BBB44DA004A3BE3 /* parsecsv.c */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.c; name = parsecsv.c; path = ../test/parsecsv.c; sourceTree = SOURCE_ROOT; };
+		7497010E0DC281BB009200D8 /* decomb.c */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.c; name = decomb.c; path = ../libhb/decomb.c; sourceTree = SOURCE_ROOT; };
 		A22C85EA0D05D35000C10E36 /* HBPresets.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = HBPresets.h; sourceTree = "<group>"; };
 		A22C85EB0D05D35000C10E36 /* HBPresets.m */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.objc; path = HBPresets.m; sourceTree = "<group>"; };
 		A25289E50D87A27D00461D5B /* enctheora.c */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.c; name = enctheora.c; path = ../libhb/enctheora.c; sourceTree = SOURCE_ROOT; };
@@ -434,6 +437,7 @@
 			isa = PBXGroup;
 			children = (
 				A25289E50D87A27D00461D5B /* enctheora.c */,
+				7497010E0DC281BB009200D8 /* decomb.c */,
 				B48359A70C82960500E04440 /* lang.c */,
 				EAA526920C3B25D200944FF2 /* stream.c */,
 				0DFA5C7E0B8DD3B60020BC09 /* declpcm.c */,
@@ -785,6 +789,7 @@
 				FC8519550C59A02C0073812C /* deinterlace.c in Sources */,
 				FC8519560C59A02C0073812C /* deblock.c in Sources */,
 				FC8519570C59A02C0073812C /* detelecine.c in Sources */,
+				749701100DC281BB009200D8 /* decomb.c in Sources */,
 			);
 			runOnlyForDeploymentPostprocessing = 0;
 		};
@@ -830,6 +835,7 @@
 				FC8519530C59A02C0073812C /* detelecine.c in Sources */,
 				B48359A80C82960500E04440 /* lang.c in Sources */,
 				A25289E60D87A27D00461D5B /* enctheora.c in Sources */,
+				7497010F0DC281BB009200D8 /* decomb.c in Sources */,
 			);
 			runOnlyForDeploymentPostprocessing = 0;
 		};
dynaflash
Veteran User
Posts: 3820
Joined: Thu Nov 02, 2006 8:19 pm

Re: Combing Detection

Post by dynaflash »

Here are a couple patches for macgui implementation of jbrjakes decomb. I am using a NSPopUp even though right now it is pretty much an On/Off type thing. The popup just has "None" and "Default" selections. Default uses jbrjakes suggested parameters.

Code: Select all

4:10:15:9:10:35:9
For now since we are testing I wanted the flexibility to add more settings if need be. If jbrjake determines that this one setting should work fine, then we can always go to a simple checkbox. Also note, these patches do not implement decomb in the preset code, so presets will *not* store your decomb settings at all. Until its finalized and goes into the svn I see no reason to save it into the users presets plist file.

This should be used if you have already applied jbrjakes patch above and just want to enable the macgui part: http://handbrake.pastebin.ca/1027096

This one is a complete patch which is both jbrjakes patch and the macgui patch combined: http://handbrake.pastebin.ca/1027100

Incidentally, in just a quick test of a short clip from an eyetv file, I got this:

Code: Select all

[08:59:28] decomb: deinterlaced 687 | unfiltered 1047 | total 1734
dynaflash
Veteran User
Posts: 3820
Joined: Thu Nov 02, 2006 8:19 pm

Re: Combing Detection

Post by dynaflash »

Phantom Menace using vfr + decomb:

Code: Select all

[13:17:25] render: lost time: 373892 (112 frames)
[13:17:25] render: gained time: 373892 (428 frames) (0 not accounted for)
[13:17:25] render: average dropped frame duration: 3338
[13:26:23] decomb: deinterlaced 90 | unfiltered 195722 | total 195812
At least in this case, clearly safe for progressive use.
sdm
Bright Spark User
Posts: 194
Joined: Mon Feb 19, 2007 4:53 pm

Re: Combing Detection

Post by sdm »

Thanks jbrjake for continued work!

I've done a little testing on some problem sources since applying the patch.
The default comb-detect has done a good job on some of those transitions we talked about before!

Here are some examples where comb-detect and deinterlace could stand a little improvement if possible.

Image
In this animated example, you can see, using an older comb-detect with a colour_diff value = 35 (weaker than old default of 30), the frame is correctly identified as not-combed and not filtered.
AND the new comb-detect with it's default values. The new comb-detect is incorrectly identifying this frame combed and deinterlacing it. I believe it is using Mode 4 (uses yadif, mode 0, with the new 4-point interpolation) to deinterlace, but its definitely still suffering from jaggies resulting from interpolation on diagonals you discussed above.

Another problem is when only small areas of a frame are interlaced.
Image
Here Wyatt's hands are still combed. In this source there are many frames with small portions still combed after comb-detect is done.

Image
Here is another example of the same thing.

I can provide you these sources (or portions of) if you'd like.

Dynaflash thanks for putting this stuff in the GUI.
Here are a couple patches for macgui implementation of jbrjakes decomb. I am using a NSPopUp even though right now it is pretty much an On/Off type thing. The popup just has "None" and "Default" selections. Default uses jbrjakes suggested parameters.
Code:
4:10:15:9:10:35:9

For now since we are testing I wanted the flexibility to add more settings if need be. If jbrjake determines that this one setting should work fine, then we can always go to a simple checkbox. Also note, these patches do not implement decomb in the preset code, so presets will *not* store your decomb settings at all. Until its finalized and goes into the svn I see no reason to save it into the users presets plist file.
I agree it would be nice for one-parameter-setting-fits-all, but I sure would like to be able to tweak these settings in the GUI for now - for testing purposes.

Again, thanks to you both.
--sdm.

edited for clarity.
jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Re: Combing Detection

Post by jbrjake »

sdm wrote:Here are some examples where comb-detect and deinterlace could stand a little improvement if possible.
sdm, I'm not sure what I can do at this point. The comb detection is still too unrefined to do what's needed for all these situations.

It's looking at a whole frame at a time. On each plane, any time it finds a pixel with a value:
- Within 10 (the equal default) of the pixel 2 lines below it...
- That's also not within 15 (the difference default) of the pixel 1 line below it....
It considers that pixel to be combed. Then, it adds up how many pixels are combed, and divides by the resolution (720x480 or 360x240 for NTSC DVD, depending on the plane). This gives a percentage value of how combed the plane is.

After it's done this for each plane, it averages those percentages together. The average counts the luma plane twice, and only gives half weight to the two chroma planes, since they're so small (maybe this is a stupid decision on my part?). If this final average is greater than the 9/1000, (the threshold default is 9), the whole frame is considered combed, and it's deinterlaced.

The problem is that it's working at too big a level (a whole frame) and also that it's working at too small a level (a single pixel). If I make the default parameters strong enough to catch Wyatt's hands, I can't make them weak enough to ignore that wire snaking across the floor. Unless, well...is that music video hard telecined? If so, it'll be fixed whenever those render changes happen, since then it'll be progressive and use the weaker settings.

The problem with running on a whole frame at once is a lot of the time, there are only tiny portions of a frame that need to be deinterlaced. It's trying to hit a gnat with a sledgehammer.

The problem with running on each pixel individually, is that it causes misfires due to noise. I've done tests with a filter to overwrite the frame with a visualization of the comb detection. It marked every combed pixel as white and the rest of the scene black. Sure, it picked up the interlacing, but it also got a *lot* of noise. That's why the threshold is set so high, at 9/1000. Set it much lower, and it starts misfiring due to random noise in the image.

The larger the equals and smaller the difference values get, the more sensitive it becomes. But the number of misfires goes up too. So you have to raise the threshold to avoid false positives, and the higher the threshold, the more of a frame needs to show combing for it to be deinterlaced.

So the next step in this will be to go all Goldilocks. If whole frames are too big and individual pixels are so small, there's gotta be a compromise that's just right. Say, 16x16 blocks (yes, rhester, I'm thinking those could be useful in the future for motion vectors too ;P ). But it'll mean reorganizing line-based yadif a lot, as well as the decomber. I'd like to get the current decombing into the svn before that, though, just because something is better than nothing, and what's there right now is a "good enough" solution for a lot of cases.

Anyway, I've asked dynaflash to look into adding a pref to customize the decomb settings, when he's got the time. So once that's in place, you'll be able to try tweaks. Please tell me what you find -- I know the current settings can still be improved.
dynaflash
Veteran User
Posts: 3820
Joined: Thu Nov 02, 2006 8:19 pm

Re: Combing Detection

Post by dynaflash »

Will be workin' on it tonight
sdm
Bright Spark User
Posts: 194
Joined: Mon Feb 19, 2007 4:53 pm

Re: Combing Detection

Post by sdm »

jbrjake wrote: sdm, I'm not sure what I can do at this point. The comb detection is still too unrefined to do what's needed for all these situations.
Yah, I've been thinking - given the effort you've been putting in - that it must be incredibly difficult to find the right algorithm suitable for the wide range of sources out there.

I'm curious if instead of checking the pixels below and two below, if it might be worth checking the pixels above and two above as well, and somehow give that a weight and use that, combined with the pixels below, to determine if the pixel is combed.
The problem with running on a whole frame at once is a lot of the time, there are only tiny portions of a frame that need to be deinterlaced. It's trying to hit a gnat with a sledgehammer.

The problem with running on each pixel individually, is that it causes misfires due to noise. I've done tests with a filter to overwrite the frame with a visualization of the comb detection. It marked every combed pixel as white and the rest of the scene black. Sure, it picked up the interlacing, but it also got a *lot* of noise. That's why the threshold is set so high, at 9/1000. Set it much lower, and it starts misfiring due to random noise in the image.

The larger the equals and smaller the difference values get, the more sensitive it becomes. But the number of misfires goes up too. So you have to raise the threshold to avoid false positives, and the higher the threshold, the more of a frame needs to show combing for it to be deinterlaced.

So the next step in this will be to go all Goldilocks. If whole frames are too big and individual pixels are so small, there's gotta be a compromise that's just right. Say, 16x16 blocks (yes, rhester, I'm thinking those could be useful in the future for motion vectors too ;P ). But it'll mean reorganizing line-based yadif a lot, as well as the decomber. I'd like to get the current decombing into the svn before that, though, just because something is better than nothing, and what's there right now is a "good enough" solution for a lot of cases.
Doing the detection/deinterlacing on 16x16 blocks sounds like a great solution - I suspect this would mean a performance hit (which doesn't matter to me) as well as implications I can't imagine yet, not to mention an the effort to get it coded.
well...is that music video hard telecined? If so, it'll be fixed whenever those render changes happen, since then it'll be progressive and use the weaker settings
Yes the video is hard telecined. Are you reffering to this?
Here's what I'd *like* to do at a later date: treat ivtc'd sequences the same as progressive, by using the weaker progressive parameters. Doing this means setting render.c to mark detelecined frames as progressive as well. This makes the comb detection more flexible. It will deinterlace most every 30fps video frame, and only deinterlace parts of progressive or ivtc'd material if they show more obvious combing. But doing this is a pain. See, decomb runs in the regular filter chain, and detelecine does too. But VFR only extends the duration of detelecined frames after a 3-frame delay buffer. This is because when detelecine drops a frame, it's going to be two frames into a telecine sequence. Render.c needs to go back and rewrite the durations for those earlier frames, to make up the lost time. It's only when the duration is rewritten -- after that buffer -- that those frames can be flagged as progressive...and that's long after decomb has run. So the regular filter loop needs to be set not to run decomb. Then I'd add a new filter loop at the bottom of render, after the delay queue, solely to run decomb. At this point I realized that wouldn't work. Why? Because decomb needs to run before the frame has subtitles applied or is scaled or cropped. That means this requires rearranging render.c. VFR will do its delay queue thing before rendering of the frames has finished. It'll caches and extends before subs or scaling or cropping are applied. I haven't made these changes to render.c. I can't get it to work right, and will need the aid of a Real Programmer. But when they are made at some future date, decomb will benefit immediately.
I do know that the in this panning scene, the deinterlace filter was being applied to the progressive frames and frames that were inverse telecined (and therefore progressive too) alike. Make sense? Maybe I'm misunderstanding ... is comb detect being run on the progressive frames in a hard telecined sequence? Or maybe you mean pure progressive content.

Anyways, I'm glad to hear there are still improvements you intend to implement! I have a few hundred more music videos that can benefit.
I'm also glad you're interested in my findings.

good luck!
--sdm.
jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Re: Combing Detection

Post by jbrjake »

sdm wrote:Yah, I've been thinking - given the effort you've been putting in - that it must be incredibly difficult to find the right algorithm suitable for the wide range of sources out there.
Well, while I agree it's incredibly difficult, don't base that conclusion on the amount of effort I've put into it. I'm a total n00b at this stuff. I've never done programming involving bitmap images or digital filters before. So the slow-going is more than in part due to it being a learning experience from beginning to end. Berrinam's workflow for AviSynth on Windows, using tritical's comb detection filter from TIVTC, is more sophisticated. But porting that is beyond me, so I putter with this.
I'm curious if instead of checking the pixels below and two below, if it might be worth checking the pixels above and two above as well, and somehow give that a weight and use that, combined with the pixels below, to determine if the pixel is combed.
I've experimented with that, but unfortunately by reaching out further like that, it increases the chances of getting a wrong result, and misses fine detail.
Doing the detection/deinterlacing on 16x16 blocks sounds like a great solution - I suspect this would mean a performance hit (which doesn't matter to me) as well as implications I can't imagine yet, not to mention an the effort to get it coded.
I got bored and already started working on it. And yes, it does have a significant performance hit :/
Yes the video is hard telecined. Are you reffering to this?
<snip quote>
I do know that the in this panning scene, the deinterlace filter was being applied to the progressive frames and frames that were inverse telecined (and therefore progressive too) alike. Make sense? Maybe I'm misunderstanding ... is comb detect being run on the progressive frames in a hard telecined sequence? Or maybe you mean pure progressive content.
Okay, good.

What's happening right now is that hard telecined frames are being analyzed with the equal and difference parameters for interlaced frames, since they show up as video (no progressive flag). Once render gets re-arranged, frames in hard telecined sequences will show up as film (with a progressive flag) and get analyzed with the weaker progressive parameters.
dynaflash
Veteran User
Posts: 3820
Joined: Thu Nov 02, 2006 8:19 pm

Re: Combing Detection

Post by dynaflash »

Okay, here is a patch which is the lastest jbrjake patch plus adds decomb to the macgui along with a field in Preferences -> Picture where you can instert your own settings. In the Picture panel under Decomb you will now have None, Default (same as before), and Custom. Custom will use the string you have input in your preferences.

The Main window now also reads out the decomb settings instead of just "On". Hope this helps with testing. NOTE: I do *not* see this as any kind of final implementation but rather a very crude hack to allow custom values.

Remember, this is the *whole* patch to apply against a fresh checkout. Tested against svn rev 1469

http://handbrake.pastebin.ca/1031173
eddyg
Veteran User
Posts: 798
Joined: Mon Apr 23, 2007 3:34 am

Re: Combing Detection

Post by eddyg »

If splitting the comb detection then give 16 line jobs to separate threads.

Cheers, Ed
jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Re: Combing Detection

Post by jbrjake »

Here's an updated patch. I finally got around to fixing the blend mode (3) so it doesn't mess up the top and bottom lines by reading past the frame size. Also includes some code clean-up, and a commented-out "mode 7" that will eventually be the macroblock level stuff.

This integrates dynaflash's GUI work, including the custom parameter box in the preferences.

At dynaflash and eddyg's urging, I'm probably going to check this in presently, even though it's still rather incomplete.

http://handbrake.pastebin.ca/raw/1033160
(no local code block copy, since the xib changes push it wellllllllllllllllllll beyond the forum's max character count)
eddyg wrote:If splitting the comb detection then give 16 line jobs to separate threads.
Geez, like it's not enough that I'm being forced to learn math, you want me to teach myself posix threading too? What is this, an open source project or a computer science class? ;P

Really though, I don't know where to begin with that. I know we have functions to wrap pthreads, but I've never touched them before, and I don't know a thing about locks, mutexes, etc. I've skimmed a few tutorials but they seem rather light on simple, real-world example code.

If something about this is going to be threaded for this deinterlacing stuff, though, it seems to me it should be yadif, since that's the biggest timesink (excepting mcdeint's use of snow). It processes pv->height[plane] / 2 lines per frame. Each one is processed individually and they're all done sequentially, even though information from each filtered line has no impact on any other filtered line. So it should be easy enough to slice the frame horizontally into CPU_COUNT chunks and process them on different cores.
jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Re: Combing Detection

Post by jbrjake »

Okay, I bit the bullet and checked it into the SVN:
http://trac.handbrake.fr/changeset/1471

Somehow it lost the comment I tried to leave in the log...
1471's missing comment wrote:"Got hair on a girl that flows to her bones / And a comb in her pocket if the winds get blown." —"No. 13 Baby," The Pixies

Adds an experimental new video filter called decomb, which selectively deinterlaces frames (with ffmpeg, yadif/mcdeint, or field blending) only when they have detectable interlacing artifacts (aka combing or mouse teeth).
Last edited by jbrjake on Thu May 29, 2008 5:30 pm, edited 1 time in total.
Reason: added log message
dynaflash
Veteran User
Posts: 3820
Joined: Thu Nov 02, 2006 8:19 pm

Re: Combing Detection

Post by dynaflash »

Bear in mind that currently the decomb setting is *not* used in any preset code, so your presets will not affect decomb.
dynaflash
Veteran User
Posts: 3820
Joined: Thu Nov 02, 2006 8:19 pm

Re: Combing Detection

Post by dynaflash »

Okay, so as of http://trac.handbrake.fr/changeset/1472 decomb can be saved in your presets along with the other picture filters. HOWEVER, only "None" and "Default" as the current thinking is that there will not be a "Custom" setting (yeah, right like we never change our minds). So if you save a preset with Custom decomb, when you click on the preset it will actually set it to "Default" upon which you would have to manually reset "Custom" in the gui. Confusing ... yes. Convoluted ... sort of. Honest, there is a method to this madness. I think.
Nonsanity
Moderator
Posts: 46
Joined: Mon Feb 12, 2007 4:34 am

Re: Combing Detection

Post by Nonsanity »

If I've followed this thread right, the code is testing every other line of a frame one pixel at a time, scoring each pixel as to "how combed" it is. Currently, the sum total of the scores for the whole frame is checked against a threshold value to tell if deinterlacing is needed for that frame.

Seems to me that when pixels that score high for possible combing are close to each other, they should be weighted more. The 16x16 blocks would help, but even just watching for runs of high scorers on a line would make a big improvement. Examples like the cord above should be ignored because there isn't a huge difference between one frame and the next.

I'm being rushed out so I haven't read over this to make sure it makes sense. I'll be back later. :)
jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Around and around we go...

Post by jbrjake »

The macroblock stuff I was looking at just isn't going to work. The results are really ugly. Even 8x8 blocks are big enough that the chunks that are false negatives are quite noticeable. The render rearrangement for VFR is going to be a lot tougher than I imagined. Eddy's informed me that we can't cache those early buffers because libhb doesn't "own" them. So it requires memcpying each frame, and that gets awful slow and complicated.

Of course, right after I checked comb detection in, I discovered a better algorithm :/

As I've mentioned a few times in this thread, nothing I'm doing here holds a candle to the sophisticated methods employed by AviSynth filters. In particular, they have comb detection based on an IsCombed() routine from the FieldDeinterlace filter. I'd looked at the code a number of times, but never could figure it out, since it's so AviSynth-centric.

Yesterday, browsing the doom9 archives, I found a brilliant post by Tritical explaining how his IsCombedT() temporal comb detector works, and then another post by him on Don Graft's forum, doing the same thing for the spatial comb detector in FieldDeinterlace.

http://forum.doom9.org/showthread.php?p ... post477843
http://neuron2.net/board/viewtopic.php? ... aacee#4262

However, Tritical explains how this stuff *should* be done. I'd played with some of this stuff around a month ago, but gave up since I didn't know how to set the metrics. Basically, you have an extra buffer that acts as a mask. Any combed pixel on it is 255, any uncombed pixel is 0. Then, you iterate through that mask in 4x8 blocks -- this is similar to what NonSanity suggested. If any 4x8 section has > THRESHOLD pixels with values of 255, the whole frame is considered combed and sent to deinterlacing. (FieldDeinterlace can then use a similar map to decide which pixels to deinterlace and how, something I'll come back to at a later date.)

I've mixed in stuff from his temporal decomb. By applying his motion check first (comparing the current column of 5 pixels against the same column in the previous and next frames), it will never detect combing in static parts of the frame. By applying his extra spatial check, it won't fire when differences between lines are due to a steady spatial gradient shifting the pixel values in one direction from line 2 to line 4. This stuff also significantly improves detection of interlaced scene cuts.

Right now the motion threshold is hard set at a shift of 6 pixel values to the left or right, the IsCombedT default. The threshold for the 4x8 window is 15. If >= 15 pixels in any 4x8 box of the combing mask is combed, the whole frame is. This is the FieldDeinterlace default.

And now that I've ported IsCombed, what do I find? That I was wrong to put it on a pedestal. While it's better than Transcode's 32detect, no doubt, it's still not that great. I get noticeable false positives on anime still. But it is progress...

I've prototyped three different comb detectors. The uncommented one is the one we use currently in the svn -- if the pixel is more like the pixel 2 below than the pixel 1 below, smoothed out by the color_equal and color_diff thresholds, then it's combed.

Then there's the IsCombed method -- multiplying the difference between the current pixel and the pixel 1 above, and the current pixel and the pixel 1 below. This is similar to what sdm was suggesting. If it's greater than a threshold (IsCombed defaults to 20^2), the pixel is combed.

Then there's tritical's method from IsCombedT, bob+blur. He applies a FIR filter before doing the actual spatial combing score.

The current pixel is in line 3, and is filtered like so:
Line 1: 1
Line 2: 0
Line 3: 4
Line 4: 0
Line 5: 1

It's compared against this filter for the alternate field:
Line 1: 0
Line 2: 3
Line 3: 0
Line 4: 3
Line 5: 0

This reinforces similarities between nearby lines associated with the same field. This makes it more noise resistant, and tritical claims it'll get fewer false positives.

I'm sticking with 32detect's for now because it has those two thresholds to tweak, making it a little more configurable than the others.

Oh yeah, and right now I'm only running on luma. This is because the "right" way to handle chroma seems to be different than what we're doing. Basically, if any chroma pixel shows combing, then on the combing mask, any of the luma pixels affected by that chroma value should get set to 255. (The chroma planes are half rez in YUV420. For example, 360*240 chroma planes versus 720*480 luma. So each chroma pixel affects a whole block of lumas, not just a single one.) I'm not sure how to do this masking efficiently, yet.

http://handbrake.pastebin.ca/1035209

Code: Select all

Index: libhb/decomb.c
===================================================================
--- libhb/decomb.c	(revision 1474)
+++ libhb/decomb.c	(working copy)
@@ -230,6 +230,205 @@
     }
 }
 
+int tritical_detect_comb( hb_filter_private_t * pv )
+{
+    int x, y, i, j, k;
+    
+    /* Motion threshold */
+    int mthresh = 6;
+    /* Spatial threshold */
+    int athresh = 15;
+    /* Block mask threshold -- The number of pixels
+       in a 4x8 block of the mask that need to show
+       combing for the whole frame to be seen as such. */
+    int threshold = 15;
+    
+    int color_equal = 10;
+    int color_diff = 15;
+    
+    /* Make a buffer to store a comb mask. */
+    uint8_t        * mask[3];
+    for( i = 0; i < 3; i++ )
+    {
+        int is_chroma = !!i;
+        int w = ((pv->width[0]   + 31) & (~31))>>is_chroma;
+        int h = ((pv->height[0]+6+ 31) & (~31))>>is_chroma;
+
+        mask[i] = malloc( w*h*sizeof(uint8_t) ) + 3*w;
+    }
+    
+    /* One pas for Y, one pass for Cb, one pass for Cr */    
+    for( k = 0; k < 1; k++ )
+    {
+        int ref_stride = pv->ref_stride[k];
+        int width = pv->width[k];
+        int height = pv->height[k];
+
+        for( y = 2; y < (height-2); y++ )
+        {
+            /* These are just to make the buffer locations easier to read. */
+            int back_2 = y - 2;
+            int back_1 = y - 1;
+            int forward_1 = y + 1;
+            int forward_2 = y + 2;
+            
+            /* We need to examine a column of 5 pixels
+               in the prev, cur, and next frames.      */
+            uint8_t frame_a[5];
+            uint8_t frame_b[5];
+            uint8_t frame_c[5];
+            
+            for( x = 0; x < width; x++ )
+            {
+                /* Fill up the frame arrays with the current pixel values.*/
+                frame_a[0] = pv->ref[0][k][back_2*ref_stride + x];
+                frame_a[1] = pv->ref[0][k][back_1*ref_stride + x];
+                frame_a[2] = pv->ref[0][k][y*ref_stride + x];
+                frame_a[3] = pv->ref[0][k][forward_1*ref_stride + x];
+                frame_a[4] = pv->ref[0][k][forward_2*ref_stride + x];
+                frame_b[0] = pv->ref[1][k][back_2*ref_stride + x];
+                frame_b[1] = pv->ref[1][k][back_1*ref_stride + x];
+                frame_b[2] = pv->ref[1][k][y*ref_stride + x];
+                frame_b[3] = pv->ref[1][k][forward_1*ref_stride + x];
+                frame_b[4] = pv->ref[1][k][forward_2*ref_stride + x];
+                frame_c[0] = pv->ref[2][k][back_2*ref_stride + x];
+                frame_c[1] = pv->ref[2][k][back_1*ref_stride + x];
+                frame_c[2] = pv->ref[2][k][y*ref_stride + x];
+                frame_c[3] = pv->ref[2][k][forward_1*ref_stride + x];
+                frame_c[4] = pv->ref[2][k][forward_2*ref_stride + x];
+                
+                /* Make sure there's sufficient motion between frame t-1 to frame t+1. */
+                int motion = 0;
+                if( abs(frame_a[2] - frame_b[2] ) > mthresh && abs(frame_b[1] - frame_c[1] ) > mthresh && abs(frame_b[3] - frame_c[3] ) > mthresh )
+                    motion++;
+                if( abs(frame_c[2] - frame_b[2] ) > mthresh && abs(frame_a[1] - frame_b[1] ) > mthresh && abs(frame_a[3] - frame_b[3] ) > mthresh )
+                    motion++;
+
+#if 0
+                /* Visualizes the pixel values being examined. */
+                hb_log("\n PLANE: %i X: %i Y: %i \n %i %i %i \n %i %i %i \n %i %i %i \n %i %i %i \n %i %i %i", k, x, y,
+                frame_a[0], frame_b[0], frame_c[0],
+                frame_a[1], frame_b[1], frame_c[1],
+                frame_a[2], frame_b[2], frame_c[2],
+                frame_a[3], frame_b[3], frame_c[3],
+                frame_a[4], frame_b[4], frame_c[4] );
+#endif
+                if( motion )
+                {
+                    /* There is movement in this pixel location. */
+                    if( ( frame_b[1] - frame_b[2] ) * ( frame_b[3] - frame_b[2] ) > 0 )
+                    {
+                        /* The pixel above and below are different,
+                           and are so in the same "direction" too.
+                           That means it's time for the spatial check. */
+                        
+                        /* Simple 32detect style comb detection */
+                        if( ( abs( frame_b[2] - frame_b[4] ) < color_equal  ) &&
+                            ( abs( frame_b[2] - frame_b[3] ) > color_diff ) )
+                        {
+                            mask[k][y*ref_stride + x] = 255;
+                        }
+                        else
+                        {
+                            mask[k][y*ref_stride + x] = 0;
+                        }
+                        
+#if 0
+                        /* The combing check is done on a bob+blur convolution. */
+                        int combing = abs(  frame_b[0] + (4 * frame_b[2] ) + frame_b[4] - (3 * ( frame_b[1] + frame_b[3] ) ) );
+
+                        /* If the frame is sufficiently combed,
+                           then mark it down on the mask as 255. */
+                       if( combing > 6*athresh )
+                            mask[k][y*ref_stride + x] = 255; 
+                        else
+                            mask[k][y*ref_stride + x] = 0;
+#endif
+
+#if 0
+                        /* This, for comparison, is what IsCombed uses.
+                           It's simpler, but more noise senstive.      */
+                           int combing = (frame_b[1] - frame_b[2]) * (frame_b[3] - frame_b[2]);
+                           if( combing > 20 )
+                               mask[k][y*ref_stride + x] = 255; 
+                           else
+                               mask[k][y*ref_stride + x] = 0;
+#endif
+                    }
+                    else
+                    {
+                        mask[k][y*ref_stride + x] = 0;
+                    }
+                }
+                else
+                {
+                    mask[k][y*ref_stride + x] = 0;
+                }
+            }
+        }
+    }
+    
+    /* Go through the mask in 4x8 blocks. If any of these windows
+       have threshold or more combed pixels, consider the whole
+       frame to be combed and send it on to be deinterlaced.     */
+    int block_score = 0;
+    for(k = 0; k < 1; k++ )
+    {
+        int ref_stride = pv->ref_stride[k];        
+        for( y = 0; y < ( pv->height[k] - 8 ); y = y + 8 )
+        {
+            for( x = 0; x < ( pv->width[k] - 4 ); x = x + 4 )
+            {
+                int block_x;
+                int block_y;
+                block_score = 0;
+                for(block_y = 0; block_y < 8; block_y++ )
+                {
+                    for(block_x = 0; block_x < 4; block_x++ )
+                    {
+                        int mask_y = y + block_y;
+                        int mask_x = x + block_x;
+                        
+                        if( mask[k][mask_y*ref_stride+mask_x] == 255 )
+                        {
+                            block_score++;
+                        }
+                    }
+                }
+                
+                if( block_score >= threshold )
+                {
+                    /* Consider this frame to be combed.
+                       Clear the mask buffer and return 1. */
+                    for( i = 0; i<3*3; i++ )
+                    {
+                        uint8_t **p = &mask[i/3];
+                        if (*p)
+                        {
+                            free( *p - 3*pv->ref_stride[i/3] );
+                            *p = NULL;
+                        }
+                    }
+                    return 1;
+                }
+            }
+        } 
+    }
+    
+    /* Consider this frame to be uncombed.
+       Clear the mask buffer and return 0. */
+    for( i = 0; i<3*3; i++ )
+    {
+        uint8_t **p = &mask[i/3];
+        if (*p)
+        {
+            free( *p - 3*pv->ref_stride[i/3] );
+            *p = NULL;
+        }
+    }
+    return 0;
+}
+
 static void yadif_filter_line( uint8_t *dst,
                                uint8_t *prev,
                                uint8_t *cur,
@@ -376,7 +575,17 @@
                           int tff,
                           hb_filter_private_t * pv )
 {
-
+    
+    int is_combed = tritical_detect_comb( pv );
+    
+    if(is_combed)
+    {
+        pv->deinterlaced_frames++;
+    }
+    else
+    {
+        pv->passed_frames++;
+    }
 #if 0
     /* Buggy, experimental code for macroblock-by-macroblock decombing.*/
     if( pv->mode == 7 )
@@ -499,7 +708,7 @@
 
                 blend_filter_line( dst2, cur, i, y, pv );
             }
-            else if( (y ^ parity) &  1 )
+            else if( (y ^ parity) &  1 && is_combed )
             {
                 uint8_t *prev = &pv->ref[0][i][y*ref_stride];
                 uint8_t *cur  = &pv->ref[1][i][y*ref_stride];
@@ -850,14 +1059,14 @@
                         pix_fmt, width, height );
 
         /* Check for combing on the input frame */
-        int interlaced =  hb_detect_comb(buf_in, width, height, pv->color_equal, pv->color_diff, pv->threshold, pv->prog_equal, pv->prog_diff, pv->prog_threshold);
+                    int interlaced = 1; // hb_detect_comb(buf_in, width, height, pv->color_equal, pv->color_diff, pv->threshold, pv->prog_equal, pv->prog_diff, pv->prog_threshold);
         
         if(interlaced)
         {
             avpicture_deinterlace( &pv->pic_out, &pv->pic_in,
                                    pix_fmt, width, height );
 
-            pv->deinterlaced_frames++;
+//            pv->deinterlaced_frames++;
             //hb_log("Frame %i is combed (Progressive: %s )", pv->deinterlaced_frames + pv->passed_frames, (buf_in->flags & 16) ? "Y" : "N");
             
             hb_buffer_copy_settings( pv->buf_out[0], buf_in );
@@ -867,7 +1076,7 @@
         {
             /* No combing detected, pass input frame through unmolested.*/
             
-            pv->passed_frames++;
+ //           pv->passed_frames++;
             
             hb_buffer_copy_settings( pv->buf_out[0], buf_in );
             *buf_out = buf_in;
@@ -894,7 +1103,7 @@
     if( pv->mode < 7 )
     {
         /* Note down if the input frame is combed */
-        pv->comb = (pv->comb << 1) | hb_detect_comb(buf_in, width, height, pv->color_equal, pv->color_diff, pv->threshold, pv->prog_equal, pv->prog_diff, pv->prog_threshold);
+        pv->comb = (pv->comb << 1) | 1; //hb_detect_comb(buf_in, width, height, pv->color_equal, pv->color_diff, pv->threshold, pv->prog_equal, pv->prog_diff, pv->prog_threshold);
     }
 
     /* If yadif is not ready, store another ref and return FILTER_DELAY */
@@ -918,7 +1127,7 @@
     {
         /* Perform yadif filtering */
         
-        pv->deinterlaced_frames++;
+//        pv->deinterlaced_frames++;
         int frame;
         for( frame = 0; frame <= ( ( pv->mode == 2 || pv->mode == 5 )? 1 : 0 ) ; frame++ )
         {
@@ -955,7 +1164,7 @@
     {
         /* Perform yadif filtering */
         
-        pv->deinterlaced_frames++;
+//        pv->deinterlaced_frames++;
         int frame;
         for( frame = 0; frame <= ( ( pv->mode == 2 || pv->mode == 5 )? 1 : 0 ) ; frame++ )
         {
jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Re: Combing Detection

Post by jbrjake »

After staring at it for the zillionth time, IsCombedTIVTC finally started to make sense to me.

This is better than the last patch.

Instead of using the clever diffa * diffb test to make sure both the difference above and below are either negative or positive, that filter uses a more explicit "if both a and b are above the threshold, or both and b are below the negative of the threshold" test. Not sure if there's any effective difference, but I dutifully aped it.

The other changes are in default params and window size. The spatial threshold for IsCombedTIVTC is 9 instead of 6. The window size used to examine the combing mask is 16x16 instead of 4x8, with a threshold of 80 out of 256, instead of 15 out of 32. These changes combine to give *much* better performance on progressive animation with thin lines.

Also went back to tritical's noise resistant detector since it seems cooler.

http://handbrake.pastebin.ca/1035434

Code: Select all

Index: libhb/decomb.c
===================================================================
--- libhb/decomb.c	(revision 1474)
+++ libhb/decomb.c	(working copy)
@@ -230,6 +230,212 @@
     }
 }
 
+int tritical_detect_comb( hb_filter_private_t * pv )
+{
+    int x, y, i, j, k;
+    
+    /* Motion threshold */
+    int mthresh = 6;
+    /* Spatial threshold */
+    int athresh = 9;
+    /* Block mask threshold -- The number of pixels
+       in a X*Y block of the mask that need to show
+       combing for the whole frame to be seen as such. */
+    int threshold = 80;
+    int block_width = 16;
+    int block_height = 16;
+    
+    /* Make a buffer to store a comb mask. */
+    uint8_t        * mask[3];
+    for( i = 0; i < 3; i++ )
+    {
+        int is_chroma = !!i;
+        int w = ((pv->width[0]   + 31) & (~31))>>is_chroma;
+        int h = ((pv->height[0]+6+ 31) & (~31))>>is_chroma;
+
+        mask[i] = malloc( w*h*sizeof(uint8_t) ) + 3*w;
+    }
+    
+    /* One pas for Y, one pass for Cb, one pass for Cr */    
+    for( k = 0; k < 1; k++ )
+    {
+        int ref_stride = pv->ref_stride[k];
+        int width = pv->width[k];
+        int height = pv->height[k];
+
+        for( y = 2; y < (height-2); y++ )
+        {
+            /* These are just to make the buffer locations easier to read. */
+            int back_2 = y - 2;
+            int back_1 = y - 1;
+            int forward_1 = y + 1;
+            int forward_2 = y + 2;
+            
+            /* We need to examine a column of 5 pixels
+               in the prev, cur, and next frames.      */
+            uint8_t frame_a[5];
+            uint8_t frame_b[5];
+            uint8_t frame_c[5];
+            
+            for( x = 0; x < width; x++ )
+            {
+                /* Fill up the frame arrays with the current pixel values.*/
+                frame_a[0] = pv->ref[0][k][back_2*ref_stride + x];
+                frame_a[1] = pv->ref[0][k][back_1*ref_stride + x];
+                frame_a[2] = pv->ref[0][k][y*ref_stride + x];
+                frame_a[3] = pv->ref[0][k][forward_1*ref_stride + x];
+                frame_a[4] = pv->ref[0][k][forward_2*ref_stride + x];
+                frame_b[0] = pv->ref[1][k][back_2*ref_stride + x];
+                frame_b[1] = pv->ref[1][k][back_1*ref_stride + x];
+                frame_b[2] = pv->ref[1][k][y*ref_stride + x];
+                frame_b[3] = pv->ref[1][k][forward_1*ref_stride + x];
+                frame_b[4] = pv->ref[1][k][forward_2*ref_stride + x];
+                frame_c[0] = pv->ref[2][k][back_2*ref_stride + x];
+                frame_c[1] = pv->ref[2][k][back_1*ref_stride + x];
+                frame_c[2] = pv->ref[2][k][y*ref_stride + x];
+                frame_c[3] = pv->ref[2][k][forward_1*ref_stride + x];
+                frame_c[4] = pv->ref[2][k][forward_2*ref_stride + x];
+                
+                /* Make sure there's sufficient motion between frame t-1 to frame t+1. */
+                int motion = 0;
+                if( abs(frame_a[2] - frame_b[2] ) > mthresh && abs(frame_b[1] - frame_c[1] ) > mthresh && abs(frame_b[3] - frame_c[3] ) > mthresh )
+                    motion++;
+                if( abs(frame_c[2] - frame_b[2] ) > mthresh && abs(frame_a[1] - frame_b[1] ) > mthresh && abs(frame_a[3] - frame_b[3] ) > mthresh )
+                    motion++;
+
+#if 0
+                /* Visualizes the pixel values being examined. */
+                hb_log("\n PLANE: %i X: %i Y: %i \n %i %i %i \n %i %i %i \n %i %i %i \n %i %i %i \n %i %i %i", k, x, y,
+                frame_a[0], frame_b[0], frame_c[0],
+                frame_a[1], frame_b[1], frame_c[1],
+                frame_a[2], frame_b[2], frame_c[2],
+                frame_a[3], frame_b[3], frame_c[3],
+                frame_a[4], frame_b[4], frame_c[4] );
+#endif
+                if( motion )
+                {
+                    /* There is movement in this pixel location. */
+                    int up_diff = frame_b[2] - frame_b[1];
+                    int down_diff = frame_b[2] - frame_b[3];
+
+                    if( (up_diff >  athresh && down_diff >  athresh) ||
+                        (up_diff < -athresh && down_diff < -athresh) )
+                    {
+                        /* The pixel above and below are different,
+                           and are so in the same "direction" too.
+                           That means it's time for the spatial check.
+                           We've got several options here.             */
+#if 1
+                        /* Tritical's noise-resistant combing scorer.
+                           The check is done on a bob+blur convolution. */
+                        int combing = abs(  frame_b[0] + (4 * frame_b[2] ) + frame_b[4] -
+                                            ( 3 * ( frame_b[1] + frame_b[3] ) ) );
+
+                        /* If the frame is sufficiently combed,
+                           then mark it down on the mask as 255. */
+                       if( combing > 6*athresh )
+                            mask[k][y*ref_stride + x] = 255; 
+                        else
+                            mask[k][y*ref_stride + x] = 0;
+#endif
+
+#if 0                        
+                        /* Simple 32detect style comb detection */
+                        if( ( abs( frame_b[2] - frame_b[4] ) < pv->color_equal  ) &&
+                            ( abs( frame_b[2] - frame_b[3] ) > pv->color_diff ) )
+                        {
+                            mask[k][y*ref_stride + x] = 255;
+                        }
+                        else
+                        {
+                            mask[k][y*ref_stride + x] = 0;
+                        }
+#endif
+
+#if 0
+                        /* This, for comparison, is what IsCombed uses.
+                           It's simpler, but more noise senstive.      */
+                           int combing = (frame_b[1] - frame_b[2]) * (frame_b[3] - frame_b[2]);
+                           if( combing > 20*20 )
+                               mask[k][y*ref_stride + x] = 255; 
+                           else
+                               mask[k][y*ref_stride + x] = 0;
+#endif
+                    }
+                    else
+                    {
+                        mask[k][y*ref_stride + x] = 0;
+                    }
+                }
+                else
+                {
+                    mask[k][y*ref_stride + x] = 0;
+                }
+            }
+        }
+    }
+    
+    /* Go through the mask in X*Y blocks. If any of these windows
+       have threshold or more combed pixels, consider the whole
+       frame to be combed and send it on to be deinterlaced.     */
+    int block_score = 0;
+    for(k = 0; k < 1; k++ )
+    {
+        int ref_stride = pv->ref_stride[k];        
+        for( y = 0; y < ( pv->height[k] - block_height ); y = y + block_height )
+        {
+            for( x = 0; x < ( pv->width[k] - block_width ); x = x + block_width )
+            {
+                int block_x;
+                int block_y;
+                block_score = 0;
+                for(block_y = 0; block_y < block_height; block_y++ )
+                {
+                    for(block_x = 0; block_x < block_width; block_x++ )
+                    {
+                        int mask_y = y + block_y;
+                        int mask_x = x + block_x;
+                        
+                        if( mask[k][mask_y*ref_stride+mask_x] == 255 )
+                        {
+                            block_score++;
+                        }
+                    }
+                }
+                
+                if( block_score >= threshold )
+                {
+                    /* Consider this frame to be combed.
+                       Clear the mask buffer and return 1. */
+                    for( i = 0; i<3*3; i++ )
+                    {
+                        uint8_t **p = &mask[i/3];
+                        if (*p)
+                        {
+                            free( *p - 3*pv->ref_stride[i/3] );
+                            *p = NULL;
+                        }
+                    }
+                    return 1;
+                }
+            }
+        } 
+    }
+    
+    /* Consider this frame to be uncombed.
+       Clear the mask buffer and return 0. */
+    for( i = 0; i<3*3; i++ )
+    {
+        uint8_t **p = &mask[i/3];
+        if (*p)
+        {
+            free( *p - 3*pv->ref_stride[i/3] );
+            *p = NULL;
+        }
+    }
+    return 0;
+}
+
 static void yadif_filter_line( uint8_t *dst,
                                uint8_t *prev,
                                uint8_t *cur,
@@ -376,7 +582,17 @@
                           int tff,
                           hb_filter_private_t * pv )
 {
-
+    
+    int is_combed = tritical_detect_comb( pv );
+    
+    if(is_combed)
+    {
+        pv->deinterlaced_frames++;
+    }
+    else
+    {
+        pv->passed_frames++;
+    }
 #if 0
     /* Buggy, experimental code for macroblock-by-macroblock decombing.*/
     if( pv->mode == 7 )
@@ -499,7 +715,7 @@
 
                 blend_filter_line( dst2, cur, i, y, pv );
             }
-            else if( (y ^ parity) &  1 )
+            else if( (y ^ parity) &  1 && is_combed )
             {
                 uint8_t *prev = &pv->ref[0][i][y*ref_stride];
                 uint8_t *cur  = &pv->ref[1][i][y*ref_stride];
@@ -850,14 +1066,14 @@
                         pix_fmt, width, height );
 
         /* Check for combing on the input frame */
-        int interlaced =  hb_detect_comb(buf_in, width, height, pv->color_equal, pv->color_diff, pv->threshold, pv->prog_equal, pv->prog_diff, pv->prog_threshold);
+                    int interlaced = 1; // hb_detect_comb(buf_in, width, height, pv->color_equal, pv->color_diff, pv->threshold, pv->prog_equal, pv->prog_diff, pv->prog_threshold);
         
         if(interlaced)
         {
             avpicture_deinterlace( &pv->pic_out, &pv->pic_in,
                                    pix_fmt, width, height );
 
-            pv->deinterlaced_frames++;
+//            pv->deinterlaced_frames++;
             //hb_log("Frame %i is combed (Progressive: %s )", pv->deinterlaced_frames + pv->passed_frames, (buf_in->flags & 16) ? "Y" : "N");
             
             hb_buffer_copy_settings( pv->buf_out[0], buf_in );
@@ -867,7 +1083,7 @@
         {
             /* No combing detected, pass input frame through unmolested.*/
             
-            pv->passed_frames++;
+ //           pv->passed_frames++;
             
             hb_buffer_copy_settings( pv->buf_out[0], buf_in );
             *buf_out = buf_in;
@@ -894,7 +1110,7 @@
     if( pv->mode < 7 )
     {
         /* Note down if the input frame is combed */
-        pv->comb = (pv->comb << 1) | hb_detect_comb(buf_in, width, height, pv->color_equal, pv->color_diff, pv->threshold, pv->prog_equal, pv->prog_diff, pv->prog_threshold);
+        pv->comb = (pv->comb << 1) | 1; //hb_detect_comb(buf_in, width, height, pv->color_equal, pv->color_diff, pv->threshold, pv->prog_equal, pv->prog_diff, pv->prog_threshold);
     }
 
     /* If yadif is not ready, store another ref and return FILTER_DELAY */
@@ -918,7 +1134,7 @@
     {
         /* Perform yadif filtering */
         
-        pv->deinterlaced_frames++;
+//        pv->deinterlaced_frames++;
         int frame;
         for( frame = 0; frame <= ( ( pv->mode == 2 || pv->mode == 5 )? 1 : 0 ) ; frame++ )
         {
@@ -955,7 +1171,7 @@
     {
         /* Perform yadif filtering */
         
-        pv->deinterlaced_frames++;
+//        pv->deinterlaced_frames++;
         int frame;
         for( frame = 0; frame <= ( ( pv->mode == 2 || pv->mode == 5 )? 1 : 0 ) ; frame++ )
         {
User avatar
JohnAStebbins
HandBrake Team
Posts: 5712
Joined: Sat Feb 09, 2008 7:21 pm

Re: Combing Detection

Post by JohnAStebbins »

Wondered if you have tried this yet. In the second of tritical's posts that you referenced the says
if the current pixel is combed (set to 0xFF) and the pixel above it is combed and the pixel below it is combed, then it says that current pixel is combed
So he's only incrementing block_score when 3 adjacent pixels have registered combed. As he says, this should make it much more resistant of false positives on thin lines.
Post Reply