Combing Detection
Forum rules
*******************************
Please be aware we are now using GitHub for issue tracking and feature requests.
- This section of the forum is now closed to new topics.
*******************************
*******************************
Please be aware we are now using GitHub for issue tracking and feature requests.
- This section of the forum is now closed to new topics.
*******************************
-
- Veteran User
- Posts: 468
- Joined: Mon Nov 02, 2009 2:18 pm
Re: Combing Detection
Dear jbrjake,
Is this new work specific to the "Default" decomb setting or is it something that might apply to the Custom combinations, specifically option "11"?
Take care,
Reuben
Is this new work specific to the "Default" decomb setting or is it something that might apply to the Custom combinations, specifically option "11"?
Take care,
Reuben
Re: Combing Detection
Heh, suppose it helps if I say how to use it ;p
The basic way to invoke it is:
--decomb=391:2:3:3:2:40
The new features are implemented as new modes, that get layered (literally added) onto the others.
1: Yadif
+
2: Blend
+
4: Cubic interpolation for yadif
+
128: Gamma scaling
+
256: Filtering
==
391
Then the other parameters are more sensitive thresholds, plus a flag to tell the filtering mode to use erosion and dilation.
So for your EEDI2 settings, you'd do 11 + 128 + 256 for the mode, or 395, followed by the more intense thresholds and filter: :2:3:3:2:40.
--decomb=395:2:3:3:2:40
The basic way to invoke it is:
--decomb=391:2:3:3:2:40
The new features are implemented as new modes, that get layered (literally added) onto the others.
1: Yadif
+
2: Blend
+
4: Cubic interpolation for yadif
+
128: Gamma scaling
+
256: Filtering
==
391
Then the other parameters are more sensitive thresholds, plus a flag to tell the filtering mode to use erosion and dilation.
So for your EEDI2 settings, you'd do 11 + 128 + 256 for the mode, or 395, followed by the more intense thresholds and filter: :2:3:3:2:40.
--decomb=395:2:3:3:2:40
-
- Veteran User
- Posts: 468
- Joined: Mon Nov 02, 2009 2:18 pm
Re: Combing Detection
Dear jbrjake,
Is this work you did something that requires a newer build of Handbrake? I was trying to find the associated change in the trac.handbrake.fr/timeline page and I didn't find it there.
Take care,
Reuben
Is this work you did something that requires a newer build of Handbrake? I was trying to find the associated change in the trac.handbrake.fr/timeline page and I didn't find it there.
Take care,
Reuben
Re: Combing Detection
It's not committed yet, you have to apply the patch yourself and build from source.randomreuben wrote:Dear jbrjake,
Is this work you did something that requires a newer build of Handbrake? I was trying to find the associated change in the trac.handbrake.fr/timeline page and I didn't find it there.
Take care,
Reuben
Re: Combing Detection
So, I've been testing this on a few progressive sources to see how well it fared against current defaults in terms of false positives, and the results are impressive indeed.
At first it seemed to be a tad too aggressive, on a 10-minute fully-progressive clip (Big Buck Bunny):but once you compare it to decomb2 defaults, it's pretty close:
Equally safe on a 2-minute Blu-Ray clip, also fully progressive:
And an additional animation source (LosslessMaikaze.mkv, another of Dark_Shikari's x264-encoded samples):
With the old (current) decomb, my personal "aggressive" decomb string was 7:2:2:2:64 (for sources where defaults let enough combing through that it was annoying).
Back when I experimented with it, I came to the conslusion that a reasonably high block threshold combined with very low spatial and motion thresholds provided a better compromise between as few false positives as possible and finding more combing than defaults, than alternatives with higher spatial and motion thresholds and a low block threshold.
Trying with decomb3 on BigBuckBunny (becomes 391:2:2:2:2:64), and then tried lowering the block threshold in increments of 8. Listed by number of unfiltered frames, highest to lowest:
new decomb3 "defaults":
At least in terms of false positives, my old conclusion still appears correct with decomb3. Whether my settings find more/less actual combing in interlaced sources remains to be tested.
At first it seemed to be a tad too aggressive, on a 10-minute fully-progressive clip (Big Buck Bunny):
Code: Select all
[22:51:31] + Decomb (391:2:3:3:2:40)
[23:11:21] decomb: deinterlaced 396 | blended 191 | unfiltered 13727 | total 14314
Code: Select all
[22:28:18] + Decomb (default settings)
[22:39:33] decomb: deinterlaced 133 | blended 324 | unfiltered 13857 | total 14314
Code: Select all
[22:46:49] + Decomb (391:2:3:3:2:40)
[22:51:25] decomb: deinterlaced 0 | blended 0 | unfiltered 3242 | total 3242
Code: Select all
[22:24:46] + Decomb (default settings)
[22:28:10] decomb: deinterlaced 0 | blended 0 | unfiltered 3242 | total 3242
Code: Select all
[23:11:23] + Decomb (391:2:3:3:2:40)
[23:12:45] decomb: deinterlaced 0 | blended 1 | unfiltered 4998 | total 4999
Code: Select all
[22:39:39] + Decomb (default settings)
[22:40:42] decomb: deinterlaced 0 | blended 0 | unfiltered 4999 | total 4999
Back when I experimented with it, I came to the conslusion that a reasonably high block threshold combined with very low spatial and motion thresholds provided a better compromise between as few false positives as possible and finding more combing than defaults, than alternatives with higher spatial and motion thresholds and a low block threshold.
Trying with decomb3 on BigBuckBunny (becomes 391:2:2:2:2:64), and then tried lowering the block threshold in increments of 8. Listed by number of unfiltered frames, highest to lowest:
Code: Select all
[22:28:18] + Decomb (default settings)
[22:39:33] decomb: deinterlaced 133 | blended 324 | unfiltered 13857 | total 14314
Code: Select all
[23:21:42] + Decomb (391:2:2:2:2:64)
[23:41:39] decomb: deinterlaced 320 | blended 203 | unfiltered 13791 | total 14314
Code: Select all
[23:48:28] + Decomb (391:2:2:2:2:56)
[00:08:29] decomb: deinterlaced 380 | blended 188 | unfiltered 13746 | total 14314
Code: Select all
[22:51:31] + Decomb (391:2:3:3:2:40)
[23:11:21] decomb: deinterlaced 396 | blended 191 | unfiltered 13727 | total 14314
Code: Select all
[00:16:00] + Decomb (391:2:2:2:2:48)
[00:36:22] decomb: deinterlaced 415 | blended 231 | unfiltered 13668 | total 14314
Code: Select all
[01:38:16] + Decomb (7:2:2:2:1:64)
[01:51:30] decomb: deinterlaced 661 | blended 3472 | unfiltered 10181 | total 14314
Re: Combing Detection
Thanks, Rodeo. I really appreciate the testing. I'll look into dropping the combing thresholds more and raising the block threshold a bit. Might also be worth exploring different block sizes, like 8x8.
Re: Combing Detection
http://handbrake.fr/pastebin/pastebin.php?show=1414
I just noticed this:
The defaults string indicates that pv->filter_mode defaults to FILTER_CLASSIC. But, line 748:
I just noticed this:
Code: Select all
@@ -22,7 +22,7 @@
Parity
Defaults:
- 7:2:6:9:80:16:16:10:20:20:4:2:50:24:1:-1
+ 7:2:6:9:1:80:16:16:10:20:20:4:2:50:24:1:-1
*****/
#define MODE_YADIF 1 // Use yadif
@@ -31,7 +31,13 @@
#define MODE_EEDI2 8 // Use EEDI2 interpolation
#define MODE_MCDEINT 16 // Post-process with mcdeint
#define MODE_MASK 32 // Output combing masks instead of pictures
+#define MODE_GAMMA 128 // Scale gamma when decombing
+#define MODE_FILTER 256 // Filter combing mask
+#define MODE_COMPOSITE 512 // Overlay combing mask onto picture
+#define FILTER_CLASSIC 1
+#define FILTER_ERODE_DILATE 2
+
/*****
These modes can be layered. For example, Yadif (1) + EEDI2 (8) = 9,
which will feed EEDI2 interpolations to yadif.
Code: Select all
+ pv->filter_mode = FILTER_ERODE_DILATE;
Re: Combing Detection
Thanks. Not really a big deal, though. That param gets ignored by default since the new filtering mode (filter_combing_mask) is off by default, and it merely controls the method of filtering employed when the new mode is enabled. And the only reason someone would use the old method with the new filtering mode instead of erode/dilate is the reason I added the old method for -- debugging and producing visual output of what the old filtering method's results looked like.
To explain the code a bit....
Traditionally decomb just looked for horizontally adjacent combed pixels as a filter. It does this as it's checking the combing mask (check_combing_mask), and it doesn't store a copy. So you can't look at the filtering results. The new filtering mode (filter_combing_mask) does store results for later review. I was going to have the new filtering mode only look for adjacent pixels horizontally *and* vertically and then do the new erosion/dilate filtering. But I wanted to compare against what used to happen, so I added a parameter to flip between the new and old techniques inside the new filtering mode. Using FILTER_CLASSIC runs the new filter_combing_mask routine but only looks for horizontally adjacent pixels and skips the erosion and dilation passes. Does the same as what happens when you don't include filtering in the main mode parameter, but keeps a copy of the filtered combing mask around incase debug output is enabled. (Debug output employed without the new filering mode overlays the raw, unfiltered combing mask on the output).
To explain the code a bit....
Traditionally decomb just looked for horizontally adjacent combed pixels as a filter. It does this as it's checking the combing mask (check_combing_mask), and it doesn't store a copy. So you can't look at the filtering results. The new filtering mode (filter_combing_mask) does store results for later review. I was going to have the new filtering mode only look for adjacent pixels horizontally *and* vertically and then do the new erosion/dilate filtering. But I wanted to compare against what used to happen, so I added a parameter to flip between the new and old techniques inside the new filtering mode. Using FILTER_CLASSIC runs the new filter_combing_mask routine but only looks for horizontally adjacent pixels and skips the erosion and dilation passes. Does the same as what happens when you don't include filtering in the main mode parameter, but keeps a copy of the filtered combing mask around incase debug output is enabled. (Debug output employed without the new filering mode overlays the raw, unfiltered combing mask on the output).
Re: Combing Detection
Thanks for the explanation.
Now I understand why default decomb settings (without using a custom string) give the same results as in non-patched builds, despite ERODE_DILATE being "enabled".
Now I understand why default decomb settings (without using a custom string) give the same results as in non-patched builds, despite ERODE_DILATE being "enabled".
Re: Combing Detection
As I mentioned in that topic, after jbrjake suggested a new way of formatting decomb custom strings, I was so thrilled with the idea that I just couldn't wait and started working on it right away.
While he may want to start from scratch, I'll post my patch anyway: http://handbrake.fr/pastebin/pastebin.php?show=1498
It's full of testing code and documentation is minimal, but appears to work in my testing so far.
While he may want to start from scratch, I'll post my patch anyway: http://handbrake.fr/pastebin/pastebin.php?show=1498
It's full of testing code and documentation is minimal, but appears to work in my testing so far.
Re: Combing Detection
I'm dealing with a mixed progressive an interlaced content and I don't arrival to get rid of some interlaced artifacts. I've used many settings in vain.
Here my command-line: Lossless is for testing purpose only.
Is the Block width can be decreased under 40 ? Actually this is the best result I have with --decomb=409:2:3:3:2:40.
What should I do ? 409 is very slow and produce same rendering has 391 or 390.
Here some screenshots.
(The star)
(hand of the girl)
Here my command-line:
Code: Select all
HandBrakeCLI -v -i LUCKY_STAR_VOL1.ISO -t 8 -c 1-6 -o \[BenproLOSS\]LS_01.mkv -m -e x264 -q 0 -a 2 -E copy --crop 4:0:4:4 -w 848 -l 480 --custom-anamorphic --pixel-aspect 427:424 --decomb=409:2:3:3:2:40 --deblock=15:2
Is the Block width can be decreased under 40 ? Actually this is the best result I have with --decomb=409:2:3:3:2:40.
What should I do ? 409 is very slow and produce same rendering has 391 or 390.
Here some screenshots.
(The star)
(hand of the girl)
-
- Experienced
- Posts: 75
- Joined: Wed Aug 22, 2007 5:21 am
Re: Combing Detection
Note to anyone using this or the version on review board: r4297 mixed with this breaks compilation, as then both render.c and decomb.c have a method void build_gamma_lut( hb_work_private_t * pv ).
These are not equivalent methods:
Render.c
Decomb.c
Does anyone know which gives more correct output, with that huge scale factor on one and not the other? Or are they each correct in their own usage?
Cheers, Andrew
These are not equivalent methods:
Render.c
Code: Select all
void build_gamma_lut( hb_work_private_t * pv )
{
int i;
for( i = 0; i < 256; i++ )
{
pv->gamma_lut[i] = 4095 * pow( ( (float)i / (float)255 ), 2.2f );
}
}
Code: Select all
void build_gamma_lut( hb_filter_private_t * pv )
{
int i;
for( i = 0; i < 256; i++ )
{
pv->gamma_lut[i] = pow( ( (float)i / (float)255 ), 2.2f );
}
}
Cheers, Andrew
- JohnAStebbins
- HandBrake Team
- Posts: 5726
- Joined: Sat Feb 09, 2008 7:21 pm
Re: Combing Detection
Thanks. I should have made that static. I'll fix that.
If you look at the comments for the one in render, you'll see the values are scaled for integer arithmatic and are designed specifically so the math that is being done there does not overflow.
If you look at the comments for the one in render, you'll see the values are scaled for integer arithmatic and are designed specifically so the math that is being done there does not overflow.
-
- Experienced
- Posts: 75
- Joined: Wed Aug 22, 2007 5:21 am
Re: Combing Detection
Well, that was fast
Just wondering, what are the chances of this going in soonish? I've used it since posting and it has produced better results on both Anime and live-action tv recordings than default decomb.
Cheers, Andrew
Just wondering, what are the chances of this going in soonish? I've used it since posting and it has produced better results on both Anime and live-action tv recordings than default decomb.
Cheers, Andrew
- JohnAStebbins
- HandBrake Team
- Posts: 5726
- Joined: Sat Feb 09, 2008 7:21 pm
Re: Combing Detection
I would like to see it go in before the next release. I mentioned it on IRC, but haven't gotten feedback yet. It's gotten lots of testing. Only real issue is that enabling the new features makes it slower. But since it's completely optional to enable, I don't see that as a problem.
- JohnAStebbins
- HandBrake Team
- Posts: 5726
- Joined: Sat Feb 09, 2008 7:21 pm
Re: Combing Detection
decomb3 has been committed and gamma compensation + erode dilate filter are enabled by default
https://trac.handbrake.fr/changeset/4308
https://trac.handbrake.fr/changeset/4308
-
- Veteran User
- Posts: 468
- Joined: Mon Nov 02, 2009 2:18 pm
Re: Combing Detection
I tried this out last night and I love the results! It looks very nice! Thank you very much for this spectacular update!