Absolutely essential feature: SWAP FIELD ORDER

[Synthetix]

You cannot play back NTSC DVD interlaced material in VLC with proper deinterlacing because VLC is designed to deinterlace PAL material, which differs in field order from NTSC material. So, if you rip a 29.97 fps DVD it will not deinterlace properly.

One solution would be to crop 1 pixel off the top of the video, and pad it by 1 pixel at the bottom. This effectively swaps the field order of the video, and now it will play back properly in VLC. HOWEVER, you can't do this in Handbrake's GUI because the cropping value is only even numbers! You can just add a checkbox here that takes the crop value and shifts it by one pixel (up or down doesn't matter). That way, NTSC material will now deinterlace OK in VLC.

You can do this with the CLI, which I have and it works perfectly. But adding a "swap fields" checkbox would be way easier.

[TedJ]

Field dominance has nothing to do with whether your source is NTSC or PAL. Most broadcast video formats are TFF (top field first), with the exception of DV and it's descendants (DVCPro, DVCam). In either case, as long as the MPEG stream is correctly flagged then VLC will correctly deinterlace it.

If you've transcoded your video using handbrake without deinterlacing, detelecine or decomb then you have a problem, as Handbrake expects and outputs progressive video which can't be deinterlaced during playback.

[Synthetix]

If you've transcoded your video using handbrake without deinterlacing,detelecine or decomb then you have a problem, as Handbrake expects and outputs progressive video which can't be deinterlaced during playback.

You can't use the deinterlace option because as you mention, you end up with progressive video. This is not a good idea with interlaced material because you are throwing away half of the temporal picture. I say "temporal" because NTSC material runs at 60 Hz, not 30. So what you're left with after a deinterlace is basically 30p, which is wrong.

Handbrake / x264 has to retain the input video's field structure to pass to VLC without using a deinterlace, otherwise the fields get blown away and we lose half of the picture. You can try encoding an interlaced source without any processing and you will see that VLC is getting the field order wrong. Of course, you must encode with the same frame height otherwise the picture scaling blurs the fields together.

[Synthetix]

as long as the MPEG stream is correctly flagged then VLC will correctly deinterlace it.

Sorry, let me clarify. Yes, you are right about the MPEG2 stream. VLC will deinterlace it properly, even in NTSC. What I was talking about was H.264 files generated by Handbrake / x264. If we want to retain an interlaced picture, this is possible but VLC will not deinterlace it properly unless the field order is swapped (at least for NTSC).

I have encoded an NTSC DVD to H.264 (matroska wrapper) successfully the way I mentioned (cropping one pixel), but I could only do this using the CLI. The CLI will allow you to enter odd pixel values for cropping (in this case, "1").

I can send you an example of what I'm talking about if you want.

[jbrjake]

Do you really think this kludge is more absolutely essential than bobbed deinterlacing?

[Synthetix]

Do you really think this kludge is more absolutely essential than bobbed deinterlacing?

Yes, because it would literally take 2 minutes to implement and would fix a huge problem. Swapping field order is a very basic function when working with format conversion tools, especially since VLC is incapable of detecting what the correct field order is with H.264 streams apparently.

Bob deinterlacing would be great, but it would double the frame rate and size of the output file. This is unnecessary when VLC can deinterlace on its own:

Frame 1 (60i)
-Field 1 -> New Frame 1 (60p)
-Field 2 -> New Frame 2 (60p)

etc.

Deinterlacing should be avoided when the frame rate can't be doubled. Otherwise, you are discarding temporal information which dramatically changes the look of the video.

[mduell]

Yes, because it would literally take 2 minutes to implement and would fix a huge problem.

I eagerly await your patch.

especially since VLC is incapable of detecting what the correct field order is with H.264 streams apparently.

Sounds like a VLC feature request.

[Synthetix]

I eagerly await your patch.

I briefly looked at the Mac code, but it would go something like this, as a quick-and-dirty example:

1. Open "PictureSettings.xib" in Interface Builder
2. Drag a check box NSButton to the window and name it "Swap Fields"

3. in PictureController.h:

Code: Select all

    IBOutlet NSButton        * fSwapFieldsCheck;

Connect in IB, etc.

4. In PictureController.m:

Code: Select all

    if( autoCrop )
    {
        memcpy( job->crop, fTitle->crop, 4 * sizeof( int ) );
    }
    else
    {
        job->crop[0] = [fCropTopStepper    intValue];
        job->crop[1] = [fCropBottomStepper intValue];
        job->crop[2] = [fCropLeftStepper   intValue];
        job->crop[3] = [fCropRightStepper  intValue];
    }
    
    if([fSwapFieldsCheck state] == NSOnState){
    	
    	//swap fields
        job->crop[0]++;
        job->crop[1]--;
    }

Sounds like a VLC feature request.

If it's possible to flag an MP4 or MKV container with the field order, then yes. VLC should detect it. I don't know if this is possible, however.

[mduell]

That seems poorly thought out.

The VLC feature would be the ability to specify TFF or BFF and handle appropriately.

[Synthetix]

That seems poorly thought out.

Well, poorly thought out or not, it would fix the problem.

I've tried to not be the guy who complains and then expects you to fix the problem. I've offered a solution here that although is admittedly quick-and-dirty (as I warned), is nonetheless an effort to improve your software. If you feel my suggestion is poorly though out, I am more than happy to collaborate on a solution. But shooting me down every time and passing the buck is not very productive, in my opinion.

The VLC feature would be the ability to specify TFF or BFF and handle appropriately.

Yes, it would be a great solution. But fixing Handbrake is a solution as well. And I've only used VLC as an example because it's my player of choice. The problem could exist with other players as well.

[JohnAStebbins]

Maybe I'm missing something. But handbrake doesn't output interlaced content. It deliberately looses all temporal information and field flags, and creates progressive output when given interlaced input. Leaving the image combed and trying to deinterlace at playback time when you don't have the necessary flags or timing information is a complete and utter kliudge. If you really think you need to transcode interlaced to interlaced, you need to pick another tool. HandBrake simply doesn't do that.

[Synthetix]

Maybe I'm missing something. But handbrake doesn't output interlaced content. It deliberately looses all temporal information and field flags, and creates progressive output when given interlaced input. Leaving the image combed and trying to deinterlace at playback time when you don't have the necessary flags or timing information is a complete and utter kliudge. If you really think you need to transcode interlaced to interlaced, you need to pick another tool. HandBrake simply doesn't do that.

Just because Handbrake doesn't flag the file as interlaced or specify a field order (although x264 has some provision for this) doesn't mean interlaced material cannot be stored and deinterlaced perfectly on playback. It is 100% possible to encode interlaced material and to keep the field structure intact in such a way that VLC (and perhaps other players) will deinterlace it on playback and present it at 60 Hz (NTSC). It doesn't matter if the frame is "labeled" progressive or not, VLC doesn't care. If the fields are present in the picture, they can be deinterlaced by VLC properly as long as the field order is as expected. VLC will default to what appears to be top field first if it cannot read a field dominance flag from the file. The problem is NTSC is bottom field! This works. I'm not trying to be rude nor arrogant by saying this, but indeed it is not even a matter that is worth disputing because it just plain works and the facts speak for themselves. Now before attacking me and telling me I don't know what I'm talking about, I will say that I am more than happy to PROVE this works by sending you EXAMPLES.

The problems are:

1. It is not made clear that it is possible to maintain the field structure even when compressed to H.264 "progressive"
2. Without any method of swapping fields, NTSC material will not play back properly in VLC because VLC assumes top-field first, which apparently is not the case with NTSC DVD.
3. This would be a trivial matter to overcome in the GUI except that cropping is only allowed by even numbers!

The solution is:

Cropping is the key to solving this, because cropping one pixel from the top and adding one to the bottom shifts the entire frame by one pixel, which puts field 2 in field 1's position and so-on.

Deinterlacing 60 Hz interlaced material is stupid. You are throwing away half of the picture with absolutely no benefit because VLC and other players can do this in realtime. You don't have to deinterlace an interlaced source because the field structure is bothering you. If that's the case, then your player is not set to the proper mode. Sure, a super duper solution would be to bob deinterlace to 60p, doubling the information in the file and the frame rate, but even then it may not be desirable to increase the size of the file.

[Synthetix]

Here's an example of what I'm talking about. Here is a still from a DVD encoded to MKV with Handbrake:



Looks terrible, doesn't it? Well, you have to open VLC and select these options:

Deinterlace mode: Linear
(then enable deinterlacing)

And WOW. Looks right. Here's the file: Sample interlaced H.264 MKV

If the deinterlacing is juddery, it's because the field order is swapped.

[TedJ]

The problem is NTSC is bottom field! This works. I'm not trying to be rude nor arrogant by saying this, but indeed it is not even a matter that is worth disputing because it just plain works and the facts speak for themselves. Now before attacking me and telling me I don't know what I'm talking about, I will say that I am more than happy to PROVE this works by sending you EXAMPLES.

Again, you seem fixated on the idea that all NTSC material is bottom field dominant and this is not so. I appreciate that your sources are BFF, but this is something that you cannot rely on... you would have to do a test encode and check playback in VLC for every new source you acquire.

While we're on the subject of VLC, what happens if you change media players to something that assumes all material is BFF unless flagged otherwise? All of your existing field inverted encodes are now broken.

[Synthetix]

Again, you seem fixated on the idea that all NTSC material is bottom field dominant and this is not so. I appreciate that your sources are BFF, but this is something that you cannot rely on... you would have to do a test encode and check playback in VLC for every new source you acquire.

I was merely speaking in the context of DVD ripping. This seems to be where the problem arises. Yes, I'm aware field order can be either depending on hardware, etc. But it's always the same for DVDs, apparently.

While we're on the subject of VLC, what happens if you change media players to something that assumes all material is BFF unless flagged otherwise? All of your existing field inverted encodes are now broken.

You're right! But by having the option to swap fields in Handbrake, we at least have some type of control over this. It'd be easier to fix it in the video than to fix the player.

[Deleted User 11865]

Again, you seem fixated on the idea that all NTSC material is bottom field dominant and this is not so. I appreciate that your sources are BFF, but this is something that you cannot rely on... you would have to do a test encode and check playback in VLC for every new source you acquire.

I was merely speaking in the context of DVD ripping. This seems to be where the problem arises. Yes, I'm aware field order can be either depending on hardware, etc. But it's always the same for DVDs, apparently.

No.

[Synthetix]

Again, you seem fixated on the idea that all NTSC material is bottom field dominant and this is not so. I appreciate that your sources are BFF, but this is something that you cannot rely on... you would have to do a test encode and check playback in VLC for every new source you acquire.

I was merely speaking in the context of DVD ripping. This seems to be where the problem arises. Yes, I'm aware field order can be either depending on hardware, etc. But it's always the same for DVDs, apparently.

No.

Well then, there you have it. If each DVD is different, and if VLC (or other programs) may only see top field first, and DVDs ripped to H.264 out of Handbrake cant be flagged top field/bottom field, then the only solution is to force top-field first by the technique I mentioned. If you think there is another solution, I am happy to hear but I think this would solve the problem.

If you wanted to get clever, you could read the field dominance from the VOB and perform this field shift automatically.

[Deleted User 11865]

Well then, there you have it. If each DVD is different, and if VLC (or other programs) may only see top field first, and DVDs ripped to H.264 out of Handbrake cant be flagged top field/bottom field, then the only solution is to force top-field first by the technique I mentioned.

If the source DVD is TFF to begin with, wouldn't swapping the fields make it BFF?

[JohnAStebbins]

Just because Handbrake doesn't flag the file as interlaced or specify a field order (although x264 has some provision for this) doesn't mean interlaced material cannot be stored and deinterlaced perfectly on playback.

Yes, of coarse you can do that. What I am saying is that HandBrake was not designed to output interlaced content. Using it to do so is a bad idea. If you wanted to output interlaced content the right way, it wouldn't be to hack the field order in a progressive frame as you are advocating. The correct way would be to actually generate real interlaced output with correct flags and timing information. I think your hack is a really bad idea because it encourages people to create broken streams as you seem intent on doing.

[Synthetix]

Just because Handbrake doesn't flag the file as interlaced or specify a field order (although x264 has some provision for this) doesn't mean interlaced material cannot be stored and deinterlaced perfectly on playback.

Yes, of coarse you can do that. What I am saying is that HandBrake was not designed to output interlaced content. Using it to do so is a bad idea. If you wanted to output interlaced content the right way, it wouldn't be to hack the field order in a progressive frame as you are advocating. The correct way would be to actually generate real interlaced output with correct flags and timing information. I think your hack is a really bad idea because it encourages people to create broken streams as you seem intent on doing.

I don't think it's a bad idea at all. I think discarding half the picture is a bad idea. Are you actually suggesting that getting rid of half of the picture is the proper way to encode the video? I don't think using the "we only support progressive streams" argument is justification to toss out half the image especially when I've already proved that this doesn't have to be the case! Really? You really prefer to throw away half of the image? I just want to make sure I'm understanding you correctly.

I agree with you that the "proper" way to do this would be to add field flags, etc as you suggest (or better yet, deinterlace and double the frame rate). But since that's not going to happen, a workaround would be to implement my "hack." Just because it's not the optimal way to do it (yes, I know players won't be able to detect field order on their own), it doesn't mean it wouldn't be a useful feature. I think the ability to retain more of the original picture is justification enough to implement this.

Regarding your comment about encouraging users to create broken streams, I'm not sure I understand what you mean since they can already do that now! At least with a field swap option, the possibility exists of generating interlaced video that plays perfectly (as I've already demonstrated) without having to get lucky with the DVD's field order.

[Deleted User 11865]

(or better yet, deinterlace and double the frame rate). But since that's not going to happen

Yes, it will happen eventually.

[JohnAStebbins]

Regarding your comment about encouraging users to create broken streams, I'm not sure I understand what you mean since they can already do that now! At least with a field swap option, the possibility exists of generating interlaced video that plays perfectly (as I've already demonstrated) without having to get lucky with the DVD's field order.

When most users do this, they come to the forums and complain about "horizontal lines" in their video. We tell them they need to enable deinterlacing in handbrake, and not that they need to deinterlace in the client. The stream is broken because it doesn't have the necessary information in the stream to inform a player that it is interlaced. So the player can not play it back correctly without user intervention.

As others have pointed out, you are essentially asking for a 'VLC' mode because VLC expects a certain default field dominance. Once encoded, the stream is only going to play back properly on VLC and only after the user has intervened to tell VLC that it needs to deinterlace. Some other players may have similar capabilities and limitations, but most will flat out display garbage.

Bobbing is being worked on. Actual interlaced output is unlikely, but you never know. If a developer with sufficient interest worked with us to add this capability, I think it would happen. None of the core developers has enough interest in it to bring it to the top of their todo lists.

Edit: btw, you would have gotten a cooler response had you toned down the subject line. This is by no means essential and shouting doesn't go over well.

[jbrjake]

I refuse your patch. What a moronic way to implement this highly questionable filter. If you want to swap field order, you should be doing it in the picture data by building a new buffer with two memcpys. As in, a video filter. Why drop two rows of data when you don't have to?

I agree with you that the "proper" way to do this would be to add field flags, etc as you suggest (or better yet, deinterlace and double the frame rate). But since that's not going to happen, a workaround would be to implement my "hack."

Imagine if you put this much effort into providing the testing and feedback I've been requesting for bobbed deinterlacing for literally over a year now.

[Synthetix]

As others have pointed out, you are essentially asking for a 'VLC' mode because VLC expects a certain default field dominance. Once encoded, the stream is only going to play back properly on VLC and only after the user has intervened to tell VLC that it needs to deinterlace. Some other players may have similar capabilities and limitations, but most will flat out display garbage.

Not necessarily. I've just used VLC as an example since I use it frequently and it offers this feature. I'm sure other players will do the same thing -- WMP on Windows, for example does deinterlacing like this I believe (it's been a while since I've tried it, though).

Bobbing is being worked on. Actual interlaced output is unlikely, but you never know. If a developer with sufficient interest worked with us to add this capability, I think it would happen. None of the core developers has enough interest in it to bring it to the top of their todo lists.

Great! I think we all agree that's a better solution. But the field swap option would be much simpler to implement. I don't know why you guys are so resistant to this. I can already do this in the CLI version using the crop trick. Fine, if you don't like the "swap fields" nomenclature then can you at least offer an "offset crop by 1 pixel" checkbox in the picture window?

Edit: btw, you would have gotten a cooler response had you toned down the subject line. This is by no means essential and shouting doesn't go over well.

I was just trying to accentuate the title of the post, I wasn't trying to shout. I feel I haven't been overly pushy or demanding, I am sincerely just trying to help improve the software. I don't know if you've been totally fair to me, though. I've tried to be helpful even to the point of supplying a basic code example, but I am still met with insulting and belittling remarks. Is this how you guys treat each other behind the scenes? No wonder bob deinterlacing is taking so long to implement.

I refuse your patch. What a moronic way to implement this highly questionable filter. If you want to swap field order, you should be doing it in the picture data by building a new buffer with two memcpys. As in, a video filter. Why drop two rows of data when you don't have to?

Well, after looking past your insult, let me answer with another question: "Why deinterlace and drop half of the video stream when you don't have to?"

Imagine if you put this much effort into providing the testing and feedback I've been requesting for bobbed deinterlacing for literally over a year now.

I'm sure every bit of it would have been met with the same resistance, so I'm not sure my involvement would have made any difference.

Look guys, all I'm trying to do here is help implement something I feel is useful and can be useful to others. I've gone out of my way to provide picture examples and even a basic code tweak (yes, I know it's a hack but it's only to illustrate my point). I'm not one of those people that that feels complaining about something being wrong without offering a solution is productive. I don't expect you to take what I say without discussion -- that's the point of collaboration. But I honestly didn't expect to met with such harsh treatment and insults.

I guess I've learned my lesson. Thanks for listening. I won't bother you about this anymore.

[nakomaru]

(or better yet, deinterlace and double the frame rate). But since that's not going to happen

Yes, it will happen eventually.

Hi there. As I see Bob support has been implemented, I am wondering if swapping the field order is possible yet. Whichever field Bob is assuming is the first field is wrong on some of my DVDs so I cannot use Handbrake to encode them yet. I tried fiddling with the custom deinterlace options (15-1:0:1) but was unsucessful. A swap field option for Bob's input would be essential for these encodes (perhaps I'm missing the setting). Thank you very much.