[0.9.3] Detelecine does not work

[Eugenia]

I have some .m2t PF24 HDV files from my HV20 camcorder, which records 24p in a 60i stream (that is, with pulldown added). Here's one such file for you to test: http://eugenia.gnomefiles.org/images/PF24.m2t

So, when I enable detelecine, and I tell it to export at 23.976, the final output is not correct. I still get the kind of ghosting you get when pulldown is added, and the length/frame count is not correct either. I have another proper way of removing pulldown with avisynth on Windows (1st method for Windows: http://eugenia.gnomefiles.org/2007/07/1 ... -pulldown/ ), a way that I know it works as it should. So when I compare your output with my avisynth output, the frames don't line up, plus as I said, there's still ghosting in the Handbrake-created picture while it's pristine in the avisynth one.

I hope you could fix the problem. Additionally, it would be nice to also support Windows' AVI DirectShow API, because if and when you fix pulldown removal for HDV PF24 footage, then we would need to export in a lossless codec (that happens to be installed Windows-wide) to edit that footage. Your current mpeg4 exporting options are lossy and very slow to edit, so they are not what most of us in the HV20/30 community need. Thanks.

[TedJ]

Ideally, we'd like to see an activity log for one of your encodes. Have you tried setting the framerate to "same as source" rather than forcing it to 23.976?

As for DirectShow support, I doubt that will ever be on the agenda... Handbrake is a cross platform application after all.

[Eugenia]

Thank you for the reply. Yes, I have tried with fps "same as source" and the ghosting is still there. Additionally, Sony Vegas' MainConcept decoder says that this file has... 100 fps, while Quicktime says it's 23.85 instead of 23.98 (which is what it reports for 23.976 footage). Finally, when I use "same as source", the audio waveform created is *longer* than the video by about 5 frames! This doesn't happen when I explicitly tell it to export at 23.976, albeit the detelecine is not correct in that case either as I mentioned on my first post.

Please let me know if you need more info. Too bad about the Windows AVI export btw, you could always add the feature just for Windows users, it's useful if you want to empower Handbrake with Avisynth, or other codecs.

Log below (sorry, I didn't find a way to actually attach the file rather than copy/pasting it):

### Windows GUI 0.9.3 2008112300
### Running: Microsoft Windows NT 5.1.2600 Service Pack 3
###
### CPU: Intel(R) Pentium(R) 4 CPU 3.00GHz
### Ram: 2047 MB
### Screen: 1280x1024
### Temp Dir: C:\Documents and Settings\Eugenia\Local Settings\Temp\
### Install Dir: C:\Program Files\Handbrake
### Data Dir: C:\Documents and Settings\Eugenia\Application Data\HandBrake\HandBrake\0.9.3.0
#########################################
[17:24:50] hb_init: checking cpu count
[17:24:50] hb_init: starting libhb thread
HandBrake 0.9.3 (2008112300) - http://handbrake.fr/
2 CPUs detected
Opening F:\Tools\Input\keeper03.m2t...
[17:24:50] hb_scan: path=F:\Tools\Input\keeper03.m2t, title_index=1
[17:24:50] scan: trying to open with libdvdread
[17:24:50] dvd: not a dvd - trying as a stream/file instead
[17:24:50] file is MPEG Transport Stream with 188 byte packets offset 0 bytes
[17:24:50] hb_ts_stream_find_pids - found the following PIDS
[17:24:50] Video PIDS :
[17:24:50] 0x810 type MPEG2 (0x2)
[17:24:50] Audio PIDS :
[17:24:50] 0x814 type MPEG1 (0x3)
[17:24:50] 0x815 type MSCODEC (0xa0)
[17:24:50] 0x811 type Unknown (0xa1)
[17:24:50] transport stream pid 0x814 (type 0x3) is MPEG1 audio id 0xc0
[17:24:50] transport stream pid 0x815 (type 0xa0, substream 0xbf) isn't audio
[17:24:50] transport stream pid 0x811 (type 0xa1, substream 0xbf) isn't audio
[17:24:50] scan: decoding previews for title 1
[17:24:50] scan: audio 0xc0: mp2, rate=48000Hz, bitrate=384000 Unknown (mp2) (2.0 ch)
Scanning title 1...
Scanning title 1...
Scanning title 1...
[17:24:51] scan: 10 previews, 1440x1088, 29.970 fps, autocrop = 0/8/0/0, aspect 1.76:1, PAR 4:3
[17:24:51] Title is likely interlaced or telecined (7 out of 10 previews). You should do something about that.
[17:24:51] scan: title (0) job->width:1440, job->height:816
[17:24:51] stream: 5 good frames, 0 errors (0%)
[17:24:51] libhb: scan thread found 1 valid title(s)
+ title 1:
+ vts 0, ttn 0, cells 0->0 (0 blocks)
+ duration: 00:00:14
+ size: 1440x1088, aspect: 1.76, 29.970 fps
+ autocrop: 0/8/0/0
+ chapters:
+ 1: cells 0->0, 0 blocks, duration 00:00:14
+ audio tracks:
+ 1, Unknown (mp2) (2.0 ch)
+ subtitle tracks:
+ combing detected, may be interlaced or telecined
[17:24:51] 1 job(s) to process
[17:24:51] starting job
[17:24:51] job configuration:
[17:24:51] * source
[17:24:51] + F:\Tools\Input\keeper03.m2t
[17:24:51] + title 1, chapter(s) 1 to 1
[17:24:51] * destination
[17:24:51] + C:\Documents and Settings\Eugenia\Desktop\test.mp4
[17:24:51] + container: MPEG-4 (.mp4 and .m4v)
[17:24:51] * video track
[17:24:51] + decoder: mpeg2
[17:24:51] + bitrate 25000 kbps
[17:24:51] + frame rate: 29.970 fps -> variable fps
[17:24:51] + dimensions: 1440 * 1088 -> 1280 * 720, crop 0/8/0/0
[17:24:51] + filter
[17:24:51] + Detelecine (pullup) (default settings)
[17:24:51] + encoder: x264
[17:24:51] + options: ref=2:bframes=2:me=umh
[17:24:51] + bitrate: 4000 kbps, pass: 0
[17:24:51] * audio track 0
[17:24:51] + decoder: Unknown (mp2) (2.0 ch) (track 1, id c0)
[17:24:51] + mixdown: Stereo
[17:24:51] + encoder: faac
[17:24:51] + bitrate: 160 kbps, samplerate: 44100 Hz
[17:24:51] dvd: not a dvd - trying as a stream/file instead
[17:24:51] reader: first SCR 188597079
[17:24:51] encx264: keyint-min: 30, keyint-max: 300
x264 [info]: using cpu capabilities: MMX2 SSE2 SSE3 Cache64
x264 [info]: profile Main, level 3.1
[17:24:51] mpeg2: "" (1) at frame 0 time 9009
[17:24:51] sync: expecting 462 video frames
[17:24:51] sync: first pts is 9009
[17:24:51] sync: adding 141 ms of silence to audio 0 start 12732, next 0
[17:26:25] hb_ts_stream_decode - eof
[17:26:25] stream: 429 good frames, 0 errors (0%)
[17:26:25] reader: done. 1 scr changes
[17:26:30] sync: got 424 frames, 462 expected
[17:26:30] work: average encoding speed for job is 4.272726 fps
[17:26:30] sync: got 424 frames, 462 expected
[17:26:30] work: average encoding speed for job is 4.272726 fps
[17:26:37] mpeg2 done: 425 frames
[17:26:37] render: lost time: 255255 (85 frames)
[17:26:37] render: gained time: 253755 (336 frames) (1500 not accounted for)
[17:26:37] render: average dropped frame duration: 3003
x264 [info]: slice I:2 Avg QP:25.87 size: 79718 PSNR Mean Y:38.11 U:45.22 V:46.51 Avg:39.51 Global:39.23
x264 [info]: slice P:165 Avg QP:24.25 size: 28816 PSNR Mean Y:38.82 U:45.28 V:46.83 Avg:40.18 Global:39.78
x264 [info]: slice B:170 Avg QP:25.95 size: 5293 PSNR Mean Y:37.89 U:45.15 V:46.69 Avg:39.31 Global:38.99
x264 [info]: consecutive B-frames: 9.3% 58.5% 32.2%
x264 [info]: mb I I16..4: 15.9% 0.0% 84.1%
x264 [info]: mb P I16..4: 1.0% 0.0% 3.3% P16..4: 58.2% 15.9% 13.2% 0.0% 0.0% skip: 8.4%
x264 [info]: mb B I16..4: 1.9% 0.0% 0.0% B16..8: 26.4% 0.9% 0.5% direct:13.1% skip:57.2% L0:23.4% L1:58.5% BI:18.1%
x264 [info]: final ratefactor: 21.63
x264 [info]: ref P L0 86.2% 13.8%
x264 [info]: SSIM Mean Y:0.9675129
x264 [info]: PSNR Mean Y:38.349 U:45.217 V:46.756 Avg:39.736 Global:39.357 kb/s:4136.30
[17:26:37] mp2-decoder done: 0 frames, 0 decoder errors, 0 drops
[17:26:37] libhb: work result = 0
Rip done!
############ End of Log ##############

[Eugenia]

Oh, one more thing:
Your log says that I have
>### Ram: 2047 MB
While I have 3 GB RAM, up and working fine.

[jbrjake]

When you don't use Same as Source FPS, it cannot drop frames, so yes you will see duplicate frames.

But I just encoded your sample with detelecine and Same as Source FPS and it looks fine, no duplicate frames, so I don't know what you mean.

It's variable frame rate detelecine like you read in the documentation, so of course different players will handle it differently and some will handle it badly.

[Eugenia]

But I just encoded your sample with detelecine and Same as Source FPS and it looks fine no duplicate frames, so I don't know what you mean.

Here's what I mean:
http://eugenia.gnomefiles.org/images/telecine.jpg
1. The first ~10 frames of Handbrake's video are interlaced! In fact, they seem to be copies of the same first frame. Only later the detelecine seems to kick in.
2. The Handbrake video is ~2-3 frames longer than the Avisynth one in the timeline!
3. The Handbrake video, even if it's longer, it's MISSING about 3-4 frames at the end of the clip! The Avisynth clip has more frames at the end than Handbrake does, and by that I mean the last frame of the Avisynth video is a completely different shot than Handbrake's.
4. Finally, the VFR is a bad idea for us who want to edit and must pick project properties of constant frame rate, is there a way to turn that off? PF24 is always telecined, it's not like some videos that some times are telecined and others are not, so what the HV20/30 community needs is a constant bitrate of 23.976 after pulldown removal.

Thanks.

[jbrjake]

1. The first ~10 frames of Handbrake's video are interlaced! In fact, they seem to be copies of the same first frame. Only later the detelecine seems to kick in.

First of all, using the sample you provided, I do not get 10 copies of the first frame. You seem to be confusing a frame with a long duration (due to offsetting to ensure AV sync) with 10 frames. There is a difference. And no, a stateless detelecining filter is not going to kick in until it sees a telecine pattern.

2. The Handbrake video is ~2-3 frames longer than the Avisynth one in the timeline!

3. The Handbrake video, even if it's longer, it's MISSING about 3-4 frames at the end of the clip! The Avisynth clip has more frames at the end than Handbrake does, and by that I mean the last frame of the Avisynth video is a completely different shot than Handbrake's.

Using the sample you provided, HandBrake's last frame of visible output is exactly the same as your sample's. I see no missing frames.

4. Finally, the VFR is a bad idea for us who want to edit

You shouldn't be using HandBrake if you want to edit in the first place. It's not designed for that.

and must pick project properties of constant frame rate, is there a way to turn that off?

....by setting a constant frame rate, just like you read in the documentation.

PF24 is always telecined, it's not like some videos that some times are telecined and others are not, so what the HV20/30 community needs is a constant bitrate of 23.976 after pulldown removal.

I'm sure the HV20/30 community eagerly awaits your patch, then.

[Eugenia]

>I do not get 10 copies of the first frame.

I do. Sony Vegas shows 10 interlaced copies of the same frame in the beginning.

>You seem to be confusing a frame with a long duration

No, I know how to move frame by frame in the Vegas timeline.

>And no, a stateless detelecining filter is not going to kick in until it sees a telecine pattern.

Yes, but PF24's stream is easily recognized by other solutions as telecined from the first 3-4 frames already, and what I get instead is 10 interlaced frame copies.

>Using the sample you provided, HandBrake's last frame of visible output is exactly the same as your sample's. I see no missing frames.

I have to look at that later, as I am not on the same PC right now, but from what I've seen in the timeline with my Avisynth output, the avisynth output had about 3 more frames, showing less of the cat on the left of the frame.

>You shouldn't be using HandBrake if you want to edit in the first place. It's not designed for that.

Well, variable frame rate is not exactly very compatible. Vimeo would probably have an A/V sync problems for example. Besides, there might be people who would want to transcode their PF24 files using handbrake for their ipods too. So far, it just doesn't work properly IMO.

>I'm sure the HV20/30 community eagerly awaits your patch, then.

I don't develop anymore. But I did give you all the info you need.

[Eugenia]

Ok, I have more proof that your detelecine process does not work with HDV pulldown added 24p footage:
1. Look at this for the last frames not being the same: http://eugenia.gnomefiles.org/images/telecine2.jpg (2-3 frames are missing from your transcoding).
2. Load your Handbrake detelecine .mp4 file with Quicktime. Move frame by frame and you will see that what I say is TRUE: the first few frames are all copies of the same INTERLACED frame. After de-telecining there should have never been no interlacing! So it's not "just Vegas" being crazy with your file, it's how your file really got transcoded, and this is NOT how it should have been, because my other pulldown removal tools (I have 3: Avisynth, Cineform, and After Effects) don't create such footage/behavior. The other tools create normal progressive footage of non-repeating frames.

So the bug is on your side. Your application just doesn't handle properly PF24 footage. And VFR doesn't help either.

You have a test file, I can get you more if you want, the ball is on your side if you would like to fix this or not.

[jbrjake]

>I do not get 10 copies of the first frame.

I do. Sony Vegas shows 10 interlaced copies of the same frame in the beginning.

But I'm not talking about what some random editing tool shows -- I'm talking about examining the video file itself and looking at the samples to time index written to the MP4 header. It is one frame with a long duration, not multiple frames, despite how a player/editor might misinterpret this.

No, I know how to move frame by frame in the Vegas timeline.

Vegas is not an authority on an MP4 file. A dump of the MP4 file is.

>Using the sample you provided, HandBrake's last frame of visible output is exactly the same as your sample's. I see no missing frames.

I have to look at that later, as I am not on the same PC right now, but from what I've seen in the timeline with my Avisynth output, the avisynth output had about 3 more frames, showing less of the cat on the left of the frame.

Ah, okay, now I see that. You threw me off when you claimed before that "the last frame of the Avisynth video is a completely different shot than Handbrake's." I was looking for a completely different shot, not a tiny nudge of the camera but essentially the same shot.

A couple of missing frames at the end of the stream just mean something isn't getting flushed properly somewhere in the rendering pipeline, and isn't a big deal either, although I will look into that.

>You shouldn't be using HandBrake if you want to edit in the first place. It's not designed for that.

Well, variable frame rate is not exactly very compatible. Vimeo would probably have an A/V sync problems for example.

That would be Vimeo's problem. The MP4 format inherently supports variable frame rates. That's why every sample can have a different duration. If a player does not support this, it is a bug. Considering that even cheap consumer devices like iPods can play variable framerate MP4 files easily, compatibility is not a major concern.

Your application just doesn't handle properly PF24 footage.

First off, that's not the assertion you started this thread with: "[0.9.3] Detelecine does not work."

Second, you haven't proven this to me yet. If you want me to believe that pullup does not handle this footage correctly you're going to have to give me a sample I can encode and see that it's failing to identify the telecine sequence during the course of the motion video. Not the first frame. Frames in the middle. I want to see it leaving behind interlaced frames, or incorrectly reweaving them, or something, anything, that would indicate a real fundamental problem with how the algorithm works. The very beginning of a stream is going to have oddities with a stateless filter.

[Eugenia]

Sorry, but "oddities" are not acceptable IMO.

Look at how Quicktime renders the video, the SAME way as Vegas does: the first few frames are *copies* of the same first frame and are INTERLACED. Did you actually looked at this, I mentioned it very specifically!!! When you load the video, tell Quicktime to stop and go to the beginning, and then use the right arrow key to move frame by frame. You will see that t he first few frames are INTERLACED, and they are the same.

When I remove pulldown with ANOTHER solution, like After Effects, Cineform or Avisynth (proven solutions for pulldown removal), they don't have interlaced, frame copies in the beginning! And they don't miss frames at the end either.

So no, from my point of view, your detelecined algorithm doesn't work. You can say what you please, but to me the output is not what it should be. And VFR makes it even worse!

And what do you mean "you're going to have to give me a sample I can encode"?? I did give you a sample .m2t PF24 file, it's linked above, and from what I can gather from your comments, you HAVE downloaded it and already tried it! All I am saying is that the output produced is wrong. It's not consisted with the other solutions, solutions that do agree with each other and they don't have interlaced or missing frames in them.

Look, I am trying to help out here. I am laying out the facts as plainly as I can, I provided logs and sample footage. But you seem to think that your detelecine algorithm works as expected, while it's not! And if this is indeed how it is supposed to work, then I don't find that result acceptable.

[jbrjake]

Sorry, but "oddities" are not acceptable IMO.

*shrug* If you find them that unacceptable, again, consider contributing a patch to the project.

Look at how Quicktime renders the video, the SAME way as Vegas does: the first few frames are *copies* of the same first frame and are INTERLACED. Did you actually looked at this, I mentioned it very specifically!!!

Yes, I looked at this, I replied to it very specifically!!!

As I told you, when actually examining the sample to time index in the encoded output MP4 file, it is clear that what the players are interpreting as copies of the first frame are, in fact, one frame with a very long duration. It's probably due to some offsetting code to ensure AV sync. Getting its duration right is something I will look into. But it is not the bug you're asserting in this thread: "Detelecine does not work." Detelecine not working would imply detelecining is not working, but clearly, it is picking up a telecine pattern in this source and successfully filtering it...after the first frame. Which makes sense, since it's a stateless filter, and it needs a temporal reference.

When I remove pulldown with ANOTHER solution, like After Effects, Cineform or Avisynth (proven solutions for pulldown removal), they don't have interlaced, frame copies in the beginning! And they don't miss frames at the end either.

I eagerly await your port of TIVTC so that people who aren't on Windows can use it.

And what do you mean "you're going to have to give me a sample I can encode"?? I did give you a sample .m2t PF24 file, it's linked above, and from what I can gather from your comments, you HAVE downloaded it and already tried it! All I am saying is that the output produced is wrong. It's not consisted with the other solutions, solutions that do agree with each other and they don't have interlaced or missing frames in them.

If the detelecine filter wasn't working, if it wasn't compatible with this PF24 telecining cadence, then it wouldn't work at all. But from the short clip you provided, all I can see is minor issues at the very beginning and end of a stream, and sandwiched between them is the actual content of your sample, which is filtered perfectly fine. Since you say "Detelecine does not work" I'm clearly missing something. I would like to have a longer sample so that I can see for myself that the filter is actually failing to work on motion video sequences. Because so far, you haven't provided me with any evidence that the pullup algorithm is failing. It seems to be correctly reweaving and dropping frames during the course of the video, which means it is correctly interpreting the telecine pattern. Another developer tried as well and said the same thing -- the source video shows telecining artifacts throughout, while the filtered output only has that one bad frame at the beginning.

Look, I am trying to help out here. I am laying out the facts as plainly as I can, I provided logs and sample footage. But you seem to think that your detelecine algorithm works as expected, while it's not! And if this is indeed how it is supposed to work, then I don't find that result acceptable.

It's not my algorithm. It's Richard Felker's. It's the pullup filter from MEncoder. It is the most sophisticated pullup algorithm that is both open source and coded so it runs everywhere. TIVTC, which you use in AviSynth, is of course better (no one is denying this, search the forum and you'll surely find examples of me saying so) but porting it to HandBrake is a lot of work. Since you feel the current situation is unacceptable, I strongly encourage you to port its logic to some plain C code that could be integrated into HandBrake and other cross-platform video encoders like Transcode and MEncoder.