Judder-free, 60 FPS telecine (?)

General questions or discussion about HandBrake, Video and/or audio transcoding, trends etc.
markfilipak
Bright Spark User
Posts: 239
Joined: Thu Aug 01, 2019 8:58 pm

Re: Judder-free, 60 FPS telecine (?)

Post by markfilipak »

mduell wrote: Tue Mar 31, 2020 7:24 pm I urge you to write and support the necessary code to integrate arbitrary filters from the libavfilter library into HB. Should keep you busy for a while.
I'm not competent to do that. I know about 14 programming languages that I've used in products. About 1/3rd are microprocessor assembly languages, 1/3rd compiled languages (but not 'C' even though I can write 'C'), and 1/3rd scripting languages. But I'm a retired hardware development engineer -- emphasis on "retired" -- and I work on movie reviews that are, shall I say, "technical". I don't need another hobby.
DrXenos
Bright Spark User
Posts: 251
Joined: Sat Mar 16, 2013 1:19 pm

Re: Judder-free, 60 FPS telecine (?)

Post by DrXenos »

markfilipak wrote: Tue Mar 31, 2020 7:05 pmThat stuff could be used to create a single, Swiss Army knife preset in HandBrake...
I've heard the develops say multiple times they're not interested in Handbrake becoming a "swiss army knife," just a solid video transcoder.
markfilipak
Bright Spark User
Posts: 239
Joined: Thu Aug 01, 2019 8:58 pm

Re: Judder-free, 60 FPS telecine (?)

Post by markfilipak »

DrXenos wrote: Wed Apr 01, 2020 5:04 pm I've heard the develops say multiple times they're not interested in Handbrake becoming a "swiss army knife," just a solid video transcoder.
Too late! HandBrake already looks and operates like a Swiss Army knife.
markfilipak
Bright Spark User
Posts: 239
Joined: Thu Aug 01, 2019 8:58 pm

Re: Judder-free, 60 FPS telecine (?)

Post by markfilipak »

mduell wrote: Tue Mar 31, 2020 7:24 pm I urge you to write and support the necessary code to integrate arbitrary filters from the libavfilter library into HB. Should keep you busy for a while.
Will you hold my hand while a try to fathom the mysteries of GCC & header files & the way that 'C' references structs? I can write 'C', but I've never written & compiled a finished app, just skeletal code that I passed to my coders. But I've written 10s of thousands of lines of Java & JS, and they are syntacticallly & structurally very 'C'-like languages.
DrXenos
Bright Spark User
Posts: 251
Joined: Sat Mar 16, 2013 1:19 pm

Re: Judder-free, 60 FPS telecine (?)

Post by DrXenos »

markfilipak wrote: Thu Apr 02, 2020 12:31 am Too late! HandBrake already looks and operates like a Swiss Army knife.
I don't believe that's true. It certainly doesn't have all the knobs that something like ffmpeg has, nor does it take the place of muxer/demuxers like mkvtoolnix, or audio/video editors like avidemux or Audacity. It especially doesn't handle decryption and ripping, like AnyDVD or MakeMKV. I think it does one thing very well: transcode videos. Definitely not a Swiss army knife.

DrX
markfilipak
Bright Spark User
Posts: 239
Joined: Thu Aug 01, 2019 8:58 pm

Re: Judder-free, 60 FPS telecine (?)

Post by markfilipak »

Well, DrXenos, this 2-month effort by me has been a bust. ffmpeg's 'telecine' filter doesn't function as advertised. Let me tell you what this:
ffmpeg -i IN -vf "telecine=pattern=5555,bwdif=mode=send_frame" OUT
actually does.

Step 1: Decode to raw video frames. Of course, this is the 1st step in any transcode (and any operation that works on fields must transcode, even if the target's codec is the same as the source's codec).
Step 2: telecine=pattern=5555 does not do 5-5-5-5 pull-down to 60 FPS. It 'silently' does 4-6-4-6 pull-down, instead. Note that it outputs fields, not frames, but that's fine.
Step 3: bwdif=mode=send_frame apparently doesn't simply reweave the fields to frames. It apparently (and unexpectedly) decombs (which is unneeded & unwanted, even for the 2 out of 10 frames that are combed).
Step 4: Encode via x264 as the default -- that's fine and can be changed (e.g., '-c:v libx265').

I've quit the ffmpeg-user mailing list. The principals there don't care about my experiments. Some of them seem to not actually understand the 'mechanics' of video and see the process solely in terms of making calls to libraries. ffmpeg is somewhat undocumented (certainly the libraries are totally undocumented), and what documentation there is, is the worst I've ever seen.

I think I need to make my pitch for 5-5-5-5 pull-down telecine to 60 FPS to the folks who write the library functions, but I don't even know which library is used (undocumented by ffmpeg) and even if I did, I don't know how to contact them.
DrXenos
Bright Spark User
Posts: 251
Joined: Sat Mar 16, 2013 1:19 pm

Re: Judder-free, 60 FPS telecine (?)

Post by DrXenos »

Ok. I'm sorry for your troubles, but I don't know what any of that has to do with what I said.
mduell
Veteran User
Posts: 7327
Joined: Sat Apr 21, 2007 8:54 pm

Re: Judder-free, 60 FPS telecine (?)

Post by mduell »

markfilipak wrote: Fri Apr 03, 2020 6:46 amI've quit the ffmpeg-user mailing list. The principals there don't care about my experiments. Some of them seem to not actually understand the 'mechanics' of video and see the process solely in terms of making calls to libraries.
Yea I'm going to guess they're not the side with a lack of understanding...
markfilipak
Bright Spark User
Posts: 239
Joined: Thu Aug 01, 2019 8:58 pm

Re: Judder-free, 60 FPS telecine (?)

Post by markfilipak »

mduell wrote: Fri Apr 03, 2020 4:57 pm Yea I'm going to guess they're not the side with a lack of understanding...
What am I not understanding? I'm sincere. I know some things about video from my career -- I'm retired now -- but I don't know ffmpeg. So it's definitely conceiveable/probable that there's something that you think indicates I lack understanding when, in reality, I'm just ignorant of ffmpeg terminology or of how some filter actually works... details ...stuff like that.
markfilipak
Bright Spark User
Posts: 239
Joined: Thu Aug 01, 2019 8:58 pm

Re: Judder-free, 60 FPS telecine (?)

Post by markfilipak »

First, may I further praise HandBrake? Despite some functionality that I'd like added, HandBrake is the most useful, most 'solid' video tool I've found. All involved, give yourselves a pat on the back! You deserve it!
(HandBrake is able to plow through really f'ed up PTSs & DTSs and produce terrific transcodes. It's an amazing tool.)

I'm returning here following 2 months working with ffmpeg. With every intention to return here, I investigated ffmpeg merely for video preprocessing. (Note that my experiences with ffmpeg and the ffmpeg-users mailing list are probably best not mentioned.)

I understand that HandBrake is not an ffmpeg front-end. However, I believe HandBrake and ffmpeg use common libraries, so my experiences may profit the HB team.

As you may recall, I endeavored to create a new, better transcode for Blu-ray p24 & "NTSC" DVD p24 soft telecined video. I succeeded.
Below is the basic ffmpeg command as a Windows script. It assumes an environmental variable, 'AVOut', as the target path.
Notes:
- The frame number, 'n', is zero-based.
- 'telecine=pattern=5' does the telecining (P P C P P P P C P P ...) by interleaving (C)ombed frames into the (P)rogressive stream.
- The modulo descriminates between progressive pictures & combed pictures based solely on frame number (n = 2 7 12 17 ...).
- 'smartblur' is the only picture processor. (Note that all that 'smartblur' does is line-double the even lines & mix them into the existing odd lines, but that's just the right amount of decombing.)
- 'smartblur' is applied solely to the combed pictures (i.e., just 2 frames out of every 10 frames). The progressive pictures are not molested.

Code: Select all

REM Suitable for BD 'M2TS' files, I have a more complex version of this script that automatically handles DVD 'VOB's.
REM The source video's d:\path\name.ext are passed in as %1
SETLOCAL ENABLEEXTENSIONS ENABLEDELAYEDEXPANSION
FOR %%X IN (%1) DO SET OUT=%AVOut%\%%~nX.MKV
SET TELECINE=telecine=pattern=5,split[A][B]
SET CFRAMES=bitor(eq(mod(n\,9)\,2)\,eq(mod(n\,9)\,7))
SET PSELECT=[A]select='not(%CFRAMES%)'[C]
SET CSELECT=[B]select='%CFRAMES%',smartblur[D]
SET INTERLEAVE=[C][D]interleave
SET ENCODE=-map 0 -c:v libx264 -crf 20 -c:a copy -c:s copy
ffmpeg -i %1 -filter_complex "%TELECINE%,%PSELECT%,%CSELECT%,%INTERLEAVE%" %ENCODE% "%OUT%"
Here's a graph of the above:

Code: Select all

IN --> TELECINE (with 'split') --> PSELECT                    --> INTERLEAVE --> ENCODE --> OUT
                               --> CSELECT (with 'smartblur') -->
What I request: Creation of a 5-5-5-5 telecine mode (preset) based on the above processing.
Why it's worthy: There's no telecine judder when 24 FPS video is viewed on a 120Hz TV, but there is telecine judder on 60Hz TVs. The processing above eliminates that judder.
What I can do: I can fund the development, and I can participate in the development -- I am an experienced coder -- if someone holds my hand a bit.
Woodstock
Veteran User
Posts: 3937
Joined: Tue Aug 27, 2013 6:39 am

Re: Judder-free, 60 FPS telecine (?)

Post by Woodstock »

Feature requests should be made through Github.

However, the answer was posted 10 days ago:
JohnAStebbins wrote: Sun Mar 29, 2020 3:22 pm Since the telecine filter generates interlaced output, and HandBrake is a tool for generating progressive output (interlacing is an abomination) we won't be adding this filter to HandBrake.
Since your proposed "just a preset" depends upon the handbrake team adding the telecine filter to handbrake, and that idea has already been shot down by one of the team, it isn't worth going through the motions.

You have a working solution with ffmpeg, so what's the problem?
markfilipak
Bright Spark User
Posts: 239
Joined: Thu Aug 01, 2019 8:58 pm

Re: Judder-free, 60 FPS telecine (?)

Post by markfilipak »

Woodstock wrote: Fri Apr 10, 2020 2:50 am Feature requests should be made through Github.

However, the answer was posted 10 days ago:
JohnAStebbins wrote: Sun Mar 29, 2020 3:22 pm Since the telecine filter generates interlaced output, and HandBrake is a tool for generating progressive output (interlacing is an abomination) we won't be adding this filter to HandBrake.
Since your proposed "just a preset" depends upon the handbrake team adding the telecine filter to handbrake, and that idea has already been shot down by one of the team, it isn't worth going through the motions.
I have great respect for John Stebbins, but he's wrong in this case. Telecine produces frames, not fields. Some of the frames are combed, but some of the frames of the present telecine are combed. There's no difference. HandBrake has a decomb filter preset, does it not?

2-3 telecine from 24fps to 30fps produces 2 combed frames (2 of 5) that adjoin: P P C C P (40%, 1/15th second as a unit), and it judders: 2-3-2-3 at 12Hz.

4-6 telecine from 24fps to 60fps produces no combed frames but it judders: 2-3-2-3 at 12Hz.

5-5 telecine from 24fps to 60fps produces combing: P P C P P P P C P P, but it's less: 2 of 10 (i.e., 20%, 1/60th second each separated by 1/15th second of zero combing), and it doesn't judder. Though the frames don't temporally judder, the combing does produce a visual judder if it's not decombed.
You have a working solution with ffmpeg, so what's the problem?
Oh, gosh, where to begin... ffmpeg can't transcode all audio streams. ffmpeg does not handle PTSs & DTSs that do not monotonically increase -- it's the dependence of ffmpeg on time stamps (rather than frame numbers) that's its Achilles heel in my opinion; it's the source of many ffmpeg encodes that don't play, or play okay for a while then switch to 2x or more for no apparent reason, etc. There are many, many more. ffmpeg is fundamentally undocumented or the documentation is only skeletal, having lists of parameter names with no explanations regarding what the parameter names mean and what they do. And support for ffmpeg problems via ffmpeg-users is problematic -- help in the form of explanation is non-existent.

These are all handled seemlessly and elegantly by HandBrake.
Woodstock
Veteran User
Posts: 3937
Joined: Tue Aug 27, 2013 6:39 am

Re: Judder-free, 60 FPS telecine (?)

Post by Woodstock »

Wait - you can't use ffmpeg because it can't handle your audio requirements? And you think handbrake, which uses the same libraries, will magically fix that, just by adding in something the developers say will not be added?
User avatar
Rodeo
HandBrake Team
Posts: 12677
Joined: Tue Mar 03, 2009 8:55 pm

Re: Judder-free, 60 FPS telecine (?)

Post by Rodeo »

I doesn't even matter whether it will be added or not, since currently there is neither a pull request nor even the beginning of a patch.
markfilipak
Bright Spark User
Posts: 239
Joined: Thu Aug 01, 2019 8:58 pm

Re: Judder-free, 60 FPS telecine (?)

Post by markfilipak »

Woodstock wrote: Fri Apr 10, 2020 12:33 pm Wait - you can't use ffmpeg because it can't handle your audio requirements? And you think handbrake, which uses the same libraries, will magically fix that, just by adding in something the developers say will not be added?
It's not just audio. It's video too. The problems may not be/probably aren't with the libraries. After all, HandBrake works great using the same libraries -- at least, I assume they're the same libraries.

For example, an ffmpeg 5-5-5-5 pull-down video works wonderfully if the video's short, but at 35:47 it spontaneously switches to fast forward. It appears to happen with every transcode.

Give me a clue, will you?

Why does everyone seem so apathetic about this development? I thought you folks would be excited: Something new & better under the sun. But you all seem to be trying to slam on the brakes.

What gives?

PS: When I wrote "play okay for a while then switch to 2x or more for no apparent reason" a day ago, I had not made many long 5-5-5-5 telecines. Now, I have. The fast foward switch happens at 35:47 every time I have tried to do 5-5-5-5 telecine with a movie. If nothing else, even if the 5-5-5-5 method is crap -- it's not -- switching to fast forward like that may be a clue to a previously undiscovered bug.
Lostless
Regular User
Posts: 77
Joined: Thu May 24, 2007 7:55 pm

Re: Judder-free, 60 FPS telecine (?)

Post by Lostless »

Forgive my ignorance, but I don't get the math of a 5555 pulldown on a 60FPS video. In a 60FPS video holding a 24FPS movie, 3:2 pull down is the only math that makes sense. You need to hold 4 frames of film for every 10 frames of video. The only way is 3+2+3+2=10 and keeping looking as smooth as possible. 5555 is 5+5+5+5=20. It would only hold 2 frames of film for every 10 frames of video. Am I missing something here?
Lostless
Regular User
Posts: 77
Joined: Thu May 24, 2007 7:55 pm

Re: Judder-free, 60 FPS telecine (?)

Post by Lostless »

From what i'm reading as I look what the OP wants to do is basically create a fake frame that's a combination of 2 frames for 1/60th every 2 frames/10. Basically create your own interpolated frame by using combing and blend interlacing. Ingenious, but is not a true representation of the original films. Basically you are looking at a 2.5:2.5 ratio and taking the 2 .5s and making a new frame. Do have this right?
markfilipak
Bright Spark User
Posts: 239
Joined: Thu Aug 01, 2019 8:58 pm

Re: Judder-free, 60 FPS telecine (?)

Post by markfilipak »

Lostless wrote: Tue May 05, 2020 6:22 pm From what i'm reading as I look what the OP wants to do is basically create a fake frame that's a combination of 2 frames for 1/60th every 2 frames/10. Basically create your own interpolated frame by using combing and blend interlacing. Ingenious, but is not a true representation of the original films. Basically you are looking at a 2.5:2.5 ratio and taking the 2 .5s and making a new frame. Do have this right?
You have the essential idea but I don't know why you write "not a true representation". How not?

It's called 55-telecine. The result is p60 (or p60/1.001). Here's a comparison of p60 55-telecine with the much more familiar p30 23-telecine.

Code: Select all

|<--------------------------1/6s-------------------------->|
[A/a__________][B/b__________][C/c__________][D/d__________] p24 Source
[A/a_______][B/b_______][B/c_______][C/d_______][D/d_______] p30 23-telecine
[__________][__________][XXXXXXXXXX][XXXXXXXXXX][__________] Combed frames
[A/a_][A/a_][A/b_][B/b_][B/b_][C/c_][C/c_][C/d_][D/d_][D/d_] p60 55-telecine
[____][____][XXXX][____][____][____][____][XXXX][____][____] Combed frames
As you probably see, 23-telecine results in 40% combing. It also has significant telecine judder.
In contrast, 55-telecine results in 20% combing (which is distributed, not concentrated) and has much less telecine judder.

A person who has a 24Hz TV should watch the p24 Source. A person who has a 120Hz TV should also watch the p24 Source. But a person who has a 60Hz TV is much better off with the 55-telecine. The worst thing possible is watching the 23-telecine that the TV then frequency doubles.

Unfortunately, the HandBrake developers don't want to develop this approach.
Last edited by markfilipak on Tue May 05, 2020 7:09 pm, edited 1 time in total.
markfilipak
Bright Spark User
Posts: 239
Joined: Thu Aug 01, 2019 8:58 pm

Re: Judder-free, 60 FPS telecine (?)

Post by markfilipak »

PS: I have done 55-telecine via ffmpeg.

Code: Select all

ffmpeg -i IN -filter_complex "telecine=pattern=5, split[A][B], [A]select='not(eq(mod((n+1)\,5)\,3))'[C], [B]select='eq(mod((n+1)\,5)\,3)',pp=linblenddeint[D], [C][D]interleave" -map 0 -codec:v libx264 -crf 18 -codec:a copy -codec:s copy -dn OUT
The result is gorgeous.
Lostless
Regular User
Posts: 77
Joined: Thu May 24, 2007 7:55 pm

Re: Judder-free, 60 FPS telecine (?)

Post by Lostless »

markfilipak wrote: Tue May 05, 2020 6:57 pm
You have the essential idea but I don't know why you write "not a true representation". How not?
Because there is no mathematical way to represent 24 frames into 60 evenly without juter through 6:4 pulldown. You need to add and create frames not in the source material as an in between . And the only way to add these in between frames is to interpolate them by whatever method you would use from the frames before and after. But those new frames to add smoothness are not in the original film. They are artificially created. Not unless your goal is to have a Veritable frame rate jump to 120FPS for a few frames every second (that a 60hz ststem wont show anyway), Its just not possible in a 60Hz system.
Unless i'm still confused with all of this?
Lostless
Regular User
Posts: 77
Joined: Thu May 24, 2007 7:55 pm

Re: Judder-free, 60 FPS telecine (?)

Post by Lostless »

I understand the combed frames. But what to do with them in a progressive system. I can A. Leave as is and have combing artifacts few times a second. B. Blur them. There's one form of an interpreted frame. C. Create a new frame based of the average motion of the frame in front and back, a "smart" interpolation. or D. remove one field, which goes right back to 3:2 pulldown jitter.
markfilipak
Bright Spark User
Posts: 239
Joined: Thu Aug 01, 2019 8:58 pm

Re: Judder-free, 60 FPS telecine (?)

Post by markfilipak »

Lostless wrote: Tue May 05, 2020 7:55 pm
markfilipak wrote: Tue May 05, 2020 6:57 pm You have the essential idea but I don't know why you write "not a true representation". How not?
Because there is no mathematical way to represent 24 frames into 60 evenly without juter through 6:4 pulldown.
I draw pictures (timing diagrams, actually) as I drew in the previous postings. Looking at the combed frames that 23 pull-down insert (i.e., the 3rd & 4th frames) and correlating the boundaries of those frames versus the boundaries of the original 24 FPS source, I realized that a different pull-down would work better. I drew what I wanted and 'discovered' 55 pull-down.
You need to add and create frames not in the source material as an in between .
23-telecine to p30 or my 55-telecine to p60 create combed frames. 46-telecine does not create combed frames, but the judder is awful.

Code: Select all

|<--------------------------1/6s-------------------------->|
[A/a__________][B/b__________][C/c__________][D/d__________] p24 Source
[A/a_______][B/b_______][B/c_______][C/d_______][D/d_______] p30 23-telecine
[__________][__________][XXXXXXXXXX][XXXXXXXXXX][__________] Combed frames
[A/a_][A/a_][A/b_][B/b_][B/b_][C/c_][C/c_][C/d_][D/d_][D/d_] p60 55-telecine
[____][____][XXXX][____][____][____][____][XXXX][____][____] Combed frames
[A/a_][A/a_][B/b_][B/b_][B/b_][C/c_][C/c_][D/d_][D/d_][D/d_] p60 46-telecine
[____][____][____][____][____][____][____][____][____][____] Combed frames
And the only way to add these in between frames is to interpolate them by whatever method you would use from the frames before and after. But those new frames to add smoothness are not in the original film. They are artificially created.
Of course they're created. That's what telecine does.
Not unless your goal is to have a Veritable frame rate jump to 120FPS for a few frames every second (that a 60hz ststem wont show anyway), Its just not possible in a 60Hz system.
Unless i'm still confused with all of this?
You can't do that. Even if you could, it would judder like crazy -- 60FPS that jumps to 120Hz every few frames? You would upchuck at the result.
markfilipak
Bright Spark User
Posts: 239
Joined: Thu Aug 01, 2019 8:58 pm

Re: Judder-free, 60 FPS telecine (?)

Post by markfilipak »

Lostless wrote: Tue May 05, 2020 8:11 pm I understand the combed frames. But what to do with them in a progressive system. I can A. Leave as is and have combing artifacts few times a second.
Yes, that's right. Every 1/12th second, there's a 1/60th second combed frame. It's so brief, it's almost invisible.
B. Blur them.
Yes, you can do that and it's even better. In the ffmpeg command line I posted earlier, 'pp=linblenddeint' is just such a technique (not blur but linear blend deinterlace).
There's one form of an interpreted frame. C. Create a new frame based of the average motion of the frame in front and back, a "smart" interpolation.
That's motion compensation. Of course I'd do that if ffmpeg would do it.
or D. remove one field, which goes right back to 3:2 pulldown jitter.
That, you cannot do. If you remove a frame -- you wrote "field", but of course you meant "frame" -- then you wouldn't get 60 FPS.

PS: Draw pictures. It helps.
Lostless
Regular User
Posts: 77
Joined: Thu May 24, 2007 7:55 pm

Re: Judder-free, 60 FPS telecine (?)

Post by Lostless »

No, I really meant field. A combed/interlaced image has 2 fields making one frame. Each field a different moment in time.
You are left with interlaced/combed frames problem in a progressive system. That is bad. You have to do something with them.
If want to add artificial frames or combing into the final output because you like the look, by all means use it, but I don't find any need to have handbrake add it for such a minor and niche use.
markfilipak
Bright Spark User
Posts: 239
Joined: Thu Aug 01, 2019 8:58 pm

Re: Judder-free, 60 FPS telecine (?)

Post by markfilipak »

Lostless wrote: Tue May 05, 2020 9:09 pm No, I really meant field.
Try to draw a picture of that. I think you'll see the problem with it.
A combed/interlaced image has 2 fields making one frame. Each field a different moment in time.
You are left with interlaced/combed frames problem in a progressive system.
First, all frames are progressive. Combing is not interlace. Interlace is for NTSC/PAL/DVB broadcast video.
Second, yes, different moments. In the case of p24 sources, 1/24th second.
That is bad. You have to do something with them.
All telecines synthesize an extra frame every 5th frame. If you have a 60Hz TV and you feed it 24FPS, then the TV telecines the video stream. But a TV will do either 46-telecine to 60FPS, or it will do 23-telecine to 30FPS and double it to 60FPS. 55-telecine is better.
If want to add artificial frames or combing into the final output because you like the look, by all means use it, but I don't find any need to have handbrake add it for such a minor and niche use.
Okay.
Locked