Closed captions / subtitles

Archive of historical feature requests.
Please use the GitHub link above to report issues.
Forum rules
*******************************
Please be aware we are now using GitHub for issue tracking and feature requests.
- This section of the forum is now closed to new topics.

*******************************
kneeslasher
Experienced
Posts: 85
Joined: Tue Mar 06, 2007 8:40 pm

Post by kneeslasher »

eddyg wrote:
human wrote:
rhester wrote:That wasn't what I was saying.
I don't think I've seen any DVDs that have Closed Captions
Literally all R1 DVDs have CCs. Sometimes with no subs, just CCs.
human
Posts: 4
Joined: Wed Aug 08, 2007 8:12 am

Post by human »

eddyg wrote: HandBrake extracts the MPEG2 video stream and passes that through a decoder to get raw video. That is where the CC information is no doubt being lost.

Thanks for responding. That seems to be the crux of it. If Handbrake is basically a front end to a black box decoder that doesn't offer the option of retaining the CC data, then game over.
Your kung fu is far superior to mine but it strikes me as incorrect to reference an Australian Hi-Def TV specification with regard to this issue. The CC I'm thinking of is an NTSC specification.
eddyg
Veteran User
Posts: 798
Joined: Mon Apr 23, 2007 3:34 am

Post by eddyg »

human wrote:
eddyg wrote: HandBrake extracts the MPEG2 video stream and passes that through a decoder to get raw video. That is where the CC information is no doubt being lost.

Thanks for responding. That seems to be the crux of it. If Handbrake is basically a front end to a black box decoder that doesn't offer the option of retaining the CC data, then game over.
It's more of a grey box, we have the source, and we can patch it as well as push changes upstream.
human wrote:
Your kung fu is far superior to mine but it strikes me as incorrect to reference an Australian Hi-Def TV specification with regard to this issue. The CC I'm thinking of is an NTSC specification.
The ozzies are references and explaining the same digital CC encoding standards as used within NTSC. They are reusing these for their HD broadcasts. It was a [Censored] finding a decent explanation of the standard saying exactly how the CCs were encoded in the MPEG2 video stream.

Cheers, Ed.
wasafiri
Posts: 8
Joined: Wed Jul 18, 2007 2:32 pm

Post by wasafiri »

If it's of any help, the guy who developed DVD Shrink told me that the closed captions are stored in the user_data block of each GOP header in the mpeg stream. He enabled support for captions by stopping the program from stripping the information in that block.
donaciano
Posts: 3
Joined: Mon Sep 24, 2007 5:10 am

Post by donaciano »

eddyg wrote: The ozzies are references and explaining the same digital CC encoding standards as used within NTSC. They are reusing these for their HD broadcasts. It was a [Censored] finding a decent explanation of the standard saying exactly how the CCs were encoded in the MPEG2 video stream.

Cheers, Ed.
Maybe the code to ccextractor would help. It's GPL and pretty small. Snip lil pieces of it and convert the text it grabs to a QT Text Track format or whatever they are using for their CC encoding and you're done.

Then all the deaf people will be saying "If a free program can do it, why can't Apple update their iTunes videos?" ;-)

Seriously your work would be VERY appreciated by deaf guys like me.
-Don.
wasafiri
Posts: 8
Joined: Wed Jul 18, 2007 2:32 pm

Post by wasafiri »

eddyg wrote:
kneeslasher wrote:The newest iTunes lends pertinent context to this thread.
How?
The newest iTunes supports closed captions.
donaciano
Posts: 3
Joined: Mon Sep 24, 2007 5:10 am

CC support

Post by donaciano »

Not only iTunes but also the iPhone 1.1.1 firmware. I really wish there was more interest in CC around here. They're totally superior to subtitles in every way.

CC can be configured to whatever fonts you want, can be turned on and off, can be searched through since they're just text.

With CC you could come up with a Spotlight plugin that would let you search for "You can't handle the truth" then pop open your "A Few Good Men" at just the spot where Jack Nicholson gives that line.

You could count the number of times a particular word was spoken or find out the name of that song that was playing. Closed Captions usually include names of songs and sounds like [gunshot] so you could find the action sequences quickly.

For the compression quality freaks here... CC won't permanently be burned in like subtitles allowing better compression bitrates.

It's just all the advantages of an .srt file but it's built in and has support from Apple, they already have sample code on their developer site. Deaf guys like me would finally be able to rip our DVDs that only have CC and no subtitles. Things like the Twilight Zone original series, etc...

-DC
eddyg
Veteran User
Posts: 798
Joined: Mon Apr 23, 2007 3:34 am

Post by eddyg »

Hi,

The biggest issue with closed captions is that they are not on most of the DVDs that we are encoding.

So for those that do have CC's on them we could extract them, if we can figure out how to persuade our MPEG 2 decoder to extract them (it does not currently).

For the movies that don't have CC's we have two choices:

1) Allow importing and encoding of .srt files supplied independently
2) Bundle OCR software with HandBrake that can "read" the subtitle bitmaps and provide the subtitles for verification and editing by the user.

Solution 1 is possible, but I don't see us providing the OCR capability when this is better served by external applications, which would then provide the .srt file that we could use with (1).

Recently I've been working on the subtitles, and I do have some NTSC material with CCs, but because the burned in subs work fine for me (e.g. for Kill Bill I don't understand Japanese, and will likely never understand Japanese it's not an issue) I have only briefly looked at extracting the CCs.

Maybe we could use something like ccextractor (http://sourceforge.net/projects/ccextractor/) to extract the cc's and then use some other tools to convert that to text and then figure out how to encode that into the MP4.

However, that's a lot of effort, mostly in learning new formats and tools, and I guess none of us are that motivated or have the time for that sort of commitment (sorry) to get all that done.

Cheers, Ed.
kneeslasher
Experienced
Posts: 85
Joined: Tue Mar 06, 2007 8:40 pm

Post by kneeslasher »

McPoodle's tools could handle CC extraction and conversion to srt files: I used them years ago without issues. Given the ease with which it was possible, I'd prefer developer interest in this to focus solely on using srt files as input for encoded files as CCs (i.e., Option 1).

Apart from the huge utility to deaf people, this is incredibly useful when watching films with friends for whom English isn't a first language: oen doesn't have to keep pausing the film to explain what's going on!
dynaflash
Veteran User
Posts: 3820
Joined: Thu Nov 02, 2006 8:19 pm

Post by dynaflash »

Okay, I think we all understand there is a demand for cc's , but as eddyg has politely tried to say, non of the current devs have the time to get into it. The demand is noted, and hopefully a developer will come along that wants to implement it and has the time and know how to contribute. I really dont think that more reasons to add it will get it done any sooner.

If its not on the trac, I will add it for down the road and hopefully someone will come along to implement it.
jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Post by jbrjake »

dynaflash wrote:If its not on the trac, I will add it for down the road and hopefully someone will come along to implement it.
Soft subs are on the Trac.
kneeslasher
Experienced
Posts: 85
Joined: Tue Mar 06, 2007 8:40 pm

Post by kneeslasher »

Great to hear :).
gauvins
Posts: 9
Joined: Thu Nov 22, 2007 3:39 am

Post by gauvins »

If I remember well, HB offers the option to "burn" subtitles over the video track.

I've never used it. I prefer to hunt for an .srt file, use titlelab to set font type, size and color, and quicktime pro to add a text track that:

o can be displayed below the original video
o switched on and off at will

--

As I see it, the problem for a Mac owner is that there is no easy solution to extract CC, so one has to rip and keep watching for the VOB source instead of having the freedom to recompress / edit the file.

Maybe Hanbrake could merely begin with the integration of ccextractor (http://sourceforge.net/projects/ccextractor/) as mentioned by eddyg.

Previous posters have indicated that CC (when available) are superior to subtitles. Others have indicated that for US broadcasts, CCing is mandatory.

I would certainly appreciate the option of generating a data file (srt) out of the CC stream.
kneeslasher
Experienced
Posts: 85
Joined: Tue Mar 06, 2007 8:40 pm

Post by kneeslasher »

For Region 1 DVDs, the CCs are already sitting there, so I would hope Handbrake will, some day, be able to create an m4v file with a CC track taken straight off the DVD. Quicktime/iTunes already have the functionality to display such CCs (on/off) as do the new iPods for external output.

I wonder how difficult it is to add a CC track onto an existing m4v file?
kneeslasher
Experienced
Posts: 85
Joined: Tue Mar 06, 2007 8:40 pm

Post by kneeslasher »

I'd prefer developer energy went towards getting Handbrake to create an m4v with audio, video *and* the CC track. Getting CCs off a DVD and into an srt file is a non-issue since (free) tools already exist to do just that. It is creating a single file with the new Apple-supported CCs which is the first issue needing work.
jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Post by jbrjake »

however i am saddened that the devs at MTR & at handbrake both regard language processing as a trivial problem!!!
excellent reasons for the devs at handbrake (and mtr) re-evaluating the priority of CC
It's good to see that there is some demand for Closed Caption support. Not, I fear, enough though.
I also understand they have to prioritize their efforts.

But I want cast my vote
I hope those involved in the Handbrake project reassess its value.
I do, however, hope that adding CC hard-encode support to Handbrake can be put on a to-do list for further development.
For developers, this is a question of quality v. quantity.
I'd prefer developer energy went towards
You all seem to be under the misapprehension that getting this feature in HB is a matter of forcing the developers to believe that there's user demand and twisting our arms to put it on the to-do list. As if, by declaring it important, it will magically get coded.

But putting it on the to-do list achieves nothing. How do I know this? Because I put container-level soft-sub support on the to-do list *before this thread was even started*.

A lot of users seem to have this delusion that everyone working on HandBrake can do anything from reverse-engineering proprietary containers to writing drivers for closed source hardware to implementing container-level support in mpeg4ip for Apple's brand-new CC format. That the only thing preventing us from tackling this is that it's not been given sufficient "priority." This is not the case. HandBrake is not a commercial project. People do not work on things that are not personally important to them.

This will happen when someone comes along who knows how to do it and is personally motivated to implement it, not by, as one person in this thread put it, "harassing" the current developers.

I'm not sure how many times this can be explained in one thread, so here's a recap:
rhester wrote:I won't say never - but I feel pretty comfortable saying "no time soon".
rhester wrote:If, on the other hand, you'd like to code the necessary patches to add this support to HandBrake, we'd welcome that. =)
rhester wrote:The reason open source software works is because people are motivated to contribute out of self-interest and "fun" rather than monetary gain.

What this means, in practice, is people tend to code and contribute features that most interest them. Since this interests you, probably moreso than any of the current developer staff, it's a natural response to invite you to help move the application forward via code contribution.

If you are unable to contribute code, of course, it somewhat dilutes the prospects of it being included anytime soon. Wink

(This is my not-so-subtle way of trying to encourage you - or anyone else interested in a specific "niche feature" - to either take a look at the source yourself or bug a local programmer friend to do so to increase our developer pool and benefit everyone.)
rhester wrote:The real issue is that none of our current "staff" knows how to extract the captions from the DVD, nor how to convert them from text form (as stored in the VOB stream) to a text overlay on the HandBrake video output stream.

Being an open-source project, there is no static HandBrake developer team - programmers come and go as they please, most often to work on/add a feature that happens to interest them specifically. Unless/until a developer not currently on the team comes along with the technical knowledge required to implement this feature, there's nothing the existing group can do (short of appealing to the larger open-source development community for these skills - which, ipso facto, happens each time someone makes this request).

I understand your need, and if there was something I could personally do to bridge the gap, I would. Unfortunately, this is a matter of "eventually someone will come along and solve the problem, but we don't know who, and we don't know when".
eddyg wrote:So I personally don't feel like implementing it as I get no benefit from it.
eddyg wrote:However, that's a lot of effort, mostly in learning new formats and tools, and I guess none of us are that motivated or have the time for that sort of commitment (sorry) to get all that done.
dynaflash wrote:Okay, I think we all understand there is a demand for cc's , but as eddyg has politely tried to say, non of the current devs have the time to get into it. The demand is noted, and hopefully a developer will come along that wants to implement it and has the time and know how to contribute. I really dont think that more reasons to add it will get it done any sooner.
Capisce?
kneeslasher
Experienced
Posts: 85
Joined: Tue Mar 06, 2007 8:40 pm

Post by kneeslasher »

I think we all understand, quite axiomatically, that our screaming <> developer work. However, since we have the hope that *some* day, our desire for CCs may be addressed, I'm sure the posters in this thread were simply interested in getting our views down so when/if the day rolls round that work *is* done on the CC issue, the developers have the HB community's thoughts down for reference, if the inclination for consultation then exists.
rhester
Veteran User
Posts: 2888
Joined: Tue Apr 18, 2006 10:24 pm

Post by rhester »

With all due respect...in this case, I don't think that's the case at all.

There aren't too awfully many ways to implement closed captions - it's either as burned-in subs or Apple-compatible closed captions. Clearly, the latter is the preferred technical direction, as the two solutions were essentially "made for each other".

As such, end-user feedback doesn't really apply here, because we're not looking for suggestions on how it should be done - we have the same limitations as described before, which are a) time and b) technical knowledge.

As we've stated that many times - and the request keeps being repeated - it can only be assumed that jbrjake's perspective was accurate and that this has been an extended effort to "harangue" the developers into acting on something they a) consider somewhat low priority and b) simply don't yet know how to do.

Honestly, the discussion has gone on long enough - everyone should now be more than clear on the respective positions of the community and the developers, and they'll get to it when it gets gotten to. =) I'll refer any future requests for this feature back to this thread for clarity, but I don't think there's much more that can be usefully added to this thread.

Rodney
gauvins
Posts: 9
Joined: Thu Nov 22, 2007 3:39 am

Post by gauvins »

a suggestion maybe, as there appears to be some kind of disconnect between those who think a feature would be cool and those who make cool features:

could we pool interested users and create a "special interest" development fund?

The idea (see kiva.org or sellaband.com for variants) is to have people pledging small amounts of money for specific actions.

I for one do not like the donation-at-large model. But I would pledge $20 for a CC extraction feature. That should be easy to do since CCextractor, mentioned earlier in the thread, does the trick for computer-fluent users (who know how to compile binaries and run UNIX command-line applications)

I perfectly understand that an OSS project like handbrake is developers driven. I am actually amazed at the quality of the end product. It requires vision and dedication and I certainly understand that the very notion of "requests" has to be extremely annoying.

But I believe that it is in the interest of both users and developers to figure out a way to resolves situations like this one.

If someone wants to work on something like this, let me know how I can be of any help.
nightstrm
Veteran User
Posts: 1887
Joined: Fri Mar 23, 2007 5:43 am

Post by nightstrm »

While I'm not a dev, I think I can answer this one. The HandBrake dev team does not accept donations of hardware, software, or basically anything of monetary value. However, they will gladly accept donations of source code that adds functionality to HB and fits within the scope of the program and vision of the team.

8)
dynaflash
Veteran User
Posts: 3820
Joined: Thu Nov 02, 2006 8:19 pm

Post by dynaflash »

gauvins wrote:But I believe that it is in the interest of both users and developers to figure out a way to resolves situations like this one.
*sigh* With all due respect gauvins and company, please understand that this is *not* going to help. We do not accept donations as per this sticky http://handbrake.m0k.org/forum/viewtopic.php?t=389 . I am inclined to go into a lengthy explanation why, but suffice it to say others have stated it better than I could. Please save your money. There is no "situation to resolve". It is what it is right now.

Please dont make me lock this thread as maybe a dev will come along and have a. the interest and b. the technical expertise and c. the time to implement what you are looking for.

nightstrm is spot on.
kneeslasher
Experienced
Posts: 85
Joined: Tue Mar 06, 2007 8:40 pm

Post by kneeslasher »

Fellow non-developers: let's all discreetly shut up lest our posting is taken as whining. There's nothing to be done here and we're better off discussing other HB issues rather than bringing down ire on this. I think we've all proved we're avidly agog for CCs.
cfsmp3
Posts: 3
Joined: Wed Nov 28, 2007 5:18 pm

Post by cfsmp3 »

nightstrm wrote:While I'm not a dev, I think I can answer this one. The HandBrake dev team does not accept donations of hardware, software, or basically anything of monetary value. However, they will gladly accept donations of source code that adds functionality to HB and fits within the scope of the program and vision of the team.
Someone pointed me to this thread (I'm ccextractor's author).

Not needed since ccextractor is GPL and it has parts from other projects (such as MythTV), but feel free to use whatever code you need from it.

Also, if someone decides to go for it I'd be willing to help as much as possible (I don't have time to code it myself though).
kneeslasher
Experienced
Posts: 85
Joined: Tue Mar 06, 2007 8:40 pm

Post by kneeslasher »

I'm hoping this comment is actually useful/informative:

http://www.cpcweb.com/whats_new/#Captio ... d%20iPhone

This seems to be software which takes an *already* encoded m4v and a prexisting CC file, and simply zips them together. Now where's that $9000 I left lying around...
gauvins
Posts: 9
Joined: Thu Nov 22, 2007 3:39 am

Post by gauvins »

I was able to extract CCs with ccExtractor and add them to an MP4 rip. Everything is opensource except for quicktime PRO.

1) join the VOB files into a single VOB file (I use D-vision tools http://www.objectifmac.com/)

Joining VOB files is "required" because otherwise CCextractor will reset the time to 0 at the beginning of each VOB

2) extract the CC from the large VOB with CCextractor http://ccextractor.sourceforge.net/

As there are no binaries of ccextractor for Mac, I've run the PC version. the interface is a command line. Not a major problem as is no need to specify multiple options.

ccextractortowin.exe -srt movepath/movie.VOB

3) adjust the content of the srt file (ex. font face, size, color / window width and height) with titlelab and save the file for quicktime
http://www.versiontracker.com/dyn/moreinfo/macosx/13374

4) create a texttrack with quicktime : http://www.apple.com/quicktime/tutorial ... racks.html

good luck
Locked