[Committed] BD PGS Subtitles

[JohnAStebbins]

I've started working on this. So far, I only have code to detect their presence in a transport stream and demux them. title->list_subtitle is set up so they can be listed in the UI. My plan is to do decoding next so they can be burned into video. Probably won't have time to start that till next weekend.

Here's the patch so far: http://handbrake.fr/pastebin/pastebin.php?show=1446

[mkelley]

That's great news, John.

Is burning going to be the only option? I'd rather burn only those that are forced, and have the other track(s) available to turn on or off. In any case, thanks for this work.

[JohnAStebbins]

I would like to do passthru and maybe transcode to vobsub as well. But burn is first because the process will flesh out most of the code that will be needed for other usages. Also, since mp4 doesn't support PGS subs, the initial implementation will suck/work equally for everyone.

[mkelley]

Yeah, I don't really want PGS subs either (my playback unit won't recognize them but needs IDX in the MKV container), so if you pass them through I'll still have to convert them (right now I'm using BDSup2Sub for converting after I mux them out with tsMuxer, and then I remux them back into the Handbrake encoded video with mkvmerge).

I hope, though, that your burn will at least recognize the difference between forced and unforced (because it *would* be nice to burn the forced subs, which is something I can't do now with my workflow). While there aren't that many movies with them, what there are are definitely needed.

[JohnAStebbins]

One step closer http://handbrake.fr/pastebin/pastebin.php?show=1462
Added a decoder for pg subs. It accepts and parses segments of pg data. Then renders to an hb_buffer_t. It parses palette data, but does not yet do anything with it. Every PG subtitle can have it's own palette of 256 32 bit entryes (Y, Cr, Cb, A). So we will need to pass the palette with every subtitle buffer.

I've only verified that it compiles. I've done zero testing.

[davidfstr]

Looks like a good start. Unfortunately I can't do any testing because I have neither BluRay source material nor a drive.

Some additional changes will be needed:

• Update work.c around line 320 to return a descriptive name for the subsource.
• Update work.c around line 790 to create your new decpgsub work-object at the appropriate place in the pipeline.

You may run into other unusual issues since this new work-object is likely going to be used in a way similar to the existing decvobsub work-object, which is handled semi-specially by various parts of the encoding pipeline, when compared with all other types of subtitle tracks.

Also, be careful that you're passing through the sequence number of the original packets within your work-object, since it is outputting renderable PICTURESUBs. I didn't check to see whether you were already doing this or not.

[JohnAStebbins]

hehe. I wrote a significant amount of the existing subtitle code, so I know where all the skeletons lie. I did add the sequence number to the PG sub buffers, but it is only used in sync to decide when it is time to forward a subtitle on to the renderer. I believe this is old crufty code that isn't needed anymore. We should just be basing this decision on the pts values (buf->start).

I have other problems to solve however. There can be multiple 'compositions' on the screen at once. A composition is a rendering of a bitmap to a specific window in the video frame. I'm rendering each composition into it's own hb_buffer_t. This means that I need to pass multiple subtitle buffers at once to render. So there will be special case code in sync to make this happen.

Another problem I have is that PG subs don't have explicit durations like vobsubs have. They end when the next pg starts. To remove subtitle from the screen, they have to insert an empty composition. We can't wait for the next subtitle before we put the current subtitle into the pipeline because a subtitle needs to make it to the renderer before it's corresponding video frame. So the subtitle must be put into the pipeline with a long initial duration. Then sync will have to handle terminating the subtitle at the proper time.

And as I mentioned, to remove a subtitle from the screen, an empty subtitle is sent. We currently use a hb_buffer_t with size 0 to indicate that we have reached the end of a stream. For rendered subs there is a special condition that simply drops the buffer when it's size is 0, but it bothers me a bit to rely on this behaviour when everywhere else, size 0 has a different meaning.

[eddyg]

hehe. I wrote a significant amount of the existing subtitle code, so I know where all the skeletons lie. I did add the sequence number to the PG sub buffers, but it is only used in sync to decide when it is time to forward a subtitle on to the renderer. I believe this is old crufty code that isn't needed anymore. We should just be basing this decision on the pts values (buf->start).

Correct - I added it when we were having issues with the PTS on DVDs jumping all over the place causing infinite loops and the like. That code has since been revamped significantly and I believe we can ditch the sequence number now. Although it is kinda nice to use now and then.

Another problem I have is that PG subs don't have explicit durations like vobsubs have. They end when the next pg starts. To remove subtitle from the screen, they have to insert an empty composition. We can't wait for the next subtitle before we put the current subtitle into the pipeline because a subtitle needs to make it to the renderer before it's corresponding video frame. So the subtitle must be put into the pipeline with a long initial duration. Then sync will have to handle terminating the subtitle at the proper time.

And as I mentioned, to remove a subtitle from the screen, an empty subtitle is sent. We currently use a hb_buffer_t with size 0 to indicate that we have reached the end of a stream. For rendered subs there is a special condition that simply drops the buffer when it's size is 0, but it bothers me a bit to rely on this behaviour when everywhere else, size 0 has a different meaning.

Agreed - pretty yuck. Sounds like maybe a subtitle buffer needs an "instruction", i.e. start now, stop now, display for duration etc.

Cheers, Ed.

[mkelley]

You guys are probably far more familiar with it than I am, but the freeware BDSup2Sub will correctly convert PGS subs and the display acts "right" as far as I'm concerned. Since it outputs text file instructions you could always look at those and see how it handles the issues you mention (I use it now in my program flow -- demux the PGS from the m2ts file, run it through BDSup2Sub, and remux the resultant idx/sub file back into the Handbrake encoded MKV file. I'm hoping that perhaps all you folks wonderful work will eliminate all these steps).

[Deleted User 11865]

You guys are probably far more familiar with it than I am, but the freeware BDSup2Sub will correctly convert PGS subs and the display acts "right" as far as I'm concerned.

Apparently it's also free software as of version 4.0. The changelog says: Changed: BDSup2Sub is now open source (see integrated HTML help for details about repository and license)

The jar won't launch under Leopard and the html files it contains don't seem to reference a license, but the jar does contain the Apache License and the LGPL v3.

http://www.videohelp.com/tools/BDSup2Sub

[boer]

Any chance we will see working support for BD subs on the nightlies soon?

[JohnAStebbins]

If by soon you mean some time this year. My attention got diverted elsewhere for a while.

[firewing1]

John, do you think this will be finished within a few months? I'm debating if I should wait a few months for transcoding Lost or if I should go ahead for now without subtitles and re-transcode later when support is added.

BTW thanks for the great job on BD support, that has been a huge help!

[JohnAStebbins]

John, do you think this will be finished within a few months?

Not unless someone else decides to do it.

[RoverTX]

btw as an update XBMC, as of 4 days ago, supports PG subtitles

http://trac.xbmc.org/ticket/10538

for some reason the FFMPEG patch that didn't work all that well at playing PG subtitles correctly doesn't seem to screw them up in XBMC when applied to FFMPEG inside XBMC from the testing of the members on the XBMC board (from my noob understanding). So the XBMC devs applied it and it seems to be working out perfect in the current XBMC beta.

@Rodeo 12/19/2010

Was just downloading BDSup2Sub for a new boot and found the page that has info on the source code

http://bdsup2sub.javaforge.com/help.htm#Source_Code

that page has the link and username and password to get bdsup2sub source code incase anyone is interested.

[eliehass]

Is there an update on PGS subtitles coming to Handbrake (at least on the nightlies for now)?

[JohnAStebbins]

Nobody is working on this yet.

[compudude]

Any updates on this? (hope-hope)

[s55]

Nobody is working on this yet.

[arovuo]

Guys,

I just learned that VLC and WD Live TV both do support PGS streams as such within .mkv !
I also understand that ffmpeg supports PGS as well.

I used [Link and Product Name Removed.] and it produced .mkv with

• 1 x Video Stream AVC
  7 x Audio streams DTS/AC3
  12 x text streams PGS

On that media file/.mkv I can switch audio and subtitle in WD Live TV and VLC.

So, I wonder what would it take to just passthru the PGS into .mkv, like {Removed} does ?
Current nightly does not even list BD subtitles, although I know they do exist on the disc / playlist.

[s55]

It's not a technical issue. It's a time / interest issue.

This is open source, not a commercial venture and as such, there is no set date/time for any feature. Features will appear when someone who is interested, contributes a patch.
So, it could be tomorrow, it could be 2 years, it could be never.

Constantly asking "Is it ready yet", or any "Me too" posts will not make this happen any faster. User demand is not a contributing factor to implementing feature requests.


Follow the Trac Timeline if your interested in knowing when new features appear.

[mkelley]

And in the meantime you CAN use Handbrake and mux the PGS subs in yourself (which I've been doing for going on two years now). My WDLive units have always played the resultant file just fine.

[arovuo]

Can you plz point me to right direction (link/url) how to rip PGS and mux those into mkv?

[mkelley]

Use TSMuxer to demux the PGS track(s), use BDSup2Sub to convert to IDX, and mux back in with mkvmerge (all Windows -- I don't know nuthin' about Macs).

(There's really no need for any more explanation).

[arovuo]

That's not PGS in the mkv with such method, it would be sub in mkv, as there is a format conversion along.
It seems that many players support PGS as such, but as of today handbrake does not support passthru for PGS, and I haven't figured out where to start src code updates to add that.