Subtitle Overlay Improvements

[Cyander]

So, I am ripping a few DVDs that I prefer having subtitles with... all imported films. I am being driven nuts by how DVD subtitle overlays are so variable from disc to disc in both position and size. Oh, and the fixed 20 pixel margin used by HB for cropping.

So, instead of coming in, whining on the requests forum, I decided to see how the codebase is, and start work on it myself... Goody, the code is actually pretty clean and well laid out. It takes me no more than 20 minutes to get titles to shift correctly using a new 'Smart Subtitle Adjustment' checkbox in the HB UI.

Before:


After:


In this case, the change is pretty small, but it does push the subtitles closer to the border of the frame without destroying its ability to be displayed on a recent HD TV (most have about 0-2% overscan, instead of the usual 5-7% of pre-HD sets).

The next step that I will work on is trying to display the selected subtitle in the picture preview frame... and after that, see if I can implement adjustable scaling of the overlay.

[maurj]

Hi Cyander,

Sounds good! If you make some improvements, we'd be very happy to see what you have Thanks for taking the dev approach too - i.e. trying it yourself, rather than just asking for it. Much appreciated

[Cyander]

Thanks for taking the dev approach too - i.e. trying it yourself, rather than just asking for it. Much appreciated

It wouldn't be polite. I make a living on writing code...

[s55]

So do I, but that hasb't stoped me contributing.

Be kind

[Cyander]

I mean it wouldn't be polite to just request and tap my foot impatiently when I /know/ what needs to be done, and how to do it.

I already have the scaling algorithm planned out, I just need a way to preview it so people aren't having to spew gobs of sample encodes to get the right scale they want.

[s55]

O right, yeh, makes sense. oops

Good work, Certainly looks interesting!

[Cyander]

jbrjake asked that I look into why subtitles lack color as well, and so I figured I would post my finding here for posterity... even though I might get around to fixing it.

Since Handbrake uses YUV420p colorspaces for all work, in order to keep the work all in one colorspace, the subtitles also need to be in YUV. DVDs are funny in that the color palette for subtitles is in RGB (from what I can tell), and it is up to the DVD player to convert the subtitle into YUV when decoding it.

But that isn't why subtitles lack color.

YUV420p is unlike RGB in that for one channel, you have 1:1 mapping of samples to pixels, but the other two channels have a 1:4 mapping of samples to pixels (2x2 pixel blocks with one value). So, when arbitrarily overlaying a subtitle onto a YUV420p image, things don't line up all that easily.

Titer seems to have taken the quick route and decided to estimate the brightness/luminance, and not implement proper chroma handling. I don't blame him, as the changes aren't all that easy to make, and subtitles look decent enough as it is now.

As I am in the middle of changing the sampling/compositing of subtitles to allow for scaling, I will likely take the 'hard' part of the fix of making the decoded subtitle a proper YUV420p buffer (with an alpha buffer), and using that for sampling and compositing. I don't know if I will get around to fixing the lack of color though, as while it would be simple, the rest of the work (scaling/adjusting of subtitles) is a bit higher priority on my list right now.

[golias]

jbrjake asked that I look into why subtitles lack color as well, and so I figured I would post my finding here for posterity... even though I might get around to fixing it.

Since Handbrake uses YUV420p colorspaces for all work, in order to keep the work all in one colorspace, the subtitles also need to be in YUV. DVDs are funny in that the color palette for subtitles is in RGB (from what I can tell), and it is up to the DVD player to convert the subtitle into YUV when decoding it.

But that isn't why subtitles lack color.

YUV420p is unlike RGB in that for one channel, you have 1:1 mapping of samples to pixels, but the other two channels have a 1:4 mapping of samples to pixels (2x2 pixel blocks with one value). So, when arbitrarily overlaying a subtitle onto a YUV420p image, things don't line up all that easily.

Titer seems to have taken the quick route and decided to estimate the brightness/luminance, and not implement proper chroma handling. I don't blame him, as the changes aren't all that easy to make, and subtitles look decent enough as it is now.

As I am in the middle of changing the sampling/compositing of subtitles to allow for scaling, I will likely take the 'hard' part of the fix of making the decoded subtitle a proper YUV420p buffer (with an alpha buffer), and using that for sampling and compositing. I don't know if I will get around to fixing the lack of color though, as while it would be simple, the rest of the work (scaling/adjusting of subtitles) is a bit higher priority on my list right now.

You are now officially my hero.

Colorless, translucent subtitles has been the #1 factor in my not using HB for a large section of my foreign-language DVD library. Make subs easy to read (and tell apart, when color is used to separate dialog), and you will make HB the best app in the universe.

If you need a couple of particularly challenging sample DVDs to test your work with, say the word and I would be more than happy to mail a disk or two to you.

[Cyander]

I have plenty of sample DVDs. I have a fair amount of subtitled movies and TV series sitting around that I can hit most of the needs.

But really, the lack of chroma information being sampled is the one reason why they can't do multi-color titles, and are translucent.

The work I am about ready to start is two-fold:

1) Fix up the subtitle buffers so that they can store chroma information, and make it color-less, instead of reading from the DVD palatte. Fix up the sampling and compositing code so that this chroma information gets used, and to allow for scaling in the future. This part is really one piece of work, and very complex, so I am doing it on its own.

2) Add an option "Greyscale Subtitles", and start reading chroma information from the DVD palatte. Greyscale Subtitles is mostly for people like me who can't stand some of the colors chosen by the subtitlers in some movies (yellow subtitles as the main color? Come on...). Otherwise, with it disabled, it will use the colors as displayed on the DVD.

[golias]

I have plenty of sample DVDs. I have a fair amount of subtitled movies and TV series sitting around that I can hit most of the needs.

But really, the lack of chroma information being sampled is the one reason why they can't do multi-color titles, and are translucent.

The work I am about ready to start is two-fold:

1) Fix up the subtitle buffers so that they can store chroma information, and make it color-less, instead of reading from the DVD palatte. Fix up the sampling and compositing code so that this chroma information gets used, and to allow for scaling in the future. This part is really one piece of work, and very complex, so I am doing it on its own.

2) Add an option "Greyscale Subtitles", and start reading chroma information from the DVD palatte. Greyscale Subtitles is mostly for people like me who can't stand some of the colors chosen by the subtitlers in some movies (yellow subtitles as the main color? Come on...). Otherwise, with it disabled, it will use the colors as displayed on the DVD.

Actually, back in the 80s when most computer monitors were still monochrome, "amber" screens began to gain ground over green or white ones, because studies had revealed that yellow and black are the highest level of contrast the human eye perceives. People read fastest with the least eye strain when reading yellow against black.

Using that logic, yellow subtitles with black outlines make a lot of sense. They are actually easier to read than white ones (and certainly MUCH easier to read than translucent!)

I think the ideal situation would be at least having the option of preserving the palate which the DVD suggests, which it sounds like your plans include, so yay for you! ^_^

[Cyander]

The problem I have with yellow is that there is too much contrast. You have this thick black border, filled with bright-as-the-sun yellow. I find my eyes hurting more with yellow subtitles on a 40" TV than I do white. Especially when viewing in theater-like conditions.

Of course, I am also the type of person who uses a white-on-black color scheme for my Terminal windows, and tone down the white to be a light grey to avoid eye-strain. So I think we could go round-and-round over what is better until we are blue in the face, and not agree on the final answer.

I agree there is a logical reason for it... but it doesn't stop it from being something I disagree with for other logical reasons.

[golias]

The problem I have with yellow is that there is too much contrast. You have this thick black border, filled with bright-as-the-sun yellow. I find my eyes hurting more with yellow subtitles on a 40" TV than I do white. Especially when viewing in theater-like conditions.

Of course, I am also the type of person who uses a white-on-black color scheme for my Terminal windows, and tone down the white to be a light grey to avoid eye-strain. So I think we could go round-and-round over what is better until we are blue in the face, and not agree on the final answer.

I agree there is a logical reason for it... but it doesn't stop it from being something I disagree with for other logical reasons.

Which is why leaving it up to the user is a great choice.

I strongly prefer a high contrast to make it easy to read text against busy backgrounds. Whether its a difference in TV display settings or a difference in visual perception, you find that sort of thing to be too bright. As long as we can set our files to use (or not use) the default colors, everybody wins.

[golias]

How is this one coming along, by the way? Are we likely to see your subtitle fixes in the next public revision? Do you need more people to help with testing?

[eddyg]

Hi Cyander,

Any progress on mapping the DVD palette (or any other palette for that matter) into chroma info for the subtitle sub pictures?

Is the issue in decsub.c where we interpret the palette and map it to just luminance values without chroma?:

Code: Select all

                case 0x03:
                {
                    int colors[4];
                    int j;

                    colors[0] = (pv->buf[i+0]>>4)&0x0f;
                    colors[1] = (pv->buf[i+0])&0x0f;
                    colors[2] = (pv->buf[i+1]>>4)&0x0f;
                    colors[3] = (pv->buf[i+1])&0x0f;

                    for( j = 0; j < 4; j++ )
                    {
                        uint32_t color = title->palette[colors[j]];
                        pv->lum[3-j] = (color>>16) & 0xff;
                    }
                    i += 2;
                    break;
                }

Cheers, Ed.