Soft Subtitle Research

[entropic]

Today i did some tests and thought I might share my findings:

It seems that subtitles quicktime accepts in mp4 are just normal text tracks with 'hdlr' atoms component subtype set to 'sbtl'
In my simple tests I was easily able to get choosable subtitles in itunes/iphone/etc..

First of all, this is not a "how-to add subtitles" post - im just sharing my proof of concept here.

To reproduce, here is one way it can be done:
1. Get the following tools: MP4Box (i used 0.4.4), Dumpster (7.1.3)
2. Get the subtitles in srt format, convert them to gpac-ttxt (MP4Box -ttxt [srtfilepath])
3. Adjust the resulting ttxt file so the subtitles will be positioned correctly and set a vaild font
4. Mux the subtitles with MP4Box -add [ttxtfilepath]:lang=[yourlangcode] [path_to_a_valid_mp4_file]
5. Open the mp4 in dumpster and find the last trak-atom; open moov.(last trak in the file).mdia.hdlr and set the subType to 'sbtl' - and save.
-> Subtitles can be selected in quicktime/itunes/iphone/etc... (and work too)

One thing i couldn't figure out yet: This only works if there is NO chapter track in the file.

I started looking at the muxing code in libhb - doesn't seem to be "too" difficult to add this there - please somebody correct me on this if im wrong. Im still pretty new to libhb/mp4v2...
Or maybe the idea of suppling an srt at encoding time is insulting to the devs? (Stay calm, I didn't use the evil acronym... eg. OCR ) In this case a separate SubInjector tool might be better?
Input from the core-hb devs would be very welcome.

-Ben

[jbrjake]

First off, remember that MP4Box uses GPAC's mp4 library, not libmp4v2. You should be looking at how things are done in MP4Creator, since that uses the same muxing lib as libhb does.

There'd be no philosophical issues with offering the ability to mux in subs obtained elsewhere. Just provide a patch, since no one else seems to feel like doing it.

It probably only works without chapters because GPAC doesn't understand Apple chapter tracks and just sees them as another text track.

Realize that doing something with an existing CLI tool like MP4Box is a lot easier than working the same capability into a library like libhb, and vice versa.

[rhester]

That having been said, I have played with this a bit now using CCExtractor and a hex editor, and I must say, the generalized solution does show some promise. Amazingly, the CCs on LOTR 1 are in mixed-case and closely track the subtitles.

There may be hope for CC extraction and embedding yet.

Rodney

[entropic]

Ok, i got libhb to the point where it spits out a text track thats recognized as subtitles by quicktime. (Unfortunately that was the trivial part)
Good news is that the problem with chapter and subtitle tracks was really gpac related as jbrjacke suspected.
Bad news is that i still have no subs visible with my impl.
Guess im missing some pieces of the puzzle...
Anyway.. I intend to get this working. Too bad that I started this on the last day of my vacation

[entropic]

Weekend at last - And I could finally spend some more time with HB.
So in case somebody cares: I just finished extending libmp4v2 to support the required atom structure for subs and implemented the basic stuff we need in libhb.
Overall, the only thing missing is an srt parser... so i'd say im pretty much done. (Pretending that there is no gui version of hb for now )
In my latest build HB now writes out files with subtitles streams (containing only test text) that can be selected on the whole itunes/QT/iphone family.
Biggest annoyance was getting the iphone to recognize the whole crap too...

Once I'm done with adding a simple srt parser and cleaning the whole code mess up ill post a patch. (Hopefully I find time to do it this weekend)

Peace,
Ben

[dynaflash]

wow.

[ctomer]

This is great news. Well done and thank you!

I tried following your intructions and I still can't get Quicktime to recognised my subtitles. I'm using Dumpster to set trak.mdia.hdlr.subType = sbtl, apart from that, is there anything else that I need to do. I noticed in a similar post on Doom9's forums that there is some hex value that needs to be set, I was sure which value it was, could you clarify if this step is indeed necessary?

[audiovizual]

One thing Im a bit puzzled about is why the text stream is showing a semi-transparent background. (as shown in entropics pic above) If you look at the file in quicktime before you set the handler tag to 'sbtl', the text stream is obviously non-selectable, but it has no background.

It seems that the background appears as soon as you set the 'sbtl' tag, but I cant for the life of me figure out what is making it do that. Its great that we can now have the soft subs, but it would be the real icing on the cake if we could find out how to turn off the background as well.

Any ideas?

[rhester]

One thing Im a bit puzzled about is why the text stream is showing a semi-transparent background. (as shown in entropics pic above) If you look at the file in quicktime before you set the handler tag to 'sbtl', the text stream is obviously non-selectable, but it has no background.

It seems that the background appears as soon as you set the 'sbtl' tag, but I cant for the life of me figure out what is making it do that. Its great that we can now have the soft subs, but it would be the real icing on the cake if we could find out how to turn off the background as well.

Any ideas?

Most certainly. It is intentionally done by Apple to prevent the white text-on-white background problem. It cannot be disabled.

Rodney

[realityking]

I remember reading that MPEG4 TTXT supports colours. Does QuickTime support them too?

[rhester]

Not in my testing - italics/bold/underlining only.

Rodney

[audiovizual]

One thing Im a bit puzzled about is why the text stream is showing a semi-transparent background. (as shown in entropics pic above) If you look at the file in quicktime before you set the handler tag to 'sbtl', the text stream is obviously non-selectable, but it has no background.

It seems that the background appears as soon as you set the 'sbtl' tag, but I cant for the life of me figure out what is making it do that. Its great that we can now have the soft subs, but it would be the real icing on the cake if we could find out how to turn off the background as well.

Any ideas?

Most certainly. It is intentionally done by Apple to prevent the white text-on-white background problem. It cannot be disabled.

Rodney

Bummer Now you've explained it, I understand why its like that, but personally I prefer it without, I find it kinda distracting...Oh well, if thats how it is...

I remember reading that MPEG4 TTXT supports colours. Does QuickTime support them too?

Not in my testing - italics/bold/underlining only.

Rodney

Actually, that is one thing you can do, I had a fiddle with that while trying to sort out the background thing, and found you can do it by setting it in the font style tag using RGBA codes within the TTXT file. Worked in Quicktime for me I havent tested extensively tho

[astroboy]

Hello All,

I've been trying to make this process (Dumpster edits and so on) transparent,
and the first thing i've been trying is first to basically hack MP4box to write 'sbtl' codes instead of 'ttxt'.

I've been doing that on the svn version, and I've had a semi-success : srt files get added as 'sbtl', and recognized as subtitles in quicktime (meaning they show up with the semi transparent background), but now, I've got a problem I have now idea how it happened, or how to resolve : the subtitles are HUGE, taking up the whole movie area (using a font size like 96 or so) and there is no way to change that by specifying a font size or different options in the MP4box command line....

I know some will give me a "that's not supported, you broke it, you fix it", but I'm trying to make the whole process easier, and before trying to write a new command line option for mp4box properly, I wanted to make sure that there was no other problems involved...

Anybody has an idea ?

[nonomia]

I belive the font name and size atoms not specified.
I want to recommend to try this program for Windows user.

http://www.bomijoa.com/iphone/download/mp4te_0.3.2.msi

Basically, you can convert the container from AVI (DX50/DVIX/XVID only) to MP4 without transcoding.
But, audio will be transcoding to AAC format.

MP4 file(s) will be open directly.

Basically, this program will add the subtitle track on MP4 file.
and If need, you can change the name of atom from text to sbtl.

Actually, this program has been written based on the infomation from this website.

[refulgentis]

@astroboy
Use dumpster to look at the "dimensions" of the text track and play around with those, you'll see how it works. I'm not sure how to fix that with MP4box, however.

[Makemose]

Does anybody have an idea what I have done wrong? The subtitles are as big as the half screen.
I used the idea for windows by nonomia.

[PluueeR]

Did you tried this tool?

http://www.bomijoa.com/iphone/download/mp4te_0.3.2.msi

[Makemose]

That´s the link nonomina has posted, so it´s the same tool I used. But why are the subtitles so big?

[rhester]

We don't offer support for MP4TE here.

Rodney

[Nemesis7]

Shouldn't it be possible to add all the tracks one wants and then edit them in Dumpster to make them work on a iPhone? Example using mp4box/nicmp4box:

mp4box -add video.h264 -add audio1:lang=eng -add audio2:lang=dut -add subs.srt -new output.mp4

- What atoms should be changed to get this working?
- Is it possible to use ttxt files instead of srt and make them visible on the iPhone?
- How can I add chapters to make the chapters contain stills of the frame they start (I've seen this being done on youtube)?

Thanks in advance,
Neme.

P.S. Maybe this should be a new post, but I'm not sure, feel free to move this to a new one.

[Nemesis7]

Guys, good news, I found this site, and I did what it said, now I have:

Video + 2 audio + 2 subs + chapters working on iPhone You need Dumpster or Ultra-Edit, mp4box and Quicktime Pro

This what you do:
- encode your movie in h264 and put it in a mp4 container (mp4box -add video.h264 -new output.mp4)
- encode your audio to AAC (max. 2 streams)
- rip your subtitles to srt (max. 2 steams)
- convert your srt subs to ttxt subs (mp4box -ttxt sub1.srt)
- add you subs each seperate in a m4v container (mp4box -add sub1.ttxt -new sub1.m4v)
- change moov/trak/mdia/hdlr/coponent subtype from 'text' to 'sbtl' in either Dumpster (hex value: 7362746c) or Ultra-Edit (just rename text to sbtl) of your (m4v) subs
- open your output.mp4 in Quicktime Pro
- open your audio1.aac in Quicktime Pro, go to Edit -> Select All -> Copy
- go to your output.mp4 en do Edit -> Add to Movie
- do this for your 2nd audio track and your 1st and 2nd subtitle track (m4v files). !Remember to bring the slider back to the beginning after eacht time you paste something into the movie
- Now go to Windows -> Show Movie Properties in your output.mp4 file
- select the 1st audio track and go to 'Other Settings', set the language, and choose the 2nd audio track in the field 'Alternate'
- select the 2nd audio track and go to 'Other Settings', set the language, and uncheck this audio track since the 1st will be your default track
- uncheck both subtitle streams, set their languages, and chosse the 2nd subtitle stream in the 'Alternate' field of the 1st subtitle stream
- save the movie and sync up!

Source: http://translate.google.com/translate?u ... l=es&tl=en

Cheers,
Neme.

P.S.For chapters see the source as well, didn't feel like typing anymore

[rhester]

I would really, really advise against this approach for VFR content, if I understood the source blog correctly.

Rodney

[bspline]

I belive the font name and size atoms not specified.
I want to recommend to try this program for Windows user.

http://www.bomijoa.com/iphone/download/mp4te_0.3.2.msi

Basically, you can convert the container from AVI (DX50/DVIX/XVID only) to MP4 without transcoding.
But, audio will be transcoding to AAC format.

MP4 file(s) will be open directly.

Basically, this program will add the subtitle track on MP4 file.
and If need, you can change the name of atom from text to sbtl.

Actually, this program has been written based on the infomation from this website.

Regarding the mp4te program, I'm havng serious problems trying to run it. The subtitles are there, and are "soft", but the video output is unplayable, full of glitches, artfacts, flares, etc. Looks like watching a scratched DVD. Using Vista, and the default instalation for mp4te_0.3.2.msi. Any hints/thoughts?

[rhester]

I have one.

This is not the MP4TE support board.

Since this is the second time I've had to point this out in the same thread, it will be locked.

Rodney