Chapters inside MP4 file

maurj · Post by **maurj** » Fri Mar 23, 2007 7:55 pm

Hi Cyander,

Sorry about that

I was kind of 3/4 of the way there...

If it helps, our patched mp4v2 now has a function to add text tracks, so you can probably use that to add a subtitle track with text subtitles if you want

Don't be put off - and best of luck in sorting the subtitles!

maurj.

Cyander · Post by **Cyander** » Fri Mar 23, 2007 8:04 pm

Text subtitles are best left for another tool, methinks. Character recognition is a nightmare, to say the least.

Although a big thanks for the work to libmp4v2, as I do have a couple ideas on a Metadata tool for MP4 files where letting someone add chapter tracks and subtitles after the fact (in addition to tagging for iTunes) would be nice features.

EDIT: Nevermind, there is still work to be done to scratch my particular itch... but nothing really Earth-shattering. I can do the work pretty easily at this point to enhance it. I was hoping I could enter the chapter titles from my DVD insert, and have those inserted into the text track instead. I might work on it a bit tonight.

maurj · Post by **maurj** » Fri Mar 23, 2007 8:19 pm

Hi Cyander,

No worries! I agree about text subtitles - OCR is a biggie, especially given the subtitles could be in any number of languages...

Good call on the "applying chapters" tool. Rhester was asking me about this earlier on - I'm sure there are lots of Handbrake users who have many ripped DVDs without chapter markers, which they'd rather not rip again. Would be very useful, I'm sure.

I haven't really gone the whole hog with adding text tracks to libmp4v2 - that is, I don't actually use the "add text track" function I added, just the "add chapter text track" function. But it should work for basic text tracks, I think. Adding text track samples is then as easy as calling MP4WriteSample with a buffer where the first two bytes specify the length of the following text, followed by the text itself. If you want to do unicode, you'll need to add an extra 'encd' atom afterwards, I think. Here's some useful links on text samples:

http://developer.apple.com/documentatio ... H205-57434

http://lists.apple.com/archives/QuickTi ... 00147.html

- maurj.

Cyander · Post by **Cyander** » Fri Mar 23, 2007 8:47 pm

maurj wrote: I haven't really gone the whole hog with adding text tracks to libmp4v2 - that is, I don't actually use the "add text track" function I added, just the "add chapter text track" function. But it should work for basic text tracks, I think. Adding text track samples is then as easy as calling MP4WriteSample with a buffer where the first two bytes specify the length of the following text, followed by the text itself. If you want to do unicode, you'll need to add an extra 'encd' atom afterwards, I think. Here's some useful links on text samples:

- maurj.

The only difference between a chapter track, and a text subtitle track is that the text atom and possibly the samples have text formatting data. Just that with a chapter track, you don't need it...

hawkman · Post by **hawkman** » Sat Mar 24, 2007 11:44 pm

I hope you don't mind me jumping in here. Excellent work, it's quite exciting to have this working!

One minor issue is that with my (limited) testing, chapter marks are very slightly too early - about 10 frames I think, although I've not tried to pin it down exactly... Is this the same issue Metadata Hootenanny used to have with the ripped file being a slightly different length to the DVD?

Cyander · Post by **Cyander** » Mon Mar 26, 2007 7:33 pm

I believe you are pretty much spot on... I extracted a text track, and re-added it to match the video's duration, and it went into sync. I am hunting around the code trying to figure out how the ripped duration gets calculated so I can get the right adjustments.

hawkman · Post by **hawkman** » Mon Mar 26, 2007 7:40 pm

Awesome. I can't wait

Cyander · Post by **Cyander** » Tue Mar 27, 2007 3:42 am

It isn't going to be as easy as I hoped to restore the sync. The problem is that the data is frame accurate /until/ it goes through the sync step. After that, the chapter breaks lose sync, because we have no good way to keep track of how the breaks are being adjusted.

I am looking into a couple ways to use the frame queues themselves to propogate these chapter breaks, so that when syncing/etc, these 'frames' will continue to be between the right two frames, making them good time markers.

EDIT: Okay, I am officially starting to be at a loss at how to fix this. The sync process for video and audio is pretty complex, and even though my results are turning out /pretty/ well, I simply cannot get the sync to turn out the way I expect. The problem tends to be that even when I attach a chapter 'tag' to individual frames as they are read from the DVD and decoded, something is just happening to make the chapter appear in the stream earlier than expected.

FURTHER: Oh man... I can't believe this is staring me right in the /face/. A pair of bugs in some older code which tries to calculate the chapter from the current 'cell' in the DVD is what seems to be preventing my sync method from working. I have fixed both bugs, and while I still need to validate with a full run of something full-length... it is working on small 3-4 chapter clips just fine. Best of all, it does appear to be frame-accurate.

I want to wrap up the support for better chapter titles (user-generated), and see if I can't get someone to check this in. It works swell.

hawkman · Post by **hawkman** » Tue Mar 27, 2007 12:20 pm

Cyander wrote:FURTHER: Oh man... I can't believe this is staring me right in the /face/. A pair of bugs in some older code which tries to calculate the chapter from the current 'cell' in the DVD is what seems to be preventing my sync method from working. I have fixed both bugs, and while I still need to validate with a full run of something full-length... it is working on small 3-4 chapter clips just fine. Best of all, it does appear to be frame-accurate.

Hehe, I hate it when that happens. Frame-accurate is better than I'd dared hope... I'll be very happy to give it a thorough test

jbrjake · Post by **jbrjake** » Tue Mar 27, 2007 2:53 pm

Cyander wrote:A pair of bugs in some older code which tries to calculate the chapter from the current 'cell' in the DVD is what seems to be preventing my sync method from working.

In HB code? Where? You're not talking about sync.c, are you?

Cyander · Post by **Cyander** » Tue Mar 27, 2007 3:34 pm

jbrjake wrote:
Cyander wrote:A pair of bugs in some older code which tries to calculate the chapter from the current 'cell' in the DVD is what seems to be preventing my sync method from working.
In HB code? Where? You're not talking about sync.c, are you?

Nah... the bugs are in dvd.c. Ironically, fixing them has /zero/ effect on the behavior of the program. Very subtle stuff too.

EDIT: Okay, hopefully the last time I have to (heavily) work on this. I thought I had the sync issue, but then there were still some (more subtle) sync issues underneath. It turns out that most DVDs have a couple pictures from a group of pictures spilling over into the next chapter's section of the disc. DVD players skip these, and move onto the next group. So I had to do a little extra work to ensure that a chapter mark got 'delayed' by the MPEG-2 decoder until the next group of pictures arrived. I also did some work on the UI so you can set the chapter titles before ripping, and get something better than simply: Chapter 01, Chapter 02. Also cleaning up maurj's text sample writing code while I am at it.

Long story short, I have been getting frame-accurate sync on chapters (compared to actual DVD players) for all DVDs I have tried, and I can name the chapters something meaningful. I'll post a diff tomorrow when I have time to remove some 'stupid printfs I used because XCode blows at debugging static libs'.

Cyander · Post by **Cyander** » Wed Mar 28, 2007 9:47 pm

Bump.

Okay, I think this thing is about ready to let the HB team look it over and decide if they want to accept/deny the patch. I ran about as close to a test suite as I can run on a Mac Pro with 2 drives, and in 8 hours. No more bugs/errors when compared to DVD Player or a standalone box.

I'll be opening up a new thread in the development forum once I am finished with the cleanup.

EDIT: BTW, I added UTF-8 support for the chapter titles when they come from the UI. Works well.

hawkman · Post by **hawkman** » Wed Mar 28, 2007 10:13 pm

You are a legend. The feature that's topped my wish list for some time

nightstrm · Post by **nightstrm** » Thu Mar 29, 2007 1:31 pm

I'm having an issue with the latest SVN and chapter markers not being created. I'm ripping off of my X-Files Season 1 DVD, and while each title has a number of chapters on the disc, the chapters are not created in the resulting file. Anyone have any thoughts on what could be causing the problem, or if there are any related known issues?

I've made sure that the 'create chapter markers' box has been checked, both on the main window and in the preferences. Also, it does not seem to matter if I rip directly from the DVD or after moving the VOB files to the hard drive w/ MtR.

hawkman · Post by **hawkman** » Thu Mar 29, 2007 2:23 pm

Well, it could depend on what your settings are - for instance, you need to be outputting to .mp4.

dynaflash · Post by **dynaflash** » Thu Mar 29, 2007 2:33 pm

As I recall, maurj (who wrote that feature) said you need to use the .m4v file extension for iTunes to recognize the chapter markers.

nightstrm · Post by **nightstrm** » Thu Mar 29, 2007 2:33 pm

Yes, sorry I should have given more information.

I am outputting a MP4 file using the AVC codec (x264 main). My average bitrate is 2250, and the audio is being downmixed to AAC 2.0 at 256kbps. The files plays fine in both QT and ATV, minus the chapters.

nightstrm · Post by **nightstrm** » Thu Mar 29, 2007 2:35 pm

dynaflash wrote:As I recall, maurj (who wrote that feature) said you need to use the .m4v file extension for iTunes to recognize the chapter markers.

Wow, that was it. Changed the extension from .mp4 to .m4v and it works fine. Now that you mention it, I do somewhat remember reading that in the IRC channel at one point. Thanks!

dynaflash · Post by **dynaflash** » Thu Mar 29, 2007 2:39 pm

No problem. I think before release, we may be forcing the .m4v extension in the gui if Chapter Markers is checked. Just to avoid the confusion.

jbrjake · Post by **jbrjake** » Thu Mar 29, 2007 3:12 pm

I tested chapter markers the other day, and they were usable in iTunes without changing the file suffix to .m4v. Worked fine with .mp4.

Neil · Post by **Neil** » Sun Apr 01, 2007 10:01 pm

Guys, you have no idea how excited I got reading this thread. I nearly soiled my panties.

This is big, big, big. All the volunteers working on HB are awesome.

I have hundreds of instructional DVDs where the chapter marks are critically important, that's why I haven't ripped many of them to mp4 (nor m4v) yet. Most of the ones that I did rip were ripped to individual chapters, which is an incredibly tedious process involving dozens of mouse clicks and navigating pop-up menus and switching back and forth to the keyboard for every single chapter. I guess you guys already know that.

User editable chapter names would be gravy. I'll be so excited when I rip my first DVDs with chapter marks and see those chapters on my iPod.

Chapter marks will also come in handy for porno movies.

Hopefully, someday I'll be able to also add chapter marks to the mp4 files I created from my VHS tape captures. I know that's not an HB issue though.

Cyander · Post by **Cyander** » Tue Apr 03, 2007 11:42 pm

Neil wrote: User editable chapter names would be gravy. I'll be so excited when I rip my first DVDs with chapter marks and see those chapters on my iPod.

I have a build that does reliably let the user edit chapter marks (and the support for frame-accurate sync has improved from the diff I showed to the dev team). This will make it into HB, it is just a matter of timing, and that it is likely better for stability of 0.8.5b1 if it was submitted for b2.

Hopefully, someday I'll be able to also add chapter marks to the mp4 files I created from my VHS tape captures. I know that's not an HB issue though.

Well, the good news is that we now know how to make chapter tracks work for iTunes. It isn't a mystery anymore, which means other tools will be able to eventually incorporate the feature in the future.

hawkman · Post by **hawkman** » Tue Apr 03, 2007 11:45 pm

Cyander wrote:...the support for frame-accurate sync has improved from the diff I showed to the dev team...

If I beg, can I test it?

Neil · Post by **Neil** » Wed Apr 04, 2007 1:28 am

Cyander wrote:...Well, the good news is that we now know how to make chapter tracks work for iTunes. It isn't a mystery anymore, which means other tools will be able to eventually incorporate the feature in the future.

Woohoo!

Vidar · Post by **Vidar** » Mon Apr 23, 2007 12:32 pm

I downloaded 8.5.1 yesterday, and just wanted to take a moment to say thanks. I know that you guys must have put a lot of work into the new beta, and I'm extremely happy that you took the time to incorporate this feature.

Great work all of you, and keep it up!

Thanks again,
Vidar.