Chapter Sync and Chapter Name Fixes/Support

maurj · Post by **maurj** » Fri Apr 27, 2007 8:26 am

Cyander, that's looking brilliant

Just trying out rev 553 now, and the chapter support looks great!

- maurj

Cyander · Post by **Cyander** » Fri Apr 27, 2007 9:10 pm

Good to hear the integration went well.

I have been using it for a couple weeks, so I hope I have all the major issues ironed out.

mdavis75 · Post by **mdavis75** » Sun May 20, 2007 10:11 am

Hi Cyander,

I have been running the svn version of HB since around build 553, and am running build 591 right now.

Ever since there was a build where the chapters could be named, I have been having the following problem:

- When converting some DVDs (Toy Story 10th Anniv Edition for example), the m4v file produced has less chapters in it than were configured in HB. In the case of toy story, there are 30 chapters (which HB finds in the scan), but only the first 14 are in the m4v file.

I have had this problem in around 50% of the discs I try.

Please let me know if you require further info. I am a programmer by trade (only java and .Net though), so am happy to look at some logs etc, whatever you like.

As a side point, I think the chapter work you have done is excellent. Keep up the good work!

Cheers,

mdavis75
(Australia)

tvcity6455 · Post by **tvcity6455** » Tue May 22, 2007 3:08 pm

Hi Cyander,

I am also experiencing the same issues as mdavis with the svn version. I have only been able to get one DVD (Pirates of the Caribbean: Dead Man's Chest) to encode with the proper chapter markers. This has happened for every build I've downloaded with this feature in place. I always encode m4v files using the x264 (h.264 main) setting.

Here is a little more detail about what I have been experiencing:

- HB skips chapter markers randomly (or seemingly randomly), meaning if the movie has 10 chapters, it will mark chapters 1, 3, 4, 5, 8, and 10 and name them using the first 6 names in the list.
- The chapters it skips are different for every movie, but it always skips the same chapters each time I encode a movie, despite changing other settings.
- It skips the same chapters even if I don't enter custom chapter names.
- It skips the same chapters whether I encode from a ripped folder on my hard drive or directly from the DVD.
- It skips a different number of chapters for each movie. For Casino Royale, it marked 24 of the 28 chapters. For The Prestige, it marked 13 of 24. I haven't been able to find any pattern about which chapters it skips.
- This happens on older DVDs and new DVDs, so I don't think it's related to any advanced copy protection (but I could be wrong).

Unfortunately, I'm not a programmer, so all I can really do is corroborate what mdavis seems to be experiencing. If you'd like more detail, please let me know. If you have the time to implement a fix, I'd be happy to help you test it or help in any other way I can. I wish I could do more.

Thanks for your work on this feature. It's been something I've really wanted.

mdavis75 · Post by **mdavis75** » Tue May 22, 2007 11:26 pm

Hi again Cyander,

Yes my experience is the same as tvcity6455.

Cheers,

mdavis75

fullerflyer · Post by **fullerflyer** » Wed May 23, 2007 5:25 am

mdavis75 wrote:Hi again Cyander,

Yes my experience is the same as tvcity6455.

Cheers,

mdavis75

Ditto...

I encoded U2 Rattle And Hum (20 Chapters) and it worked flawlessly...

I encoded U2 Vertigo 2005 Live From Chicago (25 Chapters) and it found only 17 chapters, and gave names to them in the order of the first 17 titles that I endered before encoding...

Cyander · Post by **Cyander** » Thu May 24, 2007 10:43 pm

Unusual, I was seeing bugs like this when I was working on it, but I could swear I nailed them down before the commit. One was exactly as described here, oddly, and I did get it nailed down and fixed, but this sounds like the exact same thing...

Worst part is that the DVDs you list I don't actually have, so I can't debug with them. But, the fact that a single DVD will recreate the bug predictably is a /good/ thing, believe me.

As an aid, list off ALL the DVDs (very specific now, it is important for discs that might have an SE printing), and I will see if I have one of them, or can get one cheaply.

The thing is... I have about a dozen DVDs ripped, including the Matrix Trilogy, with intact chapters... so either some code didn't make it into SVN that needed to (I am willing to say this is doubtful, unless it is an error on my part), or my couple dozen DVDs is completely different from someone else's couple dozen DVDs, so I need the list so I can reproduce the problem.

Neil · Post by **Neil** » Thu May 24, 2007 11:59 pm

Do you think it matters which codec, container, and other settings were used?

I had the same problems with missing chapters. The DVD's were instructional for Argentine Tango dance, so I'm sure you don't have a copy around your house to test. I also had another problem that you fixed a long time ago. My marks came much later in the rip than in the DVD.

I accidentally ripped one DVD into the main profile. The chapters were perfect though. I went back and re-ripped it into the iPod flavor of x.264 and the chapter marks came 2-3 seconds later. This is an NTSC DVD ripped with the same frame rate as the source, so the marks were around 60-90 frames late which is much worse than what you described previously. It worked out great for me because I prefer to skip the chapter's title page anyway. The only reasons I mentioned it is because it might be related to the missing chapter problem and it could represent something else that might have to get worked on eventually if it happens to other people too.

Many thanks for all your hard work. I'm already loving the chapter marks even if the implementation isn't perfect yet.

EDIT: I went back and the title page lasts ten seconds from the start of the chapter on the DVD. I don't even see the title page on the .mp4 file when I jump to the chapter mark. That means the mark is around 300 frames off target. I'm sorry if the problem was caused by something I did, but I can't imagine how I could screw it up like that, but I'm very glad that it worked out that way for me. Ten seconds feels like an eternity when I'm constantly jumping to chapters, so I'm glad I don't have to sit through it anymore on that video.

mdavis75 · Post by **mdavis75** » Fri May 25, 2007 12:06 am

Hi cyander,

I'll test this t onight when I get home from work, but I think the builds where you CAN'T name the chapters work fine, the builds where you CAN name the chapters have the problem described.

I converted Toy Story initially on the build where you couldn't name the chapters, and all chapters were marked correctly. On builds where I can name the chapters, the problem occurs.

If somebody can verify this before I can, please feel free to comment.

If this is the case, it does sound a bit like a code versioning problem I guess.

Cheers,

Michael

Cyander · Post by **Cyander** » Fri May 25, 2007 7:11 am

mdavis75 wrote:Hi cyander,

I'll test this t onight when I get home from work, but I think the builds where you CAN'T name the chapters work fine, the builds where you CAN name the chapters have the problem described.

I converted Toy Story initially on the build where you couldn't name the chapters, and all chapters were marked correctly. On builds where I can name the chapters, the problem occurs.

If somebody can verify this before I can, please feel free to comment.

If this is the case, it does sound a bit like a code versioning problem I guess.

Cheers,

Michael

Unfortunately, this is a bit like pointing out the sky is blue. Before my code was checked in, all Handbrake did was estimate the times where chapter markers should have been added. It didn't account for much of anything else.

The chapter marker code I wrote was all new, and will introduce all-new bugs that have nothing to do with the old code.

If you have any good repro discs that you haven't already mentioned, let me know, and I will do my best to figure things out. The iPod comment is good to know. I am not sure why it would cause any weirdness though, as my marker code shouldn't be getting touched by the encoder workers at all... and it is all written to disc by the same code in the end.

Neil · Post by **Neil** » Fri May 25, 2007 7:26 pm

I should have done more testing before I said anything about that iPod rip. It turns out that the sync is perfect when I play the videos in iTunes. The ten second mis-timing only happens on the iPod. It could very well be an iPod issue rather than an HB problem. I'll be at that Tango dance convention for the next nine days, so I wont be here to report on further testing. I'll check back in when I return.

Cyander · Post by **Cyander** » Fri May 25, 2007 11:51 pm

Neil wrote:I should have done more testing before I said anything about that iPod rip. It turns out that the sync is perfect when I play the videos in iTunes. The ten second mis-timing only happens on the iPod. It could very well be an iPod issue rather than an HB problem. I'll be at that Tango dance convention for the next nine days, so I wont be here to report on further testing. I'll check back in when I return.

Good report. That actually helps a fair amount, even though it pretty much means I won't have the time to look at the bug anytime soon, and might not be able to generate a fix for it.

So it looks like the main issue that needs attention is chapter markers being skipped. I still need more discs that repro the issue.

mdavis75 · Post by **mdavis75** » Sat May 26, 2007 1:12 am

Discs that have chapter problem:

Chicken Run
Finding Nemo
Harry Potter and the Prisoner of Azkaban
Toy Story 2

I will add more as I test more.

mdavis75 · Post by **mdavis75** » Sun May 27, 2007 11:54 am

Another problematic disc:

Clerks.

Cheers,

mdavis75

eddyg · Post by **eddyg** » Mon May 28, 2007 5:03 am

When the DVD scan is taking place sometimes "short" chapters are merged.

scan: chapter 29 too short (179 blocks, cells=30->30), merging

Could that be what is happening?

Cheers, Ed.

tvcity6455 · Post by **tvcity6455** » Mon May 28, 2007 6:25 pm

I thought it might be related to short chapters as well, but in the case of Casino Royale, there were 2 short chapters that were exactly the same length and it marked one and not the other. It may be related to the length, but again, no noticeable pattern (ie, all chapters under 2:00 are skipped).

Here is a list of DVDs I haven't been able to encode. I did try a few more older discs that seemed to work, so maybe I was wrong and it is somehow related to how they are marking chapters on new discs.

Madonna: The Confessions Tour
Pirates of the Caribbean: Curse of the Black Pearl
Inside Man
Children of Men
Syriana Special Edition
The Prestige Special Edition
Casino Royale (this was a VIDEO-TS I downloaded)
The Queen Special Edition
Kill Bill Volume 2 Special Edition

eddyg · Post by **eddyg** » Mon May 28, 2007 9:48 pm

tvcity6455 wrote: Children of Men

Well I've got & encoded Children of Men, and I've 20 chapters 1-20, all looks good to me - unless there are supposed to be more than 20

However I don't really care that much, I use them as a way of skipping forwards quickly given that the AppleTV search forward/reverse is so broken.

Cheers, Ed.

tvcity6455 · Post by **tvcity6455** » Mon May 28, 2007 10:11 pm

Well I've got & encoded Children of Men

Cyander, would it help if we posted our settings? If some of us can get them to work and some can't, maybe it has something to do with settings.

eddyg, did you encode from the DVD or a ripped folder?

eddyg · Post by **eddyg** » Mon May 28, 2007 10:22 pm

tvcity6455 wrote:
Well I've got & encoded Children of Men
Cyander, would it help if we posted our settings? If some of us can get them to work and some can't, maybe it has something to do with settings.

eddyg, did you encode from the DVD or a ripped folder?

[/quote]

I encoded from a folder ripped with MTR.

I'm using the SVN, which may make a difference.

AFAIK I haven't had any missed chapters with any of my encodes.

Cheers, Ed.

mdavis75 · Post by **mdavis75** » Sat Jun 02, 2007 12:06 pm

Add 'Jindabyne'

Contains 24 chapters, converted file had 23.

mdavis75

Cyander · Post by **Cyander** » Mon Jun 11, 2007 3:51 pm

tvcity6455 wrote:
Well I've got & encoded Children of Men
Cyander, would it help if we posted our settings? If some of us can get them to work and some can't, maybe it has something to do with settings.

eddyg, did you encode from the DVD or a ripped folder?

Yes, it would. There was a bug I found earlier where the code meant to sync audio and video was incorrently dropping chapter markers depending on the video framerate. If you can post your settings, that would help figure out if this is related to it. Unfortunately, debugging this stuff is PAINFUL since I need to wait around for a particular breakpoint to get hit. :/

tvcity6455 · Post by **tvcity6455** » Tue Jun 12, 2007 12:36 am

My settings are typically:

m4v container
x264 main codec
1500 or 2000 avg. bitrate
frame rate same as source
2 pass
anamorphic (cropping as necessary)
deinterlacing only on interlaced sources
audio, typically dolby pro logic or dolby surround, depending on source
sample rate 48 hz, 160 bitrate

These are sometimes different, but this is typically what I start with. I get the same chapter marker results no matter what, though.

As an aside, I got a new Mac Pro (8x 3ghz) last week and Children of Men ripped just fine. I tried a couple others and got the same result as my old 4x 2.5 ghz G5, but haven't tested a lot of discs on it. Not sure why the processor architecture (or some other hardware component) affected the encode.

Cyander · Post by **Cyander** » Fri Jun 15, 2007 6:51 pm

tvcity6455 wrote:My settings are typically:

m4v container
x264 main codec
1500 or 2000 avg. bitrate
frame rate same as source
2 pass
anamorphic (cropping as necessary)
deinterlacing only on interlaced sources
audio, typically dolby pro logic or dolby surround, depending on source
sample rate 48 hz, 160 bitrate

These are sometimes different, but this is typically what I start with. I get the same chapter marker results no matter what, though.

The only settings that can have an effect on chapter markers should be the frame rate. The problem with HB is that 'same as source' doesn't actually do what it says on the tin. Same as source always does 23.98 fps, even if the source is really 29.97 generated from pulled-down source. This causes frame drops that make for entertaining debugging.

As an aside, I got a new Mac Pro (8x 3ghz) last week and Children of Men ripped just fine. I tried a couple others and got the same result as my old 4x 2.5 ghz G5, but haven't tested a lot of discs on it. Not sure why the processor architecture (or some other hardware component) affected the encode.

Very interesting indeed.

I think what I will have to do is produce a debugging build of HB which logs the logic HB goes through to a file and have you reproduce the bug. It will help me track down the errant code a lot faster, since I only have one title that currently reproduces, and I am spewing logs anyways.

hawkman · Post by **hawkman** » Sat Jun 16, 2007 3:09 pm

I just had chapters drop out after ch. 13 on Pirates of the Caribbean 2, PAL version. First time I've seen it happen! svn r622.

tvcity6455 · Post by **tvcity6455** » Sat Jun 16, 2007 3:17 pm

Cyander - let me know when the debugging build is ready and I'll start doing the encodes for the logs.

Just to see what would happen, I did try a couple encodes yesterday and today setting the frame rate to 29.97 instead of same as source but the chapters still dropped.