[patch] BluRay, AVCHD & partial DVB H.264 support

[van]

This patch adds support for UNENCRYPTED BluRay and AVCHD media streams and partial support for DVB H.264 streams (e.g., digital TV in NZ & Norway).

The BluRay support has been tested with MPEG2 & H.264 video but not VC-1 (there's code for VC-1 but it hasn't been tested since I don't have any bitstreams that use it). Note that this code works on the bitstreams (the .m2ts files). It doesn't know anything about the disk metadata so it won't do chapters, concatenate multiple .m2ts files into a complete feature, etc. I.e., this is a very long way from complete BluRay disk support.

The ffmpeg support for high profile H.264 decoding has gotten pretty good over the past few weeks but is really bad in the version of ffmpeg in the HB contribs. I'd suggest updating ffmpeg to the latest svn if you're going to play with this. Apply this patch after the main patch above to make HB's contrib Jamfile & ffmpeg patches compatible with the latest ffmpeg (I did most of my testing with r13086 but today's r13111 also seems to work well).

The AVCHD support has been tested with a Panasonic HDC-SD1 & Canon HF100 but should work with anything that follows the standard.

There's complete support in HB for DVB H.264 but unfortunately there's a huge hole in the open source support for the LATM-encapsulated AAC audio used in NZ & Norway. I tried to get Nicola Sabbi's LATM AAC mods for mplayer working in ffmpeg using both faad2-2.8.1 & the current faad2 SVN with no luck (Nicola's mods had already been added to faad2 SVN but got reverted a couple of weeks ago for unspecified "problems"). I think this is an issue that has to be fixed in faad2 (and possibly the ffmpeg interface to faad2) but as soon as it is fixed (i.e., as soon as ffplay can handle the bitstreams on http://www.geekzone.co.nz/Fossie/4877) and faad2 is added to the HB contribs so that ffmpeg can use it, HB will handle these streams.

The patch touches a lot of files because too many things 'knew' they were dealing with an mpeg2 bitstream, particularly 'scan'. scan now uses the video decoder work object to create previews rather than a private interface to libmpeg2. It also uses the audio decoder work object to extract audio bitstream information. So scan no longer has any built-in bitstream knowledge - that all comes from routines local to a particular decoder. This change required adding two new methods to the work object, one used for video decoders to supply information about an in-progress decode and one used for audio decoders to extract bitstream characteristics from a packet.

The other big change was to support polymorphic codecs like ffmpeg without adding an individual HB wrapper for each supported format. There's now one HB decoder object for all ffmpeg audio decoders & one for all ffmpeg video decoders and these HB decoder objects contain a 'codec_param' that selects the particular ffmpeg codec to use for the HB decoder instance.

The patch above enables all ffmpeg decoders & demuxers. This is much more than what's needed for BluRay but I'm hoping to have a patch that makes use of the other stuff soon.

A few unrelated fixes got lumped into this patch because I was too lazy to factor them out:

- fixed a race in reader.c that would occasionally corrupt audio data when both AAC & pass-thru were selected.
- start scan's top cropping search at line 2 rather than 0 so it doesn't get confused by the close caption data encoded on line 0.
- simplified & sped up the buffer allocator and got rid of a small leak.

[s55]

Incredible. I'm Speechless.
Fantastic work!

[cvk_b]

This is huge. Great!

[jbrjake]

This changes everything =)

Here's a condensed version -- both patches in one, plus the new version of ffmpeg (which I've placed on download.m0k.org), plus some minor header location changes (libavcodec/avcodec.h instead of ffmpeg/acvodec.h, etc).

http://handbrake.pastebin.ca/1014967

NOTE: If you're on a PowerPC Mac still, it seems libswscale has a few minor issues. If it fails to compile, then in the libavcodec part of contrib/Jamfile, add --extra-cflags='-faltivec' to the configure options for ffmpeg. Will update later today or tomorrow with some PPC detection to insert it automatically when needed.

UPDATE:

Wow, I just tried this out on an H.264/AC3 .ts broadcast. Seeing the log output coming from HandBrake, saying it was encoding an H.264 input track, is a beautiful thing. Thank you so much for this, van. You've made HandBrake relevant.

[Ritsuka]

And if you have trouble compiling ffmpeg and an intel mac, update the developer's tools to the latest iphone sdk (it contains a fixed version of ld).

[brianlees]

I think I just wet my pants!

[eddyg]

Hi Van,

I like the new buffer management, I agree that there was no need for the complexity of dynamically allocated buffer pools - just stick with ^2 ones. Sure simplifies things.

I'll have to take a closer look at the buffer recycling with the new code on a real encode. It was tricky to read the code in that diff, and I haven't patched it. I'm a little worried that the reader which uses a lot of small buffers at a time will flush the big buffers that render and subtitles uses from the buffer reuse queues.

Well done on the codec independence, that is a huge first step towards any to any encoding. And also a huge thanks for the DVB-T AVC support - my EyeTV will arrive this week - and I'm not sure I want to use the EyeTV exporter for my encodes, would prefer to use HB.

With changes this big I propose that we do a snapshot build, then Van commit his changes, and we flush out any bugs then - living with any instability. We're not really very well equipped for large code reviews on this project.

Cheers, Ed.

[dynaflash]

With changes this big I propose that we do a snapshot build, then Van commit his changes, and we flush out any bugs then - living with any instability.

Personally, I am ecstatic and completely agree, snapshot then commit. Work stuff out on the fly right in the svn. As has been said before, this is huge.

[eddyg]

And the next step will be to have different input file types.

Having an AVI and MKV demuxer would be a nice start, feeding the streams from those containers into the nice new infra that Van has provided.

Cheers, Ed.

[van]

And the next step will be to have different input file types.

Yes, that's what I was referring to in this obscure comment:

The patch above enables all ffmpeg decoders & demuxers. This is much more than what's needed for BluRay but I'm hoping to have a patch that makes use of the other stuff soon.

The only remaining piece of HB that's hardwired for mpeg2 is the demuxer. That's much easier to fix that the audio & video since it's only used in two places, scan & reader, and is stateless. In the next patch set the demuxer type is a parameter in the title just like the video codec is. There are only two demuxers needed, the current hb_demux_ps and a new hb_demux_null that just returns its input buf (since its designed to be used with an ffmpeg file reader that has already done the demuxing).

With this in place it was easy to add an interface to ffmpeg using the existing hb stream api (hb_stream_open / close / title_scan / read / seek) since everything but title_scan maps trivially into the ffmpeg api (av_open_input_file / av_read_packet / etc.). The nice part of this is that a two line change to reader & scan allows HB to handle any file that ffmpeg can handle but 100% of the HB UI & functionality is preserved. It also preserves HB's ability to handle richly structured input objects (ffmpeg deals with files & has nothing equivalent to HB's DVD model). And it lets us pick & choose between HB codecs & ffmpeg's codecs. Currently I think HB's TS demuxer is better than ffmpeg's. And I did a lot of my debugging of the patch above using the ffmpeg mpeg2 decoder rather than HB's which convinced me that HB's mpeg2 decoder is currently superior to ffmpeg's -- ffmpeg is very casual about time stamps - it's hard to associate a pts with a frame and, at best, the closest you can get on video with bframes is within four or five frames of the correct one. HB's mpeg2 decoder hard ties the pts to the frame all the way through the parsing & decoding pipeline so its frame times are exact (which is why we can do frame accurate chapter marks & sub-frame accurate A/V sync).

I currently have this ffmpeg reader code limping & am hoping to post a patch by the end of the week (I leave on a two week business trip then & want to get the patch out before I go). It works pretty much as expected - you can feed HB avi's, mkv's or mp4's & they work just as well as mpg's (mp4's require that you stick faad2 in HB's contrib directory & patch a bunch of things so that ffmpeg will find faad during its configure & add aac decoder hooks).

[jbrjake]

you can feed HB avi's, mkv's or mp4's & they work just as well as mpg's

:O
=)

[eddyg]

Oh Yum!

[realityking]

:O
Van should get a medal.

[refulgentis]

wow...this is awesome. i'm putting a hold on converting any more files until the mp4 and mkv demuxers are available so I can finally have the power of a gui instead of cobbling together a mencoder/ffmpeg/mp4box shell script like I have now.

[Cavalicious]

Oh the horror of being a Mod when this hits Prime Time....

...But love it none the less...DV here we come!

[OwlBoy]

man, if support for anything ffmpeg can do comes to pass, HandBrake will become even more popular. I second the post above me about mods. And I would throw in devs too. They get frustrated enough with the users who just do DVD rips. Let alone people who "have an AVI I wanna convert".

[Firedpottery]

Hey Van, just curious, right now, how long does it take to rip a BluRay stream? Just an example is fine (4 minute clip taking 40 minutes or insert time and details of quality settings, etc), and what kind of rig are you runnin? I suspect your latest achievement has doomed some of us to an 8-core Mac Pro

[nightstrm]

I understand that everyone is excited about the possibilities opened up by this thread, but can we please keep this post to feedback about the code, and not general questions about support or gushing "OMG this is t3h awesome" posts?

Thank you.

[van]

An updated patch that adds support for ffmpeg-based file reading is at http://pastebin.ca/1019443. I've tested it on a random set of avi, wmv, mkv & mov files and it's functional. A/V sync isn't great but I'm out of time and will have to work on that when I get back. The code is ugly. Eventually I want to make a 'content reader' object that encapsulates all the low-level HB I/O methods (open/title_scan/read/seek/etc.). This would simplify a bunch of stuff in scan & reader and allow stream.c to be split into pieces. In particular it would allow all the ffmpeg stuff to be split off into its own self-contained source.

Some random notes:

- All the audio & video in non-mpeg files is decoded by ffmpeg decoders (e.g., no pass-thru AC-3 from mkv's). I hope to fix this but it takes a lot of time to understand the coupling between the ffmpeg reader & decoders and things currently die a horrible death when I uncouple them.

- Because of the coupling, there's a new audio decoder for 'internal' ffmpeg streams (in ffmpeg the stream decoders are very tightly coupled with the reader/demuxer/parser). The GUI doesn't know about it & it shows up as 'LPCM'. This can only be used with streams from the ffmpeg demuxer.

- A lot of the files I tested have 1:1 PARs which HB doesn't handle very well. I (or someone) need to do some work on the aspect ratio stuff in scan to handle more than 4:3 & 16:9 since we now get a lot of weird sizes.

- I didn't include the changes to add faad to the contrib/Jamfile patch since it requires adding the tar to the repository. So right now you can't decode a file with aac audio (since ffmpeg doesn't decode aac unless it detects faad at configure time). I did configure faad locally & tested that you can feed an mp4 output from HB back into HB

I'll be traveling for two weeks with intermittent Internet connectivity. Please report bugs or problems but understand that there might be a significant delay in the response.

- Van

[eddyg]

Have a good trip Van - and well done and thanks for this patch.

Cheers, Ed.

[jbrjake]

Van, anytime you need, want, or just would find it convenient to have any new contrib libraries put up on the server, don't hesitate to send me or dynaflash or sr55 a message.

Thank you, again, for what you're providing here.

I've only tried it on one source so far, but it worked perfectly. 30 minutes, 25fps, xvid+mp3 in avi, and it came out fine with no audio de-synch. I'll be throwing everything I've got at it while you're away. I have a feeling a lot of other people will be as well

[dynaflash]

- I didn't include the changes to add faad to the contrib/Jamfile patch since it requires adding the tar to the repository. So right now you can't decode a file with aac audio (since ffmpeg doesn't decode aac unless it detects faad at configure time). I did configure faad locally & tested that you can feed an mp4 output from HB back into HB

Last night on irc eddyg provided a mod to van's patch that properly compiles and links ffmpeg agains faad2 if it was put in the users local contrib directory. I took that this morning and modded it to get faad2 from our contrib repo on rocky, and uploaded faad2-2.6.1.tar.gz (the same version vlc uses) to the repo.

jbrjake then patched decavcodec.c to better handle the 1:1 aspect ratios that van mentions above ( I will leave those specifics to jbrjake).

Here is the modified patch against hb svn 1462 which will compile without any local changes: http://handbrake.pastebin.ca/1020321

I just fed an hb mp4 encode back into itself... its a beautiful thing

[eddyg]

I was using this for on the fly conversions of my apple tv files to iPod files. Which I thought was pretty cool.

[eddyg]

if you are getting crashes or audio issues then check your audio sample rate. Look in the scan output and make sure it matches the settings for the original file.

[s55]

With dynalfash's patch it won't compile on cygwin.

http://pastebin.ca/1020576


Edit: Fixed by adding libfaad to the trunk jamfile