Using external copies of e.g. x264

[pcordes]

Does anyone use HandBrake with a libx264.a from /usr/local/lib (for example), instead of letting it use the version of x264 that HB downloads during its build? HandBrake's x264 tarball is a bit out of date compared to the latest x264 git tree. (It's not that old, though: contrib/patch-x264-aq.patch is now obsolete, since AQ was merged upstream. It's not used by the Jam rule, AFAICT.) I always build my own latest-git/svn x264 and mencoder (and now HandBrake) before I encode something if I haven't updated them in a while.

Anyway, so I really want to use the latest git x264, but it would be nice to use the Ubuntu-packaged versions of other libraries, like liba52, libogg, libvorbis, etc, just to save time downloading/compiling them.

I just thought I'd ask here before I try to figure out how to tell jam about alternate ways to satisfy dependencies by looking in system include and lib paths...

Thanks,
Peter

[saintdev]

Does anyone use HandBrake with a libx264.a from /usr/local/lib (for example), instead of letting it use the version of x264 that HB downloads during its build? HandBrake's x264 tarball is a bit out of date compared to the latest x264 git tree. (It's not that old, though: contrib/patch-x264-aq.patch is now obsolete, since AQ was merged upstream. It's not used by the Jam rule, AFAICT.) I always build my own latest-git/svn x264 and mencoder (and now HandBrake) before I encode something if I haven't updated them in a while.

Anyway, so I really want to use the latest git x264, but it would be nice to use the Ubuntu-packaged versions of other libraries, like liba52, libogg, libvorbis, etc, just to save time downloading/compiling them.

I just thought I'd ask here before I try to figure out how to tell jam about alternate ways to satisfy dependencies by looking in system include and lib paths...

Thanks,
Peter

There are several patches floating around for this (although most of them attempt to use dynamic libs not static). It wouldn't really be that hard to do. However, we do have a much more up-to-date version of x264 in the svn code.

[pcordes]

There are several patches floating around for this (although most of them attempt to use dynamic libs not static).

Ok, I'll take a look. I want to use the shared libs from Ubuntu, except for x264.

It wouldn't really be that hard to do. However, we do have a much more up-to-date version of x264 in the svn code.

Yeah, that's what I'm using. HandBrake SVN downloads x264 core 60 r912M, from July 12. (The M means modified from upstream git copy). Latest git is core 60, r928+7. (err, the +7 is local git changes, all but one of which are just merges. If you look at x264's version.sh, rxxx is just a count of git changes.). Also, I compile x264 with profile-directed optimization (make fprofiled VIDS=../sample.y4m). x264 defaults to building only a static lib, since it runs faster, so that's what I do.

Most of the git changes between r912 and r928 are purely cosmetic, e.g.
"stats summary: print distribution of numbers of consecutive B-frames", but also
"Improve intra RD refine, speed up residual_write_cabac"

Thanks.

[jbrjake]

I want to use the shared libs from Ubuntu, except for x264.

Uh, good luck with that. You realize that HB applies all sorts of patches to the libs it uses and requires very specific versions of, say, mpeg4ip to even compile? It's not like we use static libs at specific versions just to be difficult. x264 is one of the very few you *can* currently safely change out.

[pcordes]

I want to use the shared libs from Ubuntu, except for x264.

Uh, good luck with that. You realize that HB applies all sorts of patches to the libs it uses and requires very specific versions of, say, mpeg4ip to even compile? It's not like we use static libs at specific versions just to be difficult.

You don't? seriously, I thought it was mostly to avoid external deps on platforms like MacOS X. I replaced some of the files in contrib/libs with symlinks to /usr/lib, for libmp3lame and libvorbis*. And to my custom-compiled libx264.a. Then I built ghb (the gtk gui), and it works. I haven't looked for a patch to the Jam/Makefiles to avoid clobbering those symlinks, though. I didn't try using the Ubuntu version of libmpeg4ip, or any of the other libraries that I thought might have custom patches.

x264 is one of the very few you *can* currently safely change out.

Yeah, ghb w/ my custom x264 works fine.

I'll just stick to doing that, since I'm not actually trying to package it for Ubuntu. If I was, I'd be a lot keener to figure out which libs HB could use the /usr/lib dynamic versions for.

Thanks.

[jbrjake]

Well for example...you said you wanted to use your system's copy of the ac3 decoder. But there's no guarantee your system's copy is going to have maurj's dpl2 downmixing patch applied to it, is there?

Now extend this to the general case and you can see why we never ever recommend doing what you're trying to do.

Any random system might have:

- the official, way out of date a52dec from the official project
- the official, way out of date a52dec from the official project with the new functionality patched in by the package maintainer
- graham booker's fork of a52dec from before he integrated maurj's patch
- graham booker's fork of a52dec from after he integrated maurj's patch

And that's not even counting how sometimes, HandBrake will submit a patch upstream, but the project maintainers will implement it with different names, providing the same functionality. And then we might be stuck with the older version for other reasons (say Windows compatibility) so we use the old names. So you can't even trust that when patches have been accepted upstream, that it's safe to put them in HandBrake, because how the stuff is called can vary in ways that break compilation.

[saintdev]

You don't? seriously, I thought it was mostly to avoid external deps on platforms like MacOS X. I replaced some of the files in contrib/libs with symlinks to /usr/lib, for libmp3lame and libvorbis*. And to my custom-compiled libx264.a. Then I built ghb (the gtk gui), and it works. I haven't looked for a patch to the Jam/Makefiles to avoid clobbering those symlinks, though. I didn't try using the Ubuntu version of libmpeg4ip, or any of the other libraries that I thought might have custom patches.

lame is pretty safe. we use aoTuV vorbis, so, you're potentially hurting quality by using the stock libvorbis. x264 is patched so we can write keyframes at chapter markers. This helps out on certian players. But if it's not patched, the request for an I frame is just ignored, so it won't really hurt anything.

[pcordes]

lame is pretty safe. we use aoTuV vorbis, so, you're potentially hurting quality by using the stock libvorbis. x264 is patched so we can write keyframes at chapter markers. This helps out on certian players. But if it's not patched, the request for an I frame is just ignored, so it won't really hurt anything.

Ah, I hadn't figured out what that patch was about. I'll see if I can merge it into my x264 git repository, then.

Thanks for the info, saintdev and jbrjake. This is the kind of thing that curious users such as myself love to know (i.e. what does HB have that other programs using the normal version of the library don't). Now I'm even more impressed with HB, since you guys are maintaining good versions of some of the libraries you use. I was aware of the sorry state of the Ubuntu versions of some multimedia stuff like liba52...

My first reaction when HB started downloading tarballs was "what the hell", but I understand that if you want to have time to ever work on HB itself, you have to do it that way. You could spend forever writing autoconf code to detect what flavour of libraries were available and trying to make HB work with any combination of old versions of things...

happy hacking,
Peter