Leopard Issues

Archive of historical development discussions
Discussions / Development has moved to GitHub
Forum rules
*******************************
Please be aware we are now using GitHub for issue tracking and feature requests.
- This section of the forum is now closed to new topics.

*******************************
dynaflash
Veteran User
Posts: 3820
Joined: Thu Nov 02, 2006 8:19 pm

Post by dynaflash »

Moved to Development. Okay, minus the obligatory "svn builds are not in any way supported" I am taking this post as its intended, just a note. Which is cool, I believe there is at an issue with some of the linking especially in connection with x264 in leopard. I did see some talking about it on irc. At any rate, it is a known issue I believe.

As has been noted, building on tiger and running on leopard should work fine in the meantime.
eddyg
Veteran User
Posts: 798
Joined: Mon Apr 23, 2007 3:34 am

Post by eddyg »

Aren't we just waiting for the x264 folks to fix it?

FYI. I've found that my SVN builds from prior the upgrade crash in x264 when doing CRF encodes.

This means that I can't use HB until the x264 stuff is fixed. I haven't tried 0.9.1, it's so old, I like the svn versions ;)

Suppose I could use "make" instead of "jam".

Cheers, Ed.
eddyg
Veteran User
Posts: 798
Joined: Mon Apr 23, 2007 3:34 am

crash at x264_pixel_satd_8x8_sse2()

Post by eddyg »

Hi Guys,

FYI when trying to run HB with the AppleTV or my usual preset in Leopard with an SVN build (built with make) I get a crash in x264 at x264_pixel_satd_8x8_sse2().

Oddly 9.0.1 works fine as do my SVN builds from before.

Cheers, Ed.
jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Post by jbrjake »

I'd imagine this might well be fixed by https://trac.videolan.org/x264/changeset/680 , no?

I'd have tested it myself days ago, but I don't have any intel Macs, so i've been waiting for one of you guys to be a guinea pig. Dynaflash has the rev up on the server and everything. All you need to do is switch out the version.txt for x264 and jam (under Tiger, I suppose).
dynaflash
Veteran User
Posts: 3820
Joined: Thu Nov 02, 2006 8:19 pm

Post by dynaflash »

eddyg, if you use the vbv 2 pass patched version, it already jams with the 680 up on the server
awk
Enlightened
Posts: 109
Joined: Sat Mar 31, 2007 11:55 pm

Post by awk »

So.....

I grabbed the 680 sources from the Handbrake server, went through quite a bit of pain to build the libraries for both ppc and intel, ran the resultant binary and :

Thread 18 Crashed:
0 org.m0k.handbrake 0x0035b04a x264_pixel_ssim_4x4x2_core_sse2 + 265

Which looks a little different than before - but is still nonethless 'dead'.

Any more thoughts (jbrjake ?) I'm nowhere near a codec expert to go tackle this one !

BTW. If your planning to build the x264 libraries from source on leopard aside from needing both a PPC and an X86 machine (I can't seem to get the makefile to emit the 'other' sort of binary) You'll also need to add:

-isysroot /Developer/SDKs/MacOSX10.4.u.sdk

to the CFLAGS line otherwise you'll be building against the sytem /usr/lib/ tree and you'll get bunch of undefined symbols involving $UNIX2003 when you try and link against the Handbrake binaries which are built with the 10.4.u SDK in Xcode.

Oh and jbrjake - if we can fix this I can get a release of my other project out (there's motivation 8-)
jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Post by jbrjake »

awk wrote: Thread 18 Crashed:
0 org.m0k.handbrake 0x0035b04a x264_pixel_ssim_4x4x2_core_sse2 + 265

Which looks a little different than before - but is still nonethless 'dead'.

Any more thoughts (jbrjake ?) I'm nowhere near a codec expert to go tackle this one
Like I explained above, I don't have any Intel boxes so I have no way of figuring this out, because I can't test. :(

See what happens if you add no-ssim to the x264 options.
awk
Enlightened
Posts: 109
Joined: Sat Mar 31, 2007 11:55 pm

Post by awk »

OK - I can try that option.

Am I correct in thinking that this problem was 'introduced' by building on Leopard ? It's not any specific SVN change that introduced this ? Looking at the trac timelines both on Handbrake and X264 it doesn't look like very much has changed in this code for a while. Handbrake 0.9.0 and 0.9.1 both use the same R665 version of X264, and so long as they're built with Xcode 2.4 on Tiger the resulting binaries work fine when run under Leopard - correct ?

I'm beginning to be suspicious of a linker change that occurred with Xcode 3.0 - the 'default' linker in Xcode 3 is ld64 which is different from prior versions (don't let the 64 suffix confuse you - it still outputs 32 bit binaries if asked, it's just more 'capable' than the prior version). I'll try fiddling with the linker used, and the version of Xcode and see what happens.
jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Post by jbrjake »

awk wrote:Am I correct in thinking that this problem was 'introduced' by building on Leopard ?
I believe so, but no guarantees :\
awk
Enlightened
Posts: 109
Joined: Sat Mar 31, 2007 11:55 pm

Post by awk »

OK - so if you build with Xcode 2.5 on Leopard the binary is fine (on Leopard). You can do this with 'make' if you change the occurrences of 'xcodebuild' in the Makefile to $(XCODEBUILD) and then define XCODEBUILD in the Makefile.config to be /Xcode2.5/usr/bin/xcodebuild This will give you an Xcode 2.5 build environment including tools, SDKs etc.

Then do your make - and everything will be fine.

Next stop - investigating the linker used in Xcode 3.0
awk
Enlightened
Posts: 109
Joined: Sat Mar 31, 2007 11:55 pm

Post by awk »

It really looks like a problem with ld64 which is now the default linker with Xcode 3. If I remake the symlinks in /Developer/usr/bin so that ld -> ld_classic and rebuild then Handbrake works fine.

In theory you can change a build setting in the project to force the use of ld_classic over ld - however I've had no luck with that.

I'll take this up on the xcode-users list and see if anyone there has any words of wisdom. In the meantime if you want to build and run on Xcode 3/Leopard change the ld, ld64 and ld_classic stuff in /Developer/usr/bin around.
awk
Enlightened
Posts: 109
Joined: Sat Mar 31, 2007 11:55 pm

Post by awk »

More definitively the problem is that Handbrake must use the 'read_only_relocs suppress' flag when linking because of the assembly language code in libx264 - this isn't too surprising, I've not looked at that code in detail but it wouldn't surprise me if it's self-modifying in portions, that's a good 'trick' for boosting performance in some cases.

Unfortunately ld64 doesn't support the read_only_relocs flag (and worse it doesn't complain). I've filed bugs with Apple for both issues.

In the meantime Handbrake will have to be built with Xcode 2.5 on Leopard (not necessarily bad - just a bit of a pain). If you want to link in Handbrake with an Xcode 3 project (like me) the libhb dlib target seems to work OK (on intel machines at least - I've not tried ppc yet).
cacepi
Posts: 14
Joined: Sun Nov 11, 2007 6:23 pm

Post by cacepi »

Unkagar wrote:

Code: Select all

Thread 15 Crashed:
0   org.m0k.handbrake             	0x0035b801 x264_pixel_satd_8x8_sse2 + 682
It's an bug in the assembly code for x264. The x264 devs know about the problem, but I don't know if there's a fix for it yet.
awk
Enlightened
Posts: 109
Joined: Sat Mar 31, 2007 11:55 pm

Post by awk »

cacepi wrote:
Unkagar wrote:

Code: Select all

Thread 15 Crashed:
0   org.m0k.handbrake             	0x0035b801 x264_pixel_satd_8x8_sse2 + 682
It's an bug in the assembly code for x264. The x264 devs know about the problem, but I don't know if there's a fix for it yet.
Actually I think I'd characterize this more as a flaw in the linker used by Xcode 3.0

If you build Handbrake with Xcode 2.5 (on Leopard) it will work fine. The problem is that the linker used by Xcode 3.0 (ld64 vs. ld_classic) doesn't support non-readonly relocations which the 'crafted' assembly language of X264 requires.

Apple is aware of the issue though they've made no comments as to when it might be addressed. The libx264 folks are certainly free to make changes to their code such that readonly relocations work of course. In the meantime Xcode 2.5 seems an OK workaround.
cacepi
Posts: 14
Joined: Sun Nov 11, 2007 6:23 pm

Post by cacepi »

awk wrote:If you build Handbrake with Xcode 2.5 (on Leopard) it will work fine.
You could just issue -classic_linker in the build options. Works for me at least.

Or not. Just crashed:

Code: Select all

Thread 11 Crashed:
0   org.m0k.handbrake             	0x00346c6b x264_encoder_headers + 10
1   org.m0k.handbrake             	0x00034da6 work_func + 4074 (work.c:550)
2   org.m0k.handbrake             	0x0002fab2 hb_thread_func + 80 (ports.c:288)
3   libSystem.B.dylib             	0x92828075 _pthread_start + 321
4   libSystem.B.dylib             	0x92827f32 thread_start + 34
A 2.5 build works like you said.
awk
Enlightened
Posts: 109
Joined: Sat Mar 31, 2007 11:55 pm

Post by awk »

cacepi wrote:
awk wrote:If you build Handbrake with Xcode 2.5 (on Leopard) it will work fine.
You could just issue -classic_linker in the build options. Works for me at least.

Or not. Just crashed:

Code: Select all

Thread 11 Crashed:
0   org.m0k.handbrake             	0x00346c6b x264_encoder_headers + 10
1   org.m0k.handbrake             	0x00034da6 work_func + 4074 (work.c:550)
2   org.m0k.handbrake             	0x0002fab2 hb_thread_func + 80 (ports.c:288)
3   libSystem.B.dylib             	0x92828075 _pthread_start + 321
4   libSystem.B.dylib             	0x92827f32 thread_start + 34
A 2.5 build works like you said.
It looks to me like -classic_linker is ignored by Xcode 3 (no errors either - nice !)
pengvado
Posts: 2
Joined: Fri Nov 16, 2007 5:48 am

Post by pengvado »

awk wrote:The problem is that the linker used by Xcode 3.0 (ld64 vs. ld_classic) doesn't support non-readonly relocations which the 'crafted' assembly language of X264 requires.
Now that's interesting. Because x264's assembly doesn't contain any non-readonly data, relocated or otherwise. It doesn't have any relocations in .rodata either, only in .text
realityking
Veteran User
Posts: 680
Joined: Tue Apr 24, 2007 12:36 pm

Post by realityking »

Today somebody submitted a patch that allows the use of x86_64 in x264, maybe that fixes the linker issue also. In addition a patch was committed that speeds fullpel motion estimation up. Hope we'll see a new x264 checkout in HandBrake soon :)
cacepi
Posts: 14
Joined: Sun Nov 11, 2007 6:23 pm

Post by cacepi »

realityking wrote:Today somebody submitted a patch that allows the use of x86_64 in x264, maybe that fixes the linker issue also.
Well, I don't know about that, but it sure makes it a lot faster.

On a 2.4 GHz iMac with a vanilla r685 x264 build using the profile generation settings encoding "Akiyo" cif:

encoded 300 frames, 215.57 fps, 123.49 kb/s
encoded 300 frames, 102.20 fps, 415.62 kb/s
encoded 300 frames, 75.00 fps, 129.23 kb/s

With the 64 bit patch (again, on "Akiyo" cif):

encoded 300 frames, 258.41 fps, 123.49 kb/s
encoded 300 frames, 121.91 fps, 415.62 kb/s
encoded 300 frames, 89.60 fps, 129.23 kb/s

And a profiled build:

encoded 300 frames, 276.21 fps, 123.49 kb/s
encoded 300 frames, 124.79 fps, 415.62 kb/s
encoded 300 frames, 93.44 fps, 129.23 kb/s

Impressive.
realityking
Veteran User
Posts: 680
Joined: Tue Apr 24, 2007 12:36 pm

Post by realityking »

Nice :o I think I got to buy leopard pretty soon. :lol:
jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Post by jbrjake »

realityking wrote:Hope we'll see a new x264 checkout in HandBrake soon :)
http://handbrake.m0k.org/trac/changeset/1075
realityking
Veteran User
Posts: 680
Joined: Tue Apr 24, 2007 12:36 pm

Post by realityking »

Thanks jbrjake :)

Has already somebody tried to compile it using Xcode 3?
jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Post by jbrjake »

realityking wrote:Has already somebody tried to compile it using Xcode 3?
Linkage error persists.
realityking
Veteran User
Posts: 680
Joined: Tue Apr 24, 2007 12:36 pm

Post by realityking »

What a shame :/

But hopefully we'll see at least an X86_64/Darwin Build of HandBrake CLI for the next release :)
jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Post by jbrjake »

Okay, we've made some progress on IRC this afternoon.

You've gotta set x264 to not run make install in the contrib/Jamfile, and instead copy the lib and header and strip the lib manually. Then, in XCode set all the targets to build with the 10.5 sdk. Then, in the Jamrules, don't build alltargets with XCode -- something fails with IHB, which is deprecated anyway. Also, you need to patch some lines in the OpenGL stuff since it doesn't work the same in Leopard. However, after that, successful compiles.

Very rough diff here, won't go into the svn in any similar fashion:
http://pastebin.com/f3399fcec
Post Reply