Multicore Performance Question

[mike693]

I have read in other forums that HandBrake's maximum CPU usage on a quad core machine is in the ballpark of 220 - 260%. As MediaFork will be my most CPU-demanding application, I am trying to understand the cost/benefit of upgrading to a new iMac vs. a Mac Pro.

Can anyone closer to the development confirm how efficiently MediaFork will use all four cores?

Thank you.

[baggss]

I'm sure someone can tell you more technical details, but I can tell you this. On my Quad G5, 200-275% is the norm with at least 50% on all 4 processors at any given time when MF is running.

[mike693]

Thanks, baggss. (Sorry haven't been able to get into the forums again until today.)

That confirms what I read elsewhere, and roughly how much more bang for my buck an extra CPU will get me.

For the more technical folks, I am still curious if CPU usage behavior is defined within the app, the OS level, or if a user can/should allocate more CPU to a given app (perhaps with "renice" or other tools).

[jbrjake]

For the more technical folks, I am still curious if CPU usage behavior is defined within the app, the OS level, or if a user can/should allocate more CPU to a given app (perhaps with "renice" or other tools).

renice works, I usually set it to -20.

[jhsweeney]

I typically get ~90% utilization on my Pentium D 945 based Window's XP box using the window's cli version. I get close to 100% on one of the cores and 80 - 90% on the other.

Average frame rate using xVid encoder, 2 Pass, 100% quality is 75 FPS

[mike693]

renice works, I usually set it to -20.

What does that get your CPU usage up to?

[mike693]

I typically get ~90% utilization on my Pentium D 945 based Window's XP box using the window's cli version. I get close to 100% on one of the cores and 80 - 90% on the other.

Average frame rate using xVid encoder, 2 Pass, 100% quality is 75 FPS

It's interesting how the XP box is getting more utilization.

[dynaflash]

My MacBook Pro Core 2 Duo gets about 80 - 90 percent on each core. However, one thing I have noticed is that encoding to my internal 5400 rpm hard drive using a fast encoder at low bitrate ie. MP4 ffmpeg at < 1000 kbps, my processor utilization drops way down to like 40% across both cores.

Same encode to a 7200 rpm firewire drive goes back up to 85-90 %. I am deducing from this that at the faster encoder settings, HB has to wait for the internal drive and therefore cpu % drops since when writing to the faster drive, cpu goes back up. There is a certain logic to this and my testing has proven it to be pretty consistent.

[s55]

mike693 - The windows port of the application isn't compiled nativly under windows so there are overheads caused by the CYGWIN enviroment it uses. This will probabily be the cause of the increased processor utilisation.

[mike693]

Thanks for the replies everyone. To sum up, to get the most performance, use a native *nix platform, do what you can to ensure you are not HDD constrained, and know that HandBrake will use all available cores.

[Fredster]

I've actually noticed that i get more CPU usage using HB, rather than MF, which is a bit strange....Does anybody know why, while using 2 pass, the CPU usage drops to less than half? I get this often on MF and now and then on the 2nd pass using HB?

About multicore performance, what does a (future?) MacPro with dual core quad crunch? For me, it all sounds like a wet dream

[rhubear]

Since multi-core is essentially 2 CPUs on one die (correct?), there should not be much difference between a 2 way smp machine, and a dual-core machine right?

I'm thinking of trying MF on a dual AMD Opteron running Ubuntu Linux, I am hoping that MF will take advantage of both cpus. I take it from the replies here (of MF using all available cores) that MF should do what I'm hoping for....?

PS: the machine has SATA disks, so I'm assuming those will allow good i/o!

[s55]

There is a performance difference between multicore and multi processor. The reson for this is you don't have 2 processors sharing the same busses etc.

The difference between multicore and multiprocessor isnt really significant though.

You should be able to take advantage of both cores no problem.

[rhubear]

I realised that I forgot an even bigger issue to consider...... Opteron cpus are 64bit.

Under Ubuntu I was running a 32bit chroot (32bit environment within the 64bit environment -- something like a VM) to run 32bit apps within the 64bit Ubuntu.

I assume MF is 32bit, not 64bit. Any plans for 64bit? Would 64bit even make a difference for video ripping converting?

I might have to investigate any 64bit utilities currently available....

[s55]

Errm Is it not like 64Bit windows whereby if the app is 32bit it'll still run alright?

Afaik Opteron's are 32/64bit compatable.


Theres no plans to compile Handbrake for 64bit processors however if someone were to do it we'd quite happily post a binary. I dunno if there would be any performance improvement.

[rhubear]

Errm Is it not like 64Bit windows whereby if the app is 32bit it'll still run alright?

Theres no plans to compile Handbrake for 64bit processors however if someone were to do it we'd quite happily post a binary. I dunno if there would be any performance improvement.

Hmmm, yes, Opterons are 32bit compatible, but when running 32 bit apps under Linux, you need the 32bit libs. One ends up building a "chroot" directory with the relevant libs, and relevent apps installed into the "chroot" directory. It ends up something like a VM.

I must look into whether chroots recognise smp hardware, or whether they only see 1 processor (like a vmware guest would only see 1 cpu even if the host hardware is smp).

Re recompiling HB for 64bit, if there was no coding (or little coding) involved, I wouldn't mind having a go. But if the app has to be rewritten, I'm not up to that! I've done a little coding years ago, but don't consider myself a programmer capable of rewriting any program the calibre of HandBrake or MediaFork.

[s55]

I dunno if any coding would be required. Why not try and see. Its pretty simple:

To compile. Download the SRC from the download page

./configure
Jame

Don't use the make build method. It downloads pre-compiled libaries for handbrake.

are the 2 commands you need. You may need to download/compile Jam if you dont have it installed:

ftp://ftp.perforce.com/pub/jam/ iirc

[rhubear]

Thanks for your info, Scott.

I will have to do this later..... I don't currently have my Linux (dual Opteron) machine active/running. I'll certainly try it once I do have the machine active (next few months).

It will be really cool if I can get MF (the source _is_ for MF not HB isn't it?) working native on Ubuntu / Opteron!!

PS: I see the other item on my wish list (hinting for streaming) is already on the todo list..... Can't wait to have MediaFork working perfectly for my needs!

[s55]

The src on the download page is Mediafork but Media Fork and Handbrake are the same thing.

That release was when we forked the project, we went under the name Mediafork but we have since reverted back to the Name handbrake. The next build will return us fully to the Handbrake name.

[Anonymous Freak]

Sorry to revive such an old thread, I found it via Google.

To note about 32 vs. 64-bit speed, there are two factors:

1. 64-bit data takes up twice as much space as 32-bit data. Therefore, on an instruction set that is identical in 32-bit mode as in 64-bit mode (as PowerPC is,) you actually take a slight performance *HIT* operating in 64-bit mode, since more data has to go from main memory to the CPU(s). This means you are more likely to saturate the processor bus.

2. On x86, though, 64-bit mode has more registers. This means that the processor can be more efficient in 64-bit mode than in 32-bit mode. This means there is usually a slight performance INCREASE in 64-bit mode. (I've seen synthetic benchmarks that show it should top out at about 15%. Real world will be less, of course.)

So on a PowerPC Mac, there is no compelling reason to compile as a 64-bit app. (The only time you will benefit is if you need to process actual 64-bit chunks of data; or you need more than 4 GB of RAM per process.) On an Intel/AMD machine, though, there *IS* a slight performance benefit to compile as 64-bit. (In any OS.) Of course, you need a 64-bit version of your OS, too. On Mac OS, this is easy, Leopard is 64-bit automatically on 64-bit hardware. On Linux or Windows, you have to choose 32 or 64-bit versions on install (Linux or Windows Vista Ultimate,) or purchase (Windows other than Vista Ultimate.)

The best option for the Mac OS binary would be to compile as a 32-bit PowerPC and 32/64-bit Intel Universal Binary. Obviously, separate 32-bit and 64-bit binaries would be needed for other OSes.