HandBrake

Okay, so I have somewhere around 12+ TV series I still need to import into iTunes from my collection. 0.9.2 with VFR has got me interested in ripping these TV series that are actually variable framerate and I wrote off awhile ago. So, I wrote a couple scripts for the CLI, loop in AtomicParsley to tag my rips immediately after the encode automagically... and I also upgraded my Rev A Mac Pro to use 2.66Ghz Clovertowns. So here is what an 8-way 2.66Ghz, 8GB Mac Pro is pulling down:

2 simultaneous encodes use over 750% CPU using x264 as the encoder. Wow.
A 26-episode half-hour series using a tweaked iPhone High Res preset (but outputting H.264) takes under 4 hours to encode using 2-pass.
Turbo-Pass is producing 250fps avg (300+ peak), while the second pass is producing 125fps avg (180 peak). I would like to see Turbo.264 pull those sorts of numbers out of its hat.

Just... obscene. And it makes my task of ripping over half a TB of DVDs seem pretty reasonable, rather than daunting as heck.

Thats just sick

Cyander wrote:Okay, so I have somewhere around 12+ TV series I still need to import into iTunes from my collection. 0.9.2 with VFR has got me interested in ripping these TV series that are actually variable framerate and I wrote off awhile ago. So, I wrote a couple scripts for the CLI, loop in AtomicParsley to tag my rips immediately after the encode automagically... and I also upgraded my Rev A Mac Pro to use 2.66Ghz Clovertowns. So here is what an 8-way 2.66Ghz, 8GB Mac Pro is pulling down:

2 simultaneous encodes use over 750% CPU using x264 as the encoder. Wow.
A 26-episode half-hour series using a tweaked iPhone High Res preset (but outputting H.264) takes under 4 hours to encode using 2-pass.
Turbo-Pass is producing 250fps avg (300+ peak), while the second pass is producing 125fps avg (180 peak). I would like to see Turbo.264 pull those sorts of numbers out of its hat.

Just... obscene. And it makes my task of ripping over half a TB of DVDs seem pretty reasonable, rather than daunting as heck.

Great, now I feel compelled to buy an 8-core Mac Pro with 8GB of RAM... do you work for Apple sales or something?

Actually, a Harpertown Mac Pro should be able to get even better performance than I am getting. I went with buying a couple Clovertown CPUs for my Rev A since I found a really cheap source for them that made it make sense over buying a new one.

Oh, and I stand slightly corrected on my own numbers... a 26-episode TV series should take about 3 hours 15 minutes, give or take a few minutes.

Cyander,

When I finally get my Mac Pro, I want to optimize its configuration for HB. How much RAM is in use while the two encodes are running?

Thank you
mike693

That depends. It shouldn't need more than a couple hundred MB at any one time (it seems to stabilize at about 200MB per instance of HB).

In my experience, 0.9.2 has a memory leak which can lead to crashes, but even then, HB only has ~200MB active. So having more RAM won't make HB any faster if it has the 200-300MB free it likes when encoding.

Thanks for the reply. I guess I meant to ask how much total memory is used when you run HB.

I have observed Inactive memory use increases significantly during HandBrake use, until a minimum of Active memory remains. Limited Active memory leads to swapping. I understand this is not HB itself. I suspect this is the filesystem cacheing the VOBs as they are read in.

I figure that, assuming Mac OS X processes can use up to 2GB each, I would need about 2GB for filesystem cache (Finder?), about 500MB for other OS overhead, and then 300MB per instance of HB. (I can't really test this as I am maxed out at 2GB on my Mac Mini.)

Does your Mac use less than 4GB when running two instances of HB?

Why would you swap is the OS is using the memory as cache? It should convert cache to app memory dynamically as required and only swap if cache approaches zero and you're out of physical free memory.

Have you actually seen this swapping occur?

Rodney

I rip four half hour episodes an hour as 1500kbps deinterlace Xvid and I have a dual core 2.13GHz. Somehow I don't feel I'm missing out on much although I am considering a quad core processor. I would expect you to have got better times unless H.264 adds a huge amount of overhead.

xVid will see more improvement from clock speed rather than more cores. it's not well multi-threaded so in total, handbrake only uses 1 to 1.5 cores. Same with ffmpeg.
Quad core isn't really the best option unless you like encoding into h.264 (using x264 which works really well on quad core processors)

rhester wrote:Why would you swap is the OS is using the memory as cache? It should convert cache to app memory dynamically as required and only swap if cache approaches zero and you're out of physical free memory.

Have you actually seen this swapping occur?

Rodney

This is what I observed after a fresh reboot and starting a new CLI encode. No other apps were running except activity monitor. As the encode progresses, Inactive memory steadily grows until 1.98 of 2GB of total memory is used. At around the 1.9GB mark, page outs start to grow from zero. At the end of the encode, page outs were 59MB.

I was just curious if the OS will cache more than 2GB for the VOBs HB uses as input, and if running HB alone will consume all free memory on a 4GB machine.

Again, I don't believe this is HandBrake memory usage - it tells you at the end of the encode in the Activity Log precisely how many bytes it allocated. I believe you are looking at disk caching, and observing the OS swapping out parts of itself to favor the cache for performance. This has nothing to do with HandBrake, it is how virtually all Un*x kernels manage memory to maximize performance - any memory not needed by an application will go to disk cache.

Rodney

sr55 wrote:xVid will see more improvement from clock speed rather than more cores. it's not well multi-threaded so in total, handbrake only uses 1 to 1.5 cores. Same with ffmpeg.
Quad core isn't really the best option unless you like encoding into h.264 (using x264 which works really well on quad core processors)

I run multiple encodes at the same time because I noticed that it wasn't well multi-threaded. If I went to quad core I would just run four encodes at the same time. That way I should expect twice the encode rate which would really be useful. I've tried playing with H.264 but the difference in output doesn't justify the extra time encoding it and I only watch the videos on my PC.

rhester wrote:Again, I don't believe this is HandBrake memory usage - it tells you at the end of the encode in the Activity Log precisely how many bytes it allocated. I believe you are looking at disk caching, and observing the OS swapping out parts of itself to favor the cache for performance. This has nothing to do with HandBrake, it is how virtually all Un*x kernels manage memory to maximize performance - any memory not needed by an application will go to disk cache.

Rodney

I have seen memory leaks in HB that will lead it to use 4GB of VM and crash with 0.9.2 on long queues or certain movies with VFR on... You are right that HB shouldn't have growing memory usage (and it shouldn't really exceed 200MB or so), but my experience and analysis with Instruments shows otherwise. I have opened a thread attempting to discuss it in the bugs forum.

I have the problem with HandBrake 0.9.3 and Leopard 10.5.6
All my RAM is going Inactive. Slows down my machine, though nothing crashes.
This article suggested that disk reads for large media files could be non-cached, which might alleviate the problem.
http://discussions.apple.com/thread.jsp ... &tstart=30
I supposed this problem is really at the core of the library which does the transcoding, as opposed to the GUI.

I have the new Intel Xeon “Nehalem" Mac Pro (2x Quad 2.26GHZ) with 8GB Ram.

Tries using Handbrake to do some file conversions using the ATV preset at 63% CB and every time Handbrake crashes about half way through. Is there something I am doing wrong as the main reason I bought a Mac Pro was to encode my DVD & HD DVD library lol

Without any logs, your only option is to send me your Mac Pro so I can perform some exhaustive testing (ahem ) at this end.

C'mon Nate, you should know better.

I'll post it out tomorrow lol

Sorry to hijack the thread slightly. It was more of a tongue in cheek comment in that I just wondered if I was being a complete pleb and not doing something correctly as its been a while since I did any encoding. I currently have one final encode running and will see if that too crashes. If it does then I will start up a new thread here and hopefully you can lend a much appreciate hand TedJ

On a slight side note, since I was last using Handbrake I must say that the quality of encodes has really stepped up a notch or two. Literally no decernable difference between source and output file even when using inbuilt presets. If I can get my Mac Pro working with it, you have a donation winging its way over to you.

Will keep people posted as I guess I am as keen as you all to see just how quick these new Intel processors are at crunching DVD's & HD stuff. Appears a few here have the previous gen Mac pro's and are getting some great results, so would love to see whats possible with the latest gen.

Encode is being left overnight as only started it a minute or two ago, so will post in the morning. Many thanks, Nathan

EDIT: It just crashed again about a third in and only getting about 30/40 fps on encode (not entirely sure if thats good or not as not as up to speed with all of this as I would like to be). Will start new thread now though and sorry to OP for slight tangent I have taken the thread. Thanks again.

nate247 wrote: If I can get my Mac Pro working with it, you have a donation winging its way over to you.

while we appreciate the thought. as know we do not accept donations.

They do take beer and liquor donations, though. Send them to my address, and I will dutifully forward all proceeds in a timely and fair fashion starting with the most polite mods and working from there.

Now who were the guys working on this program again?

You know, slightly off topic (but concerning donations), it was mentioned awhile back that as a substitution for donating to the Dev crew, a donation be made to a worth-while international organization. You guys ever give any more thought to that? I have several dollars that will not be going to my usual political party due to complete and utter failure to perform. I'd love to send it elsewhere to something else I derive value and entertainment from...

The problem with the idea, is deciding where. It's not something that's been thought about much yet.

I still like the EFF idea. Lately, they've assisted in some precedent setting cases with both the RIAA and MPAA, not to mention wire-tapping and DRM-related issues. I'll shift this over to the already started thread just to keep things organized.

Well I would most certainly be happy to do so if it ever came to fruition...

Back on the subject of speed. My Mac Pro (newest 8 core model) is running at about 50/60 frames per second average whilst currently encoding my three series of Battlestar Galactica DVD's. Approx 15/16 mins per 45 minute episode. Have no idea if this is good or not, so looking for you guys to say yes / no lol

CBR 59% being used BTW...

nate247 wrote:Back on the subject of speed. My Mac Pro (newest 8 core model) is running at about 50/60 frames per second average whilst currently encoding my three series of Battlestar Galactica DVD's. Approx 15/16 mins per 45 minute episode. Have no idea if this is good or not, so looking for you guys to say yes / no lol

CBR 59% being used BTW...

well, I presume by "CBR" (which would be constant bit rate) you mean Constant Quality. At any rate, the options for x264 would probably mean even more than the "CQ" being used as far as benchmarks are concerned.