Hardware Choices and encode times?

[evilHRcat]

This is more for my own curiosity than anything else, but I wonder what kind of hardware everybody else is running *NIX HandBrake on and what kind of utilization you are seeing and why...

Here goes:
Re-purposed Dell Power Edge 5900:
Intel Xeon 5900 series CPU: (x2)
2.5 GHz
Quad core
12MB Cache
12GB of 667 MHz FBDIMM DDR2 RAM (12x1GB)
(4x) 264GB 10,000 RPM Fujitsu SAS HDDs in a RAID 5 Configuration on a Dell iPERC RAID controller with 1GB buffer and dedicated battery backup.
(4x) 1TB 7,200 RPM Hitachi SATA HDDs in a RAID 5 configuration (on the second backplane) on a Dell iPERC RAID controller with 512MB buffer and dedicated battery backup.
16x SATA DVDR/RW ODD
Running Fedora 12 x64

I am lazy, so I run the GUI version of HB

So I have a total of 8 cores (2x4) and 12 GB of RAM. For no reason I can divine, I am only using a maximum of about 65% on any given CPU and VERY high quality jobs take upwards of 16 hrs to finish... if HB would just max out all 8 cores I should think it would be done a lot faster...

And for high quality, I use these options:
subme=8:trellis=2:ref=16:bframes=16:me=umh:merange=64:no-fast-pskip=1:no-dct-decimate=1:b-adapt=2:direct=temporal
And have a target size of 4096MB, storage is not an issue and I stream them over gigabit ethernet to a box hooked to the TV. I want the MAXIMUM quality possible, and a fast rip/encode. is that so much to ask?

Am I doing something wrong?
is there an option I am missing somewhere?
Anybody running anything else like this and having the same "issue"?

[JohnAStebbins]

Your not doing anything wrong. x264 peters out at about 6 cores. I see the same behaviour on my 8 core hyperthreaded (16 cores detected) xeon 5520s. I think my next system will be the highest clocked 6 core cpu I can afford.

[evilHRcat]

Bummer. With the relative cheapness of computers especially if you figure $/core, its just getting silly, you'd think that you could ramp up the thread count to some insane number if the hardware could support it.
I guess we'll just have to wait for Intel to release the Polaris before that really becomes an issue.
O well...

[mduell]

You'll gain a lot more speed without giving up much on quality from reasonable ref/bframe/merange settings (say 6/6/16) than expecting more parallel acrobatics out of the x264 team.

Just because you can turn the setting up to "11" doesn't mean it's a worthwhile tradeoff.

[evilHRcat]

I know. And again, its not really that important. I was more curious than anything else, I run my encodes at night and its done by the next afternoon. I go so crazy on the quality because I have a really nice TV and every little imperfection sticks out like a sore thumb. Every blocky, pixelated, distorted frame shows up really bad. My BD player sucks at playing DVD's, that's why I'm ripping all my DVD's and streaming them, I miss the quality of my old Panasonic DVD player that bit the dust.

[creamyhorror]

subme=8:trellis=2:ref=16:bframes=16:me=umh:merange=64:no-fast-pskip=1:no-dct-decimate=1:b-adapt=2:direct=temporal

1. direct=temporal? Why? direct=spatial or auto makes more sense; under auto mode, spatial is used the lion's share (>95%) of the time.

2. You use pointlessly slow settings like bframes=16, ref=16 and merange=64, but leave subme at 8? Doesn't make sense. Barely any live-action uses more than 6 or 7 bframes at most, and merange=32 is about the highest I've ever heard recommended. The use of very high ref and bframes gives basically negligible objective-quality returns, from an assessment I've seen, despite their very high time cost. OTOH, it's worth using subme 9 instead of 8 if you have the time to spare.

I suggest this instead:

subme=9:trellis=2:ref=7:bframes=7:me=umh:merange=32:no-fast-pskip=1:no-dct-decimate=1:b-adapt=2:direct=auto:rc-lookahead=50:aq-strength=1.2:deblock=-2,-2

(you can up ref and bframes to 9 for simple cartoons/anime)

And don't use a target size of 4GB, that's inefficient if applied to every movie (it's too much for probably 98% of them). Use constant quality (CRF) mode with a quality level that almost guarantees transparency, like 18.5, 18, 17.5, or 17. At these values during normal playback you're certainly not going to be able to tell if you're watching the original or a transcode.

[evilHRcat]

I'll definitely try Constant Quality on my next run. And again, the problem I have is ripping DVD's at anything less than lossless causes blockyness, pixelation, and other artifacts that show up on my TV all too well. The settings I use come from about 20 rip/encode sessions that looked like crap, so I progressively turned up the settings until it looked good. We are deviating massively from my original question: Why does my 8-core machine not fully utilize all 8 cores? x264 only uses 6 cores. question answered. This is still really interesting. I don't have any screenshots or anything, but if you could see it, you would all know what I'm talking about. The worst one I have seen so far has been Avatar: A good example is at 0:26:15 to 0:26:26. The tree on the left and the light beams on the right of the frame pixelate and block up very noticeably as the shot pans. On anything less than the "insane" settings I use, it looks like total crap (again, on my TV). Also, Anything less than these settings (and what a lot of the 20 tests were based on) makes The Animatrix almost unwatchable.
Thanks for the advice creamyhorror. I'll copy/paste that into HB and do a test run tonight and see what happens.

[olmari]

What kernel and CPU scheduler does your OS use? AFAIK there is some major speed loss in certain kernels that have been fixed, as per x264 developers says in here: http://x264dev.multimedia.cx/?p=185

[creamyhorror]

I don't have any screenshots or anything, but if you could see it, you would all know what I'm talking about. The worst one I have seen so far has been Avatar: A good example is at 0:26:15 to 0:26:26. The tree on the left and the light beams on the right of the frame pixelate and block up very noticeably as the shot pans. On anything less than the "insane" settings I use, it looks like total crap (again, on my TV). Also, Anything less than these settings (and what a lot of the 20 tests were based on) makes The Animatrix almost unwatchable.

The sort of artifacting you're talking about (pixelation and blocking) is pretty serious and shouldn't appear at any decent level of settings. Obvious artifacting is generally not due to lower/faster settings, but rather indicates a deeper problem, or incorrect preprocessing.

If you play the encode on your computer and don't see the artifacts, then the decoder in your box (a BD player?) is problematic. In that case, it's not a matter of quality settings, but instead simply an incompatibility issue. For example, quite a few decoders (including some BD players) don't support weightp=2 (a default setting in x264/Handbrake), and the result is very obvious blocking and visual corruption. If you encode with weightp=0, the issue disappears.

[TedJ]

If the issue is in fact with your playback device, be sure to send them a written complaint... weighted P-frames have been part of the H.264 spec since 2004.

http://x264dev.multimedia.cx/?p=212

[dynaflash]

For example, quite a few decoders (including some BD players) don't support weightp=2 (a default setting in x264/Handbrake), and the result is very obvious blocking and visual corruption. If you encode with weightp=0, the issue disappears.

Specific example would be the AppleTV. Add even weightp=1 and it turns to an absolute disaster. Decoder just does not grok it. Period. Does not matter what else you do with settings it sucks at best.

[TonyPh12345]

Home-brewed AMD Phenom II X6 1055T processor on an OC'd MSI Motherboard and 6GB of DDR3/1333 RAM.

Running Fedora FC12 kept up-to-date.

Using High Profile, 4:3 content I get about about 190fps. 16:9 content around 170.