How to increase CPU usage? (8-core Mac Pro)

[xenon supra]

I'm on my 8-core Mac Pro 3.0 ghz and it is hardly using the potential my computer has. It seems to be at about 40-50% CPU usage.

Any tricks to increasing this??

I searched around but had no luck so forgive me if this has already been answered. Thanks!

[xenon supra]

I actually did some searching around and found out that you can speed up a certain process in os x using the renice command in terminal.

Code: Select all

sudo renice -20 [pid]

where -20 is the fastest value and 20 would be the slowest value. 0 is normal

[pid] is the process id number from activity monitor. my cores are at 85-90% now.

[Duke]

I tried the Nice/renice commands through the GUI shareware program PlayNice. I did not notice any difference in encoding rates though I am getting my data from a firewire 800 disc and sending it back to that drive.

The only way I have been able to consistently increase CPU utilization on machines over 2 cores is to run more than one instance of the application. This will yield 90% utilization with 2 instances and combined 200fps or better on first pass encoding with my 8 core 2.8 Ghz Penryn. It has the added benefit if a file disrupts the encoding process in the middle of the night and that process terminates you still have the output from the second instance of the application.

[Frank]

I also have an 8 core mac, but a little bit older than yours

to make an encode go faster:

a) run two instances of Handbrake
b) encode in MPEG4 not H264
b) see (a) or (b)

If you do one-pass encodes, then it will consume a fairly broad swatch of the processing power available, but at the expense of quality.

If you do two-pass encodes, the first pass through will barely touch the processors and the next pass will use a fair bit of the processors.

Basically H264 encoding is not going to be hugely sped up by having many processors available since they are all interdependent to a certain extent and various encoding sequences on the different processors have to wait for results from their neighbours before they continue.

So if you still don't have that much usage, add a few more instances of Handbrake and encode like crazy.

So far my record is 4 simultaneous MPEG4 encodes using Handbrake, each encoding at about 250fps. Then again, those were iPod-only encodes.

[graysky]

@op - if it makes you feel any better, handbrake 0.93 only uses 66-70 % of my four cores. See my post here for more w/ screenshots like your's. What's odd is that I know x264 can use >99 % of the cores because when I was running XP and MeGUI, my 2nd pass DVD encodes would routinely use >99 % of the cores.

Oh well, it's still the best thing out there for LINUX x264 encodes that I've found.

[cjones051073]

Hi,

The encoding process can only go as fast the the slowest link in process, and if you are not using your full CPU power this could mean that something other than CPU is the limiting factor. Changing nice values or tricks like that is not going to help. You need to find a way to feed the data into your CPUs faster... For instance, if you are encoding direct from a DVD, then it could be your DVD cannot read fast enough to supply your CPUs. This was certainly the case for me with a 2 pass H264 encode during turbo pass1.

My solution was to first rip the DVD to disk, and then encode from the disk copy. In general harddisks can sustain a faster transfer rate than DVD drives.

Chris

[bradycl]

Are you passing threads=x (where x is # of cores) to x264?

[dynaflash]

course the other thing is filters, deinterlace for example is not threaded very well and you will bottleneck on the filter. Many thing's can affect cpu utilization.

[Gorloth]

I have a 2009 2 x 2.26 Mac Pro w/12G ram and was getting low CPU usage. I found the best way to fully max out my CPU was to
make a sub folder called Handbrake in my applications folder. Then copy 4 copies of Handbrake, renaming them
Handbrake_1, Handbrake_2 etc into this folder. Now when I run batch files I have 4 copies doing the work. Result,
all 8 cores maxed out. It used to take me 1.5 hrs to encode .H264 on my 2.4 iMac. On my MAc Pro it only takes 11 minutes for the same
file/settings. Now that rocks. Also I recommend that you use a temperature monitoring app and SMC fan control. Don't rely on Apple's
default setting to keep your rig cool. Sure it's noisy but it's cranking. With SMC Fan control my CPU never get over 50C, memory
is almost 25C cooler. I encoded 33 episode of Bi Mung Chu ~ 42 min each. It took a total of a bit over 6 hours. .h264 dual pass,
deinterlace, subtitles, DVD resolution

[JohnTheGeek]

The only way I have been able to consistently increase CPU utilization on machines over 2 cores is to run more than one instance of the application. This will yield 90% utilization with 2 instances and combined 200fps or better on first pass encoding with my 8 core 2.8 Ghz Penryn. It has the added benefit if a file disrupts the encoding process in the middle of the night and that process terminates you still have the output from the second instance of the application.

Can you elaborate? When I try to do this both instances fight over the Queue - since it is now written to disk. Any advice? Or are you just talking about the CLI version?

[TedJ]

The only way I have been able to consistently increase CPU utilization on machines over 2 cores is to run more than one instance of the application. This will yield 90% utilization with 2 instances and combined 200fps or better on first pass encoding with my 8 core 2.8 Ghz Penryn. It has the added benefit if a file disrupts the encoding process in the middle of the night and that process terminates you still have the output from the second instance of the application.

Can you elaborate? When I try to do this both instances fight over the Queue - since it is now written to disk. Any advice? Or are you just talking about the CLI version?

Looking at the date of his post, Duke would have been running 0.9.1 which didn't have a persistent queue. Running multiple instances of the current code will only work with the CLI version.

[Pecos Bill]

The only way I have been able to consistently increase CPU utilization on machines over 2 cores is to run more than one instance of the application.

On my Mac dual proc 2.0GHz dual core, the four cores are fully utilized with a single instance of the app. As cjones051073 correctly stated, whatever bottlenecks there are will be the weakest link. For me, I think it helps greatly that I rip the content using MacTheRipper then queue them for compression in HandBrake sourcing the hard drive (a fast device). If that's not enough, possibly a striped RAID (level 0) would be needed to increase the source throughput. However, if the code isn't well threaded (per dynaflash), there might not be much you can do if running a version with the persistent queue. Since I've managed to crash HB a few times with badly ripped files, I'm grateful for that persistence.

If the other projects' code is the limitation and there's multiple items enqueued, the HB developers could just work on two or more items at once spreading them to the number of cores appropriate. That would remove the need to run multiple instances and ensure full utilization.

Nice won't speed up a process but change it's processing priority. The only way you'd see an increase in speed is if you're running another processor intensive task. A -20 would make the rest of the system less responsive too.

[mduell]

The only way I have been able to consistently increase CPU utilization on machines over 2 cores is to run more than one instance of the application.

On my Mac dual proc 2.0GHz dual core, the four cores are fully utilized with a single instance of the app. As cjones051073 correctly stated, whatever bottlenecks there are will be the weakest link. For me, I think it helps greatly that I rip the content using MacTheRipper then queue them for compression in HandBrake sourcing the hard drive (a fast device). If that's not enough, possibly a striped RAID (level 0) would be needed to increase the source throughput. However, if the code isn't well threaded (per dynaflash), there might not be much you can do if running a version with the persistent queue. Since I've managed to crash HB a few times with badly ripped files, I'm grateful for that persistence.

Unless you're using an antique hard drive, the hard drive is not your limitation and RAID will buy you nothing. The most likely cause of poor CPU utilization is filters. But as long as the OP refuses to post an encoding log, all we can do is guess wildly.

If the other projects' code is the limitation and there's multiple items enqueued, the HB developers could just work on two or more items at once spreading them to the number of cores appropriate. That would remove the need to run multiple instances and ensure full utilization.

Already available in the nightlies.

[Rodeo]

If the other projects' code is the limitation and there's multiple items enqueued, the HB developers could just work on two or more items at once spreading them to the number of cores appropriate. That would remove the need to run multiple instances and ensure full utilization.

Already available in the nightlies.

Not really. The nightlies feature multi-instance, rather than single-instance, multi-encode.

[louwrentius]

To run two instances of handbrake on a bunch of files that must be (batch) processed: you may use ppss:

./ppss -d /home/user/dirwithfiles/ -c 'HandBrakeCLI -i "$ITEM" -o "$ITEM".mp4" ' -p 2

The multi-instance feature of nightlies may also be interesting.

[bluepill]

Hi,
Can someone give some advice on where my bottleneck might be?

I am running Handbrake 0.94 (32bit) on Windows 7 x64 on an 2 CPU (8 core's in total) AMD Opteron system with 16GB RAM.

I am encoding a bluray (m2ts, 1080p, H264) source into a m4v (720p, H264) and I get about 65% CPU utilization spread across all the cores. I'm having a look at the Windows Performance Monitor and I'm not sure what I should be looking for or if the numbers are good or bad.

I can see that my data is being pulled from the network drive at about 1.7MB per second and my local drive is being written to at about 150KB per second; also as I said 65% cpu utilization. All these numbers are way below what the system is capable of.

This is what my encode log has at the beginning:

### Windows GUI 0.9.4 2009112300
### Running: Microsoft Windows NT 6.1.7600.0
###
### CPU: Quad-Core AMD Opteron(tm) Processor 2352
### Ram: 4095 MB
### Screen: 1920x1200
### Temp Dir: C:\Users\***\AppData\Local\Temp\
### Install Dir: E:\Program Files (x86)\Handbrake
### Data Dir: C:\Users\***\AppData\Roaming\HandBrake\HandBrake\0.9.4.0
#########################################

### CLI Query: -i "Z:\m2ts\****.m2ts" -t 1 -c 1 -o "E:\****.m4v" -f mp4 -w 1280 --loose-anamorphic --detelecine --decomb -e x264 -q 20 -r 23.976 -a 1,1 -E faac,ac3 -6 auto,auto -R 48,Auto -B 160,32 -D 0.0,0.0 --markers="C:\Users\***\AppData\Local\Temp\***-1-chapters.csv" -x b-adapt=2:rc-lookahead=50 -C 8 -v 1
### Custom Query: False

#########################################

[16:56:22] hb_init: checking cpu count
[16:56:22] hb_init: starting libhb thread
HandBrake 0.9.4 (2009112300) - MinGW i386 - http://handbrake.fr
8 CPUs detected
Forcing 8 CPUs


Any suggestions on how I can improve my performance or troubleshoot the problem?

Thanks

[bluepill]

I just did a quick test and I can drag a file off he network drive and write it to my drive at an average of 90MB per second; so I think this can rule out the network drive read speed and the local drive write speed as the bottleneck.

I can't think of anything other than the settings for my encode and the Handbrake software itself... Any help is greatly appreciated.

[Rodeo]

The decoder isn't threaded, and since your source is HD, that could be the bottleneck.

Try a nightly build (see the announcement). The decoder still isn't threaded, but decoding performance should have improved a bit.

[mduell]

1) Disable your filters (decomb and detelecine) since your source sounds like 24p.

2) Use more complex encoding settings. Won't go any faster, but will use more CPU and produce smaller or higher quality files.

[bluepill]

Thanks for the help.
It's a shame the decoder isn't threaded. Seems a bit strange, I would expect it would be a useful part of the program to be threaded...

Yes, I am pretty sure the source is 24p. I left the default for the decomb and detelecine; which is "default". I thought it would auto-magically turn the filters on/off depending on the video content. I'll try turning them off explicitly and see if that helps.

I'm not sure what you mean by 'more complex encoding settings'; I selected the high profile from the presets. I haven't touched the advanced options tab settings; they're a bit scary, I don't know what any of it means. Any suggestions?

Thanks again.

[mduell]

The decomb and detelecine filters won't apply deinterlacing or detelecining if they don't detect interlacing or telecining, but they still need to do analysis that slows down encoding.

Paste b-adapt=2:rc-lookahead=60:ref=8:subme=9:direct=auto:me=umh:partitions=all:trellis=2 in the box as your x264 advanced options for higher CPU usage.

[bluepill]

Sorry for the delay in replying. Thanks for the advanced options. They certainly do shoot all CPU's up to 100%.