Efficient use of high core counts in next gen chips

Archive of historical feature requests.
Please use the GitHub link above to report issues.
Forum rules
*******************************

Please be aware we are now using GitHub for issue tracking and feature requests.

- This forum is now closed to new topics.
- Existing topics will remain open for a short while as the transition over occurs.
*******************************
Post Reply
db1964
Posts: 12
Joined: Thu Jul 30, 2015 11:05 am

Efficient use of high core counts in next gen chips

Post by db1964 » Fri Jul 28, 2017 3:46 pm

Please describe what you would like to see added to HandBrake:
According to the documentation Handbrake shows diminishing returns on processors with more than 6 cores. The recently released AMD Ryzen has 8 cores and the i9 has 10 with apparently more on the way ( possibly all the way to 18 ) . Are there any plans to develop Handbrake to fully utilize these core counts ? If so any guess on time scales ?

Why would you like to see this added:
Everyone want to speed up their encoding and fully use the capabilities of their hardware.

What version of HandBrake are you currently running:
1.0.7

What operating system and version and you currently running (e.g. OSX 10.11, Windows 7, Ubuntu 14):
Windows 10

mduell
Veteran User
Posts: 6106
Joined: Sat Apr 21, 2007 8:54 pm

Re: Efficient use of high core counts in next gen chips

Post by mduell » Fri Jul 28, 2017 3:56 pm

Entirely dependent on the underlying libraries, HB doesn't do any work here.

Also the "6" figure is a bit misleading, given the incredible range of sources and settings. It can be anywhere from 2 to 20 depending on your source and settings.

User avatar
BradleyS
Moderator
Posts: 1356
Joined: Thu Aug 09, 2007 12:16 pm

Re: Efficient use of high core counts in next gen chips

Post by BradleyS » Fri Jul 28, 2017 5:48 pm

x264 does have diminishing returns beyond about 8 cores at reasonable settings. You can use a slower x264 preset to better saturate more cores. Generally speaking, this is a mature codec and a limitation of its design. That said, there are new AVX-512 optimizations just hitting x264 master now, so performance will definitely improve overall on newer hardware soon enough.

x265 is based on x264 and seems to scale better with high core counts.

mduell
Veteran User
Posts: 6106
Joined: Sat Apr 21, 2007 8:54 pm

Re: Efficient use of high core counts in next gen chips

Post by mduell » Fri Jul 28, 2017 7:39 pm

BradleyS wrote:
Fri Jul 28, 2017 5:48 pm
x264 does have diminishing returns beyond about 8 cores at reasonable settings.
But the video encoder is only part of the story; there's a decoder thread(s), various filters that can be multithreaded, etc

User avatar
BradleyS
Moderator
Posts: 1356
Joined: Thu Aug 09, 2007 12:16 pm

Re: Efficient use of high core counts in next gen chips

Post by BradleyS » Fri Jul 28, 2017 7:44 pm

Absolutely. NLMeans for one can always use more cores and faster memory I/O. Should be considerably faster on upcoming hardware based on currently published specs.

nhyone
Bright Spark User
Posts: 196
Joined: Fri Jul 24, 2015 4:13 am

Re: Efficient use of high core counts in next gen chips

Post by nhyone » Sat Jul 29, 2017 2:08 am

For slower presets (slower, veryslow, placebo), x264 is limited by the encoder. You can use 10+ cores, no problem. (But they won't be at 100% for other reasons.)

For faster presets, x264 is bottlenecked by its lookahead. It can't analyze the video fast enough to feed the encoder.

There are two more considerations when using many cores.

First, the output video will not be as compressed. The output is maybe 1-5% bigger. Quality is the same.

Second, if you are using dual CPU or Ryzen CPU with its CCX architecture (4 core per CCX), efficiency drops when you use all the cores. It is better to use each CPU / CCX to encode one video at the same time.

db1964
Posts: 12
Joined: Thu Jul 30, 2015 11:05 am

Re: Efficient use of high core counts in next gen chips

Post by db1964 » Sat Jul 29, 2017 2:22 pm

Are you saying that Handbrake will be looking at releasing a version that can encode multiple sources simultaneously ? That would be another way to fully leverage the available hardware I suppose.

rollin_eng
Veteran User
Posts: 2848
Joined: Wed May 04, 2011 11:06 pm

Re: Efficient use of high core counts in next gen chips

Post by rollin_eng » Sat Jul 29, 2017 2:41 pm

You can run multiple instances of HB already.

User avatar
s55
HandBrake Team
Posts: 9187
Joined: Sun Dec 24, 2006 1:05 pm

Re: Efficient use of high core counts in next gen chips

Post by s55 » Sat Jul 29, 2017 2:42 pm

There is nothing stopping you running multiple instances of HandBrake today. That may eventually one day become 1 Instance, multi-encodes but there is a lot of other work to do first.

DrXenos
Bright Spark User
Posts: 195
Joined: Sat Mar 16, 2013 1:19 pm

Re: Efficient use of high core counts in next gen chips

Post by DrXenos » Mon Aug 07, 2017 4:54 pm

Trying to reduce execution time via adding more cores will alway butt up against Amdahl's Law. There will be a point of diminishing returns for any process that isn't fully parallelized.

Post Reply