Cores, cores, and more cores

[deathtical]

Now that AMD and Intel are offering such large core/thread count CPUs, I was wondering if there were plans to either "improve" the current HandBrake to utilize more cores/threads (I believe the officially supported count is 6 cores) in order to get better encoding performance, or perhaps a a separate build that supports more cores/threads? I currently have an AMD Ryzen 1700X with 8 cores / 16 threads and it would be nice if HandBrake could take advantage of all the processing power, especially on my 4K encodes.


Also, could someone just clarify something for me. In the documentation, it states that HandBrake performs best with up to 6 cores with diminishing returns there after. When you say 6 cores, are you referring to actual physical cores or are you referring to processing threads?

Thanks

[mduell]

(I believe the officially supported count is 6 cores)

That is not in any way what any official documentation says.

or perhaps a a separate build that supports more cores/threads?

Dumbest idea yet.

I currently have an AMD Ryzen 1700X with 8 cores / 16 threads and it would be nice if HandBrake could take advantage of all the processing power, especially on my 4K encodes.

HB as-is easily supports that many cores for a 4K encode assuming you're not bottlenecking it with a single threaded filter/decoder or something dumb.

Also, could someone just clarify something for me. In the documentation, it states that HandBrake performs best with up to 6 cores with diminishing returns there after. When you say 6 cores, are you referring to actual physical cores or are you referring to processing threads?

Actual cores doing work. Diminishing isn't nearly as strong as you think it is, for reasonable encode settings and core counts.

Even a 1080p x264 encode with no filtering and reasonable settings can easily saturate 8+ cores. Very high core counts (24+) depend on the OS scheduler to be non-retarded for good performance, see recent Linux vs Windows benchmarking on that topic.

[deathtical]

That is not in any way what any official documentation says.

It most certainly does...
https://handbrake.fr/docs/en/latest/tec ... mance.html

"Hardware
The hardware you run on can have a large effect on performance. HandBrake can scale well up to 6 CPU cores with diminishing returns thereafter.
So a 4 Core CPU can be nearly twice as fast as a Dual Core equivalent."

Dumbest idea yet.

No need to be rude. Lots of software packages have branches that support either specific processors or other types of hardware.

HB as-is easily supports that many cores for a 4K encode assuming you're not bottlenecking it with a single threaded filter/decoder or something dumb.

I typically use the presets for the codec I wish to use, so if it is using a single thread filter by default, then I am too. I notice that when I run HB, Windows says all cores and threads are being maxed out, but Windows isn't always the most accurate measurement.

Actual cores doing work. Diminishing isn't nearly as strong as you think it is, for reasonable encode settings and core counts.

This doesn't really answer the question as a single core on some processors can process two threads at once where some cannot. So are you saying that, if the CPU supports it, HB will fully utilize a 6 core CPU with hyperthreading enabled (6cores/12threads)?

Even a 1080p x264 encode with no filtering and reasonable settings can easily saturate 8+ cores. Very high core counts (24+) depend on the OS scheduler to be non-retarded for good performance, see recent Linux vs Windows benchmarking on that topic.

I will look into this, but I have been reading that recent HB builds for Linux have been having issues.

Lastly, there have been several hardware reviews where they use HB as a benchmark that show that more cores/threads do not equal better encoding performance, at least not what one would expect if HB were truly taking advantage on a core by core basis. I'll see if I can dig one up.

[s55]

Scaling is near linear to 6 in most reasonable circumstances after which, it can drop off but varies widely depending on source. It doesn't say we only support 6. People have maxed out 16 core systems with HandBrake without issue.

AT some point we'll update the paragraph to be a bit clearer since too many people are reading things that's not there.

If Windows is saying all the Cores/Threads are maxed out, then it's fully utilised your system.

Since your 8 core, 16thread is maxed out, I think you can deduce the answer to the next question.

Benchmarks are not always useful to determine real world performance.

At this point, we are not going to be seeing much more in terms of massive breakthroughs in performance on current hardware with this generation of video encoders.

[deathtical]

@s55

Thank you. I guess I'm just stuck with 4K encodes taking 12+ hours (in some cases 16+ hours) per encode *sigh*. I guess I had gotten spoiled when my 1080p encodes were only taking a couple of hours each.

[s55]

Yeh, orders of magnitude more data = longer encoders. So, Either buy faster hardware, or use faster settings. You may still find you can adjust the settings to get more speed without a meaningful impact to quality or filesize.

Try adjusting the Preset slider on the Video tab, and maybe tweak the quality slider +/-1 to see how that goes. You might shave a bit of time off. Also, if your dealing with 4K, you probably don't need decomb/deinterlace turned on, which they probably are if your using defaults, so try turning those off.

Or, post an encode log and we can make better suggestions.

[mduell]

That is not in any way what any official documentation says.

It most certainly does...
https://handbrake.fr/docs/en/latest/tec ... mance.html

"Hardware
The hardware you run on can have a large effect on performance. HandBrake can scale well up to 6 CPU cores with diminishing returns thereafter.
So a 4 Core CPU can be nearly twice as fast as a Dual Core equivalent."

Which is completely different from what you said: (I believe the officially supported count is 6 cores)
Diminishing does not mean zero.

HB as-is easily supports that many cores for a 4K encode assuming you're not bottlenecking it with a single threaded filter/decoder or something dumb.

I typically use the presets for the codec I wish to use, so if it is using a single thread filter by default, then I am too. I notice that when I run HB, Windows says all cores and threads are being maxed out, but Windows isn't always the most accurate measurement.

With no log it's hard to say, but Windows showing the cores/threads maxed out is a good sign you're not bottlenecking on a single threaded filter/decoder.

Actual cores doing work. Diminishing isn't nearly as strong as you think it is, for reasonable encode settings and core counts.

This doesn't really answer the question as a single core on some processors can process two threads at once where some cannot. So are you saying that, if the CPU supports it, HB will fully utilize a 6 core CPU with hyperthreading enabled (6cores/12threads)?

6 cores means 6 cores, regardless if you schedule 6 or 12 or any other (reasonable) number of threads on them. The extra threads don't get much work done since x264 is highly optimized, but you eek a few percent out of it.

No, I am not saying HB will fully utilize a 6 core CPU, I've never said that and the documentation doesn't say that either. You can easily construct an encode that will only use a couple cores. But for typical encodes, HB has no problems scaling well to 10-30 cores with some diminishing returns.

Lastly, there have been several hardware reviews where they use HB as a benchmark that show that more cores/threads do not equal better encoding performance, at least not what one would expect if HB were truly taking advantage on a core by core basis.

Great, you've found an example of diminishing returns.

[deathtical]

@s55

Thanks. I have tried tweaking the presets a little but the time saved is negligible. I have gotten in the habit of starting an encode when I go to bed and by the time I get back from work then next day it's done... usually.

@mudell

You really need to check that superiority complex of yours. You don't need to talk down to someone simply because you think you know more than them. I may not know a lot about media encoding, hence why I'm on the forum, but I am a network and backup systems engineer. So I have a very good understanding of how software utilizes CPUs and how computer hardware, operating systems, software, and drivers interact. Frankly, your explanation doesn't give the impression that you do. So while you may have a better understanding of how media encoding works, don't assume that the person you're talking to is ignorant. If you can't be helpful, don't post.