Make HB single/dual threaded

[Havor]

To make HB more efficient on multi-core CPUs, would not making HB single, dual, triple or max quad threaded during multiple cued items not be a good idea?

Why?
From what i understand, the more threads you use, the less efficient HB becomes, and beyond 6 threads it becomes really noticeable, and there is a trend of more and cores in a system (i7 5960X).

Would it not be more efficient to encode multiple items at the same time, so that you can utilize all the cores at there total maximum.

I don't really know ware the tipping point is, at how many threads your not always using 100% of your cores, but i assume you already lose some efficiency whit more then one threat (or two if HT is used).

Downside
If it is not possible to add more cores during the encoding process that became available as they ware finishing there last cued items, all the efficiency gains would be lost on the last encodes.

Still there are people (like me) that are literately encoding 24/7, and they would see benefits.

I my self have a dual 6 core E5-2640 system(*), ware i have 4 HB processes running that i manage with Process Lasso, ware i have assigned each HB 3 cores (6 inc. HT), and my CPU utilization went up from around 60~70% to little over 90%.
3 processes did not work as well, as one process was using cores on two different CPU and was a little less efficient as the other two.

(*)Saved the workstation from a container as it's housing was badly damaged by fire at a engineering firm, but the mobo and Quadro video card ware still working, and the mobo i now use as my server, ware i soled the Quadro card for 600 euro, the SSDs and PSU ware dead tho.

[mduell]

I don't think you understand the terminology you're using... on my quad-core non-HT machine HB is already using 24 threads (as counted by Windows Task Manager) by default on a single encode with only a resize filter.

If you provided an encoding log we could see what you're choosing, and if it includes any decoder, filters, encoder, or settings that don't scale well.

[Havor]

Maybe my terminology is off, but i think you still get my the point i am trying to bring across don't you?

Reducing the amount of cores used on a single cued item, and start one or more cued items at the same time, to have most efficient use of CPU power.

[mduell]

I gather you want multiple encodes running at the same time; it's a very uncommon need.

If you posted an encoding log we could see why you're not seeing good scaling. There may be changes you can make to see better scaling.

[Havor]

I gather you want multiple encodes running at the same time; it's a very uncommon need.

I am not speaking about my system alone, but whit the new 8 core i7s, i am not anymore in the small group that uses more then 6 cores to there encoding.

And i would be just fine with 2 HB processes, one on each CPU, as i stay then still with in the useable range before more cores are just gone deliver diminished returns, just trying to squeeze the last droops out of my CPU.

If you posted an encoding log we could see why you're not seeing good scaling. There may be changes you can make to see better scaling.

I Know why its not scaling well, the developers them self say that above 6 cores you will see diminished returns.

But we are getting way of topic, i was asking if it ware not a good idea, to simultaneous encode more items at the same time to optimize maximum CPU usages.

[Djfe]

the encode itself runs on several threads/cores already
it's just the filters that are not threadable, yet (the devs are working on it)

that's why mduell asked for your logs

if you would share them, he could give you more details about what is slowing down your encode

->you don't need several instances of handbrake to fully utilize your cpu

[JohnAStebbins]

HandBrake fully occupies all cores of my 6 core sandybridge-e when using farily typical encoding settings. There are only a few filters that can cause a bottleneck and they are generally used for special situations and would not be needed for most encode jobs.

decomb, detinterlace, and rotate are all multi-threaded.

detelecine single threaded, but is fast enough that it can keep the pipeline full as long as you are not using very fast encoder settings. When using x264's "fast" preset, all cores remain busy. But with "faster" x264 preset, the pipeline downstream may not remain full.

scaling filter is single threaded but so fast that it keeps the pipeline full no mater what other settings you use.

denoise and deblock are single threaded (at the moment). These can have a large imact, but are not needed for most jobs. There is a patch to multi-thread the nlmeans denoise filter which should get committed soon (https://reviews.handbrake.fr/r/758/). But even threaded, nlmeans is a beast and very slow and should only be used when needed.

Another factor that affects cpu utilization is which encoder you use and what encoder settings you apply.

The x264 preset "ultrafast" will use all cores, but only about 80% of each. Slower x264 presets more fully utilize each core.

The mpeg2 and mpeg4 encoders use all cores, but only about 70% of each.

VP8 uses all cores, but in a very weird pattern where some cores will be utilized at only 30% while others are used at 90%.

Theora is single threaded and will only use 1 core 100%.

You can also be bottlenecked by audio. Most of the audio encoders are fast enough that you would not see any impact. But the fdk-aac encoder bottlenecks my encodes at about 200fps. So if you are using video encoder settings that are faster than this, you would see under utilization of cpu cores.

[Deleted User 13735]

But the fdk-aac encoder bottlenecks my encodes at about 200fps.

If that ever becomes a problem for me, I'll certainly let you know . . .