Windows: START AFFINITY not working with Handbrake CLI v.1.2.2

HandBrake for Windows support
Forum rules
An Activity Log is required for support requests. Please read How-to get an activity log? for details on how and why this should be provided.
Post Reply
kaltert
Posts: 8
Joined: Wed Jul 03, 2019 9:52 am

Windows: START AFFINITY not working with Handbrake CLI v.1.2.2

Post by kaltert »

Description of problem or question:
I have been running handbrake using the command

Code: Select all

start /low /wait /b /affinity F0 HandBrakeCLI.exe
and it only run on 4 threads (out of eight).

- I was using version 1.0.7 (HandBrakeCLI-1.0.7-win-x86_64).
- I just tried to use version 1.2.2 (HandBrakeCLI-1.2.2-win-x86_64) and affinity is not working (all cores are utilised). I even tried changing affinity for 1 or two threads, it did not work.
- Reverting back to 1.0.7 fixes the problem

I should note that I call handbrake from a perl script.

Also I running for a few second in order to get the activity log, if, for some reason, more is needed I can leave it running for more....


HandBrake version (e.g., 1.0.0):
1.2.2



Operating system and version (e.g., Ubuntu 16.04 LTS, macOS 10.13 High Sierra, Windows 10 Creators Update):
Windows v1903 (build 18362.175)



HandBrake Activity Log ***required*** (see How-to get an activity log)

Code: Select all

[13:04:27] hb_init: starting libhb thread
[13:04:27] thread 4269410 started ("libhb")
HandBrake 1.2.2 (2019022300) - MinGW x86_64 - https://handbrake.fr
8 CPUs detected
Opening ENCODE/afile_test.mp4...
[13:04:27] CPU: Intel(R) Core(TM) i7-4770K CPU @ 3.50GHz
[13:04:27]  - Intel microarchitecture Haswell
[13:04:27]  - logical processor count: 8
[13:04:27] Intel Quick Sync Video support: no
[13:04:27] hb_scan: path=ENCODE/afile_test.mp4, title_index=1
udfread ERROR: ECMA 167 Volume Recognition failed
src/libbluray/disc/disc.c:323: failed opening UDF image ENCODE/afile_test.mp4
src/libbluray/disc/disc.c:424: error opening file BDMV\index.bdmv
src/libbluray/disc/disc.c:424: error opening file BDMV\BACKUP\index.bdmv
[13:04:27] bd: not a bd - trying as a stream/file instead
libdvdnav: Using dvdnav version 6.0.0
libdvdread: Encrypted DVD support unavailable.
libdvdread:DVDOpenFileUDF:UDFFindFile /VIDEO_TS/VIDEO_TS.IFO failed
libdvdread:DVDOpenFileUDF:UDFFindFile /VIDEO_TS/VIDEO_TS.BUP failed
libdvdread: Can't open file VIDEO_TS.IFO.
libdvdnav: vm: failed to read VIDEO_TS.IFO
[13:04:27] dvd: not a dvd - trying as a stream/file instead
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'ENCODE/afile_test.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 512
    compatible_brands: isomiso2mp41
    creation_time   : 2019-02-03T01:52:07.000000Z
    encoder         : HandBrake 1.1.1 2018061800
  Duration: 00:39:08.46, start: 0.000000, bitrate: 2788 kb/s
    Stream #0:0(und): Video: hevc (Main) (hvc1 / 0x31637668), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 2621 kb/s, 29.97 fps, 29.97 tbr, 90k tbn, 29.97 tbc (default)
    Metadata:
      creation_time   : 2019-02-03T01:52:07.000000Z
      handler_name    : VideoHandler
    Stream #0:1(spa): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 160 kb/s (default)
    Metadata:
      creation_time   : 2019-02-03T01:52:07.000000Z
      handler_name    : Stereo
[13:04:27] scan: decoding previews for title 1
[13:04:27] scan: audio 0x1: aac, rate=48000Hz, bitrate=160212 espa?ol (AAC LC) (2.0 ch)

Scanning title 1 of 1, preview 1, 10.00 %
Scanning title 1 of 1, preview 4, 40.00 %
Scanning title 1 of 1, preview 6, 60.00 %
Scanning title 1 of 1, preview 9, 90.00 %[13:04:28] scan: 10 previews, 1920x1080, 29.970 fps, autocrop = 0/0/0/0, aspect 16:9, PAR 1:1
[13:04:28] scan: supported video decoders: avcodec qsv
[13:04:28] libhb: scan thread found 1 valid title(s)
+ Using preset: CLI Default
+ title 1:
  + stream: ENCODE/afile_test.mp4
  + duration: 00:39:08
  + size: 1920x1080, pixel aspect: 1/1, display aspect: 1.78, 29.970 fps
  + autocrop: 0/0/0/0
  + chapters:
    + 1: cells 0->0, 0 blocks, duration 00:39:08
  + audio tracks:
    + 1, español (AAC LC) (2.0 ch) (iso639-2: spa)
  + subtitle tracks:
[13:04:28] 1 job(s) to process
[13:04:28] json job:
{
    "Audio": {
        "AudioList": [
            {
                "Bitrate": 160,
                "CompressionLevel": -1.0,
                "DRC": 0.0,
                "DitherMethod": "auto",
                "Encoder": "av_aac",
                "Gain": 0.0,
                "Mixdown": "stereo",
                "NormalizeMixLevel": false,
                "PresetEncoder": "av_aac",
                "Quality": -3.0,
                "Samplerate": 0,
                "Track": 0
            }
        ],
        "CopyMask": [
            "copy:aac",
            "copy:ac3",
            "copy:eac3",
            "copy:dtshd",
            "copy:dts",
            "copy:mp3",
            "copy:truehd",
            "copy:flac"
        ],
        "FallbackEncoder": "av_aac"
    },
    "Destination": {
        "AlignAVStart": false,
        "ChapterList": [
            {
                "Name": ""
            }
        ],
        "ChapterMarkers": false,
        "File": "ENCODE/afile_test_x265.mp4",
        "InlineParameterSets": false,
        "Mp4Options": {
            "IpodAtom": false,
            "Mp4Optimize": false
        },
        "Mux": "m4v"
    },
    "Filters": {
        "FilterList": [
            {
                "ID": 6,
                "Settings": {
                    "mode": 2,
                    "rate": "27000000/900000"
                }
            },
            {
                "ID": 11,
                "Settings": {
                    "crop-bottom": 0,
                    "crop-left": 0,
                    "crop-right": 0,
                    "crop-top": 0,
                    "height": 540,
                    "width": 960
                }
            }
        ]
    },
    "Metadata": {},
    "PAR": {
        "Den": 1,
        "Num": 1
    },
    "SequenceID": 0,
    "Source": {
        "Angle": 0,
        "Path": "ENCODE/afile_test.mp4",
        "Range": {
            "End": 1,
            "Start": 1,
            "Type": "chapter"
        },
        "Title": 1
    },
    "Subtitle": {
        "Search": {
            "Burn": true,
            "Default": false,
            "Enable": false,
            "Forced": false
        },
        "SubtitleList": []
    },
    "Video": {
        "ColorMatrix": 1,
        "ColorPrimaries": 1,
        "ColorTransfer": 1,
        "Encoder": "x265",
        "Profile": "auto",
        "QSV": {
            "AsyncDepth": 4,
            "Decode": false
        },
        "Quality": 20.0,
        "Turbo": false,
        "TwoPass": false
    }
}
[13:04:28] starting job
[13:04:28] job configuration:
[13:04:28]  * source
[13:04:28]    + ENCODE/afile_test.mp4
[13:04:28]    + title 1, chapter(s) 1 to 1
[13:04:28]    + container: mov,mp4,m4a,3gp,3g2,mj2
[13:04:28]    + data rate: 2788 kbps
[13:04:28]  * destination
[13:04:28]    + ENCODE/afile_test_x265.mp4
[13:04:28]    + container: MPEG-4 (libavformat)
[13:04:28]  * video track
[13:04:28]    + decoder: hevc
[13:04:28]      + bitrate 2621 kbps
[13:04:28]    + filters
[13:04:28]      + Framerate Shaper (mode=2:rate=27000000/900000)
[13:04:28]        + frame rate: 29.970 fps -> peak rate limited to 30.000 fps
[13:04:28]      + Crop and Scale (width=960:height=540:crop-top=0:crop-bottom=0:crop-left=0:crop-right=0)
[13:04:28]        + source: 1920 * 1080, crop (0/0/0/0): 1920 * 1080, scale: 960 * 540
[13:04:28]    + Output geometry
[13:04:28]      + storage dimensions: 960 x 540
[13:04:28]      + pixel aspect ratio: 1 : 1
[13:04:28]      + display dimensions: 960 x 540
[13:04:28]    + encoder: H.265 (libx265)
[13:04:28]      + profile: auto
[13:04:28]      + quality: 20.00 (RF)
[13:04:28]      + color profile: 1-1-1
[13:04:28]  * audio track 1
[13:04:28]    + decoder: espa?ol (AAC LC) (2.0 ch) (track 1, id 0x1)
[13:04:28]      + bitrate: 160 kbps, samplerate: 48000 Hz
[13:04:28]    + mixdown: Stereo
[13:04:28]    + dither: triangular
[13:04:28]    + encoder: AAC (libavcodec)
[13:04:28]      + bitrate: 160 kbps, samplerate: 48000 Hz
[13:04:28] sync: expecting 70383 video frames
x265 [info]: HEVC encoder version 2.9
x265 [info]: build info [Windows][GCC 8.2.0][64 bit] 8bit+10bit+12bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
x265 [info]: Main profile, Level-3 (Main tier)
x265 [info]: Thread pool created using 8 threads
x265 [info]: Slices                              : 1
x265 [info]: frame threads / pool features       : 3 / wpp(9 rows)
x265 [warning]: Source height < 720p; disabling lookahead-slices
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge         : hex / 57 / 2 / 2
x265 [info]: Keyframe min / max / scenecut / bias: 30 / 300 / 40 / 5.00
x265 [info]: Lookahead / bframes / badapt        : 20 / 4 / 2
x265 [info]: b-pyramid / weightp / weightb       : 1 / 1 / 0
x265 [info]: References / ref-limit  cu / depth  : 3 / on / on
x265 [info]: AQ: mode / str / qg-size / cu-tree  : 1 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress            : CRF-20.0 / 0.60
x265 [info]: tools: rd=3 psy-rd=2.00 rskip signhide tmvp strong-intra-smoothing
x265 [info]: tools: deblock sao
[13:04:28] sync: first pts audio 0x1 is 0
[13:04:28] sync: first pts video is 1920
[13:04:28] sync: Chapter 1 at frame 1 time 1920
^C
User avatar
s55
HandBrake Team
Posts: 10350
Joined: Sun Dec 24, 2006 1:05 pm

Re: Windows: START AFFINITY not working with Handbrake CLI v.1.2.2

Post by s55 »

Affinity is a feature of Windows, not HandBrake.
HandBrake has no code or settings to make changes to affinity.

Using the command you listed above, with arguments to actually perform an encode, on 1903, works fine for me. I can see affinity is correctly set in task manager and CPU usage is about right for 4 "CPU"s being selected.

My guess is there is a local side problem on your system or with your script.
kaltert
Posts: 8
Joined: Wed Jul 03, 2019 9:52 am

Re: Windows: START AFFINITY not working with Handbrake CLI v.1.2.2

Post by kaltert »

s55 wrote: Wed Jul 03, 2019 5:53 pm Affinity is a feature of Windows, not HandBrake.
HandBrake has no code or settings to make changes to affinity.

Using the command you listed above, with arguments to actually perform an encode, on 1903, works fine for me. I can see affinity is correctly set in task manager and CPU usage is about right for 4 "CPU"s being selected.

My guess is there is a local side problem on your system or with your script.
You are not wrong about AFFINITY but I was hoping that there is something in the latest HandBrake that messes with affinity.

The only thing I change in the script is the HandBrake exe. No other parameter. And the script works fine with 1.0.7 but not with 1.2.2
Furthermore, I run it without the script, straigth from the command line:

Code: Select all

start /low /wait /b /affinity F0 .\HandBrakeCLI.exe
and I get the same problem just by changing the HandBrakeCLI.exe version
User avatar
s55
HandBrake Team
Posts: 10350
Joined: Sun Dec 24, 2006 1:05 pm

Re: Windows: START AFFINITY not working with Handbrake CLI v.1.2.2

Post by s55 »

I can't explain that unfortunately. All I can tell you is it appears to be a problem unique to you. Having tested this it appears to work correctly for me so that probably rules out any Windows breakage.

I guess there is also the possibility that some app on your system is doing it however the only apps I know that do this, require you to setup rules to do it so you'd know about it.
kaltert
Posts: 8
Joined: Wed Jul 03, 2019 9:52 am

Re: Windows: START AFFINITY not working with Handbrake CLI v.1.2.2

Post by kaltert »

Thanks, for your responses.

I just wanted to make sure it has nothing to do with HandBrake. On to Stack Overflow :)
kaltert
Posts: 8
Joined: Wed Jul 03, 2019 9:52 am

Re: Windows: START AFFINITY not working with Handbrake CLI v.1.2.2

Post by kaltert »

UPDATE: maybe this changes something and helps you help me pinpoint the problem

The full command I was running was

Code: Select all

start /low /wait /b /affinity F0 .\HandBrakeCLI.exe -i "INPUT_FILE" -o "OUTPUT_FILE" --width 960 --height 540 --auto-anamorphic --format av_mp4 --encoder x265 --encoder-profile auto --quality 20 --rate 30 --pfr --aencoder av_aac -B 160 --mixdown stereo
I tried playing with the parameters and found out that removing the "--encoder x265" parameter or replacing it, for example with with "--encoder x264" or "--encoder mpeg4" restricts handbrake to the intended cores.

Could it be the case that x265 particularly spawns some extra processes, somehow?
kaltert
Posts: 8
Joined: Wed Jul 03, 2019 9:52 am

Re: Windows: START AFFINITY not working with Handbrake CLI v.1.2.2

Post by kaltert »

BTW, I tried all versions after 1.0.7 and they seem to produce the same problem for me
User avatar
s55
HandBrake Team
Posts: 10350
Joined: Sun Dec 24, 2006 1:05 pm

Re: Windows: START AFFINITY not working with Handbrake CLI v.1.2.2

Post by s55 »

I'm seeing no difference locally between x264/5. Affinity is correctly applied on both.

Task manager shows the exact cores that I select as being utilised.
Deleted User 13735

Re: Windows: START AFFINITY not working with Handbrake CLI v.1.2.2

Post by Deleted User 13735 »

What is your goal?
Using more than optimal cores is not a goal, because it may slow things down.
User avatar
s55
HandBrake Team
Posts: 10350
Joined: Sun Dec 24, 2006 1:05 pm

Re: Windows: START AFFINITY not working with Handbrake CLI v.1.2.2

Post by s55 »

He's trying to use less cores, not more, but it's unclear why.
Deleted User 13735

Re: Windows: START AFFINITY not working with Handbrake CLI v.1.2.2

Post by Deleted User 13735 »

musicvid wrote: Thu Jul 04, 2019 6:05 pm What is your goal?
Using more than optimal cores is not a goal, because it may slow things down.
So will using less than optimal cores.
kaltert
Posts: 8
Joined: Wed Jul 03, 2019 9:52 am

Re: Windows: START AFFINITY not working with Handbrake CLI v.1.2.2

Post by kaltert »

s55 wrote: Thu Jul 04, 2019 6:16 pm He's trying to use less cores, not more, but it's unclear why.
The reason is that I am trying to work otherwise on the PC, so I need few cores unencumbered (although with the upcoming 2900X, things will be better for me)
Woodstock
Veteran User
Posts: 4614
Joined: Tue Aug 27, 2013 6:39 am

Re: Windows: START AFFINITY not working with Handbrake CLI v.1.2.2

Post by Woodstock »

Handbrake normally tells the operating system to assign it a low priority, which helps with latency for foreground tasks. If you leave it running in the foreground (selected window), the operating system thinks you want it to have a higher priority.
User avatar
s55
HandBrake Team
Posts: 10350
Joined: Sun Dec 24, 2006 1:05 pm

Re: Windows: START AFFINITY not working with Handbrake CLI v.1.2.2

Post by s55 »

The CLI doesn't, so he'd still have to start /low in that scenario. It's only the Windows UI that sets that by default (unless the user changes the preference)

That said, "start /low" should be sufficient on a system like yours to not have

Typically setting low or below normal is more than enough to have the system perform well whilst handbrake is running in the background. Windows does a good job here. I've left it running in the background whilst playing games without any significant lag noticed. (Not really something I recommend but it does seem to work on a decent enough system)

Even with affinity, HandBrake can potentially still max out shared caches etc, so often it wouldn't have the desired effect anyway.
Deleted User 13735

Re: Windows: START AFFINITY not working with Handbrake CLI v.1.2.2

Post by Deleted User 13735 »

The sage advice is, "Don't multitask while encoding video," not just with Handbrake.
Addicted gamers and movie buffs don't seem to understand the wisdom of playing on another machine while rendering
kaltert
Posts: 8
Joined: Wed Jul 03, 2019 9:52 am

Re: Windows: START AFFINITY not working with Handbrake CLI v.1.2.2

Post by kaltert »

Well, people who actually work on their computer may be sage enough to be patient and willing to leave the encoding to fewer threads. Not all people have many machines...
User avatar
s55
HandBrake Team
Posts: 10350
Joined: Sun Dec 24, 2006 1:05 pm

Re: Windows: START AFFINITY not working with Handbrake CLI v.1.2.2

Post by s55 »

Setting HandBrake to Low Priority means pretty much any other process on the box that needs CPU time, will take that CPU time from HandBrake. It doesn't make an awful lot of sense to also set affinity. For most sensible use cases, it'll work fine.
mduell
Veteran User
Posts: 8187
Joined: Sat Apr 21, 2007 8:54 pm

Re: Windows: START AFFINITY not working with Handbrake CLI v.1.2.2

Post by mduell »

musicvid wrote: Sat Jul 06, 2019 9:39 pm The sage advice is, "Don't multitask while encoding video," not just with Handbrake.
Addicted gamers and movie buffs don't seem to understand the wisdom of playing on another machine while rendering
lol, 1999 called, they want their advice back.

Modern computers and operating systems multitask quite well.
kaltert
Posts: 8
Joined: Wed Jul 03, 2019 9:52 am

Re: Windows: START AFFINITY not working with Handbrake CLI v.1.2.2

Post by kaltert »

s55 wrote: Sat Jul 06, 2019 10:10 pm Setting HandBrake to Low Priority means pretty much any other process on the box that needs CPU time, will take that CPU time from HandBrake. It doesn't make an awful lot of sense to also set affinity. For most sensible use cases, it'll work fine.
I agree with you and it should work well in theory, but unfortunately the method seems to be imperfect. I actually monitored the 8 threads and Handbrake still takes a significant percentage, even when higher priority tasks are running. While Low Priority ameliorates things, it is not as good as restricting to few threads.

All of this is contingent on the assumption that everything is OK with my machine and there is not something else obstructing the /low, /affinity arguments
User avatar
s55
HandBrake Team
Posts: 10350
Joined: Sun Dec 24, 2006 1:05 pm

Re: Windows: START AFFINITY not working with Handbrake CLI v.1.2.2

Post by s55 »

Results.png
Results.png (10.85 KiB) Viewed 638 times
You should be seeing something along the lines of that. It generally drops about 50~70 Cinebench points which equates to about a 96/4 share of CPU time.

If your running light applications that don't really hold much CPU time. it's virtually impossible to tell with the rate task manager actually updates.

Your system specs look reasonable so it should be possible to replicate this.
While Low Priority ameliorates things, it is not as good as restricting to few threads
HandBrake internally won't change the thread count. It still sees an 8 thread CPU, so will create, several dozen threads for all the component parts. Although I assume your just talking about affinity here. I thought I'd clarify anyway.


Honestly have no idea why your system is behaving like this :/
nhyone
Bright Spark User
Posts: 252
Joined: Fri Jul 24, 2015 4:13 am

Re: Windows: START AFFINITY not working with Handbrake CLI v.1.2.2

Post by nhyone »

s55 wrote: Sun Jul 07, 2019 9:46 am Honestly have no idea why your system is behaving like this :/
I recall I had the same experience on Linux with an old version of HandBrake with the x265 encoder. I thought I posted it here, but I couldn't find it, so maybe not.

My solution was to apply affinity on the shell that runs HandBrake. Then the x265 encoder would respect it.

There are a couple of reasons for me to use affinity. One is that it is a shortcut way to set the number of threads. x264/x265 auto-detects the #cores. As we know, too many threads = less space efficient.
Last edited by nhyone on Mon Jul 08, 2019 7:54 am, edited 1 time in total.
Deleted User 13735

Re: Windows: START AFFINITY not working with Handbrake CLI v.1.2.2

Post by Deleted User 13735 »

mduell wrote: Sat Jul 06, 2019 10:35 pm Modern computers and operating systems multitask quite well.
Yes, they do. They just render slower, now as always.
kaltert wrote: Sat Jul 06, 2019 10:08 pm Well, people who actually work on their computer may be sage enough to be patient and willing to leave the encoding to fewer threads. Not all people have many machines...
People whose actual work is encoding video will have different perspectives on that, one can be sure.
Post Reply