As we have suggested earlier we were observing low CPU utilizations for large machines with HandBrake. We have managed to bring up a stable version of our suggested parallel implementation of HandBrake. We are finding the initial results of the parallel implementation to be promising. We are currently testing our implementation in AMD Threadripper 2990wx (with 64 threads) and our parallel versions is showing higher CPU and Memory utilization compared to the sequential version.
PFA our average execution time (https://drive.google.com/file/d/14GKewj ... sp=sharing
), CPU utilization (https://drive.google.com/file/d/1A_H4sU ... sp=sharing
) and memory utilization (https://drive.google.com/file/d/1e1YV5v ... sp=sharing
) for serial, dynamic parallel and static parallel modes. In static parallel mode the user can select the number of jobs to be run in parallel. In the charts we are providing the results for 3,4 and 5 jobs in parallel (marked as parallel 3, parallel 4 and parallel 5 respectively in the charts). In dynamic parallel (marked as just parallel in the charts), our online dynamic cost function framework decides on the number of jobs to be run in parallel.
The queues have been randomly selected to cover as much use cases as possible.
We are still actively trying to improve our parallel logic, however if you could try our parallel implementation out and provide your valuable suggestions it would be great.
Please find attached our CLI version of parallel HandBrake here. https://drive.google.com/file/d/1nOr-vF ... sp=sharing
To run the sequential mode : HandBrakeCLI.exe --queue-import-file file.json [The execution time for this mode is similar to traditional HandBrake]
Dynamic parallel mode : HandBrakeCLI.exe --queue-import-file file.json --parallel
Static parallel mode : HandBrakeCLI.exe --queue-import-file file.json --parallel=n (where n represents number of jobs to be run in parallel )