51 hours to copy, somethibg is wrong. Settings are ...

[kevin]

source video-ts chapter 1 to 16 dur 1.43

format mp4 codes avc/h.264 video/ aav audio

fram: same as source
encoe x264
average bit rate 1500
encoding 10f1 20.53% 3.05 fps

this is being going on since yesterday

[fullerflyer]

source video-ts chapter 1 to 16 dur 1.43

format mp4 codes avc/h.264 video/ aav audio

fram: same as source
encoe x264
average bit rate 1500
encoding 10f1 20.53% 3.05 fps

this is being going on since yesterday

...nothing is wrong. It all depends on your computer speed and your choice of settings..

I run a G4 Powerbook, with super high settings (i.e... "Exhaustive" Motion Estimation Range under the "Advanced" Tab. ...takes 200+ hours, sometimes.

...try .35fps...

I'm about done ripping the U2 go HOME Live From Slane Castle, Ireland. I started it sometime last week.

Because it's "Video" (29.97fps) with interlacing and telecining... In order to get a good quality encode to the degree that I am satisified, I need to max out all the settings.

[robjohn]

...nothing is wrong. It all depends on your computer speed and your choice of settings..

I run a G4 Powerbook, with super high settings (i.e... "Exhaustive" Motion Estimation Range under the "Advanced" Tab. ...takes 200+ hours, sometimes.

...try .35fps...

I know that a PowerBook G4 is no longer a powerhouse (I am using a 1.5 GHz 1GB PowerBook G4), but while running HandBrake, I looked at the Activity Monitor, and I noticed about 2/3 of the CPU was devoted to 'nice' time. The last time I saw this, I was exporting from QuickTime, and noted that QTPlayerHelper had its nice set to 3. When I reniced it to 0, it sped up considerably.

Looking at HandBrake, its nice was already 0. I looked deeper and saw that the thread that was doing most of the work was given a scheduling priority of 0. When I set the nice to -20, the rest of the threads perked up, but not the thread doing most of the work:

Code: Select all

PID  PRI  NI      VSZ    RSS WCHAN  STAT  TT       TIME  %CPU     STIME     UTIME
871   49 -20   237916  79792 -      S     ??  534:41.95   1.5   8:48.06  21:26.39
871   51 -20   237916  79792 -      S         534:41.95   0.0   0:08.46   0:11.79                 
871   51 -20   237916  79792 -      S         534:41.95   0.0   0:00.00   0:00.00                 
871   51 -20   237916  79792 -      S         534:41.95   0.0   0:00.03   0:00.01                 
871   44 -20   237916  79792 -      S         534:41.95   4.4   1:42.61  20:03.65                 
871   20 -20   237916  79792 -      R         534:41.95   0.0   0:05.23   1:11.65                 
871   51 -20   237916  79792 -      S         534:41.95   0.0   3:46.95   1:12.40                 
871   51 -20   237916  79792 -      S         534:41.95   0.0   0:29.57   0:19.03                 
871   20 -20   237916  79792 -      R         534:41.95   0.2   3:30.04  11:17.14                 
871   20 -20   237916  79792 -      R         534:41.95   0.2   3:20.55  30:57.05                 
871    0 -20   237916  79792 -      R         534:41.95  70.3   4:52.82 282:14.07                 
871   20 -20   237916  79792 -      R         534:41.95   0.0   0:12.46   1:47.13                 
871   20 -20   237916  79792 -      R         534:41.95   0.1   1:08.45  13:09.30                 

When I paused HandBrake, no CPU was devoted to nice and the working thread was boosted to 20:

Code: Select all

PID  PRI  NI      VSZ    RSS WCHAN  STAT  TT       TIME  %CPU     STIME     UTIME
871   51 -20   233496  73460 -      S     ??  535:27.83   1.5   8:48.83  21:28.34
871   51 -20   233496  73460 -      S         535:27.83   0.0   0:08.46   0:11.80                 
871   51 -20   233496  73460 -      S         535:27.83   0.0   0:00.00   0:00.00                 
871   51 -20   233496  73460 -      S         535:27.83   0.0   0:00.03   0:00.01                 
871   48 -20   233496  73460 -      S         535:27.83   4.3   1:42.78  20:06.30                 
871   20 -20   233496  73460 -      S         535:27.83   0.0   0:05.24   1:11.79                 
871   51 -20   233496  73460 -      S         535:27.83   0.0   3:47.27   1:12.51                 
871   51 -20   233496  73460 -      S         535:27.83   0.0   0:29.63   0:19.07                 
871   20 -20   233496  73460 -      R         535:27.83   0.1   3:30.46  11:18.20                 
871   20 -20   233496  73460 -      R         535:27.83   0.0   3:20.97  31:00.93                 
871   20 -20   233496  73460 -      R         535:27.83   0.0   4:53.44 282:45.17                 
871   20 -20   233496  73460 -      R         535:27.83   0.0   0:12.51   1:47.37                 
871   20 -20   233496  73460 -      R         535:27.83   0.0   1:08.60  13:11.03                 

When a thread does not block frequently enough, the system will reduce its scheduling priority. I don't know exactly how the OS X scheduler determines this, but it might be that on slower machines, HandBrake does not block frequently enough. Perhaps HandBrake blocks after a certain number of cycles, and that number of cycles takes too long on slower machines. If this is the case, perhaps an adaptive determination of when to block would help those of us running slower processors. Things are already slow enough.

Getting rid of the time sliced off to nice could speed things up 3 times or more.

[eddyg]

Thanks for the analysis, this is right along the lines of what I was thinking. I've noticed that on a Mac Pro HB is causing a lot of thrashing in the schedular.

At the moment the HB thread scheduling works by having producers and consumers with fifos between them. Each consumer consumes what it can until until its output fifo is full, at which point it sleeps a set interval.

So there are two factors that determine the scheduling interval, the size of the fifo, and the length of time that we sleep within the thread.

These are at present fixed, in fact I just *increased* the fifo size in the SVN version of HB due to the schedular thrashing too much on faster machines. This will impact slower machines adversely as you have pointed out.

The ideal situation is where all the producers and consumers are running at the optimum rate to feed the encoder thread which is the slowest one. Ideally we don't spend too much time spinning around on the other threads whilst x264 is busy encoding.

Do you have any ideas on what algorithm to use to dynamically set the sleep time and fifo size?

There are a number of fifos,

dvd reader reads and puts the buffers on one of:

mpeg2 fifo
audio fifo
subtitle fifo

These then get decoded and end up arriving in the sync thread which then syncs them and puts them on the sync fifo
Which then ends up going into the render thread and out of its fifo
And then into the encoder (e.g. h.264) and its fifo
Then out of that and into the appropriate muxer for the container, e.g. mp4.

So there are quite a number of fifos that can be tuned. At the moment they are all the same size apart from one of them which is a lot bigger, the mpeg2 one I think, But I don't recall.

Cheers, Ed.