VBV Talk

[sdm]

I wouldn't lower the percentage, but I would add vbv to cap off the bitrate.

Cavalicious, I've been looking into VBV and browsed this:
http://trac.handbrake.fr/wiki/VBVRateControl,

A non-discriminatory capping of bitrate would lead to serious image degradation in highly complex scenes - right?

I get the impression VBV isn't blindly 'capping' the bitrate, rather using some sort of buffer and intelligently distributing bits to cap bitrate spikes? Is that true?

If so, without becoming an expert, how do I know what settings are appropriate?

On an extremely bit-hungry encode of a music video (66% crf, uncapped resulted in 9.88 mbits/s), I tried this:

Code: Select all

:vbv-maxrate=4900:vbv-bufsize=3500

at the end of my advanced x264 options. ( I just copied that from somewhere else in this thread)

The resulting capped encode looked pretty good, but not as good as uncapped on a frame-by-frame comparison, and the activity log was filled with 100-200 lines like this:

Code: Select all

x264 [warning]: VBV underflow (-33308 bits)
x264 [warning]: VBV underflow (-15743 bits)
x264 [warning]: VBV underflow (-60623 bits)
x264 [warning]: VBV underflow (-31599 bits)
x264 [warning]: VBV underflow (-29343 bits)
x264 [warning]: VBV underflow (-6965 bits)
x264 [warning]: VBV underflow (-1020 bits)

I'm wondering if I should be concerned about this warning?

I'm mostly playing videos back on a mac mini, which doesn't seem to have a problem with uncapped video.
I think I'd like to raise the cap. Can I increase the numbers, if so, to what?

Thanks in advance for any advice.
--sdm.

[Cavalicious]

If so, without becoming an expert, how do I know what settings are appropriate?

You do like us...fine a base and go from there. It really boils down to what the playback device can handle. But more importantly, how it looks to you.

I'm wondering if I should be concerned about this warning?

Depends? How does it look? Is it smooth? Does it playback fine on the preferred device.

I'm mostly playing videos back on a mac mini, which doesn't seem to have a problem with uncapped video.
I think I'd like to raise the cap. Can I increase the numbers, if so, to what?

So i suspect since your playback device is a mac mini, you're capping to tame the size of the file. If true, then find numbers that gives you a file size you like. Just be sure that the maxrate is larger than bufsize.

[sdm]

So i suspect since your playback device is a mac mini, you're capping to tame the size of the file. If true, then find numbers that gives you a file size you like. Just be sure that the maxrate is larger than bufsize.

Yup - exactly, I want to cap it to tame the file size. Its a question of principle really. I can't wrap my head around 300 mb for a 5 minute SD video.

I'm curious what the warning was telling me really.

I'm also curious if my assessment of VBV above is correct? I'd love it if someone could sum up the theory of VBV in a paragraph or so?

Just be sure that the maxrate is larger than bufsize.

Any hints how much? How do the numbers relate to each other?

I'm also curious - theoretically, if a video is mostly CRF encoded near the cap say 4500kbps, would it then make sense to say the CRF of say 66% is not really being met, and therefore, in theory, a 2-pass ABR encode at 4500kbps would provide better results?

I realise there are no absolute answers here, and it depends on the source, and output device etc, etc., but I'm still interested in the theory any other users'/developers' experience.

Thanks,
--sdm.

[rhester]

A non-discriminatory capping of bitrate would lead to serious image degradation in highly complex scenes - right?

Capping of bitrate is going to lead to degradation of complex scenes, yes. The alternative is stuttering. You have to choose one devil or the other.

I get the impression VBV isn't blindly 'capping' the bitrate, rather using some sort of buffer and intelligently distributing bits to cap bitrate spikes? Is that true?

It would be nice if it were. Unfortunately, it is very blind and reactive. The built-in x264 VBV tends to damage I-frames pretty badly - our patched version instead tends to damage P-frames.

One of the lead x264 developers is researching VBV lookahead to behave more like you describe, but it seems to be a long way off.

The resulting capped encode looked pretty good, but not as good as uncapped on a frame-by-frame comparison, and the activity log was filled with 100-200 lines like this:

Code: Select all

x264 [warning]: VBV underflow (-33308 bits)
x264 [warning]: VBV underflow (-15743 bits)
x264 [warning]: VBV underflow (-60623 bits)
x264 [warning]: VBV underflow (-31599 bits)
x264 [warning]: VBV underflow (-29343 bits)
x264 [warning]: VBV underflow (-6965 bits)
x264 [warning]: VBV underflow (-1020 bits)

I'm wondering if I should be concerned about this warning?

Of course it won't look as good as uncapped - that's the entire point.

Those warnings mean you're not filling the buffer and x264 is having to fill it with nulls (effectively) to make up the difference. Nothing you can do about this, it's a limitation of the VBV implementation.

Rodney

[sdm]

Thanks Rodney.
There's just one other question above:

I'm also curious - theoretically, if a video is mostly CRF encoded near the cap say 4500kbps, would it then make sense to say the CRF of say 66% is not really being met, and therefore, in theory, a 2-pass ABR encode at 4500kbps would provide better results?

What do you think?

--sdm.

[rhester]

I'm also curious - theoretically, if a video is mostly CRF encoded near the cap say 4500kbps, would it then make sense to say the CRF of say 66% is not really being met, and therefore, in theory, a 2-pass ABR encode at 4500kbps would provide better results?

It is true the CRF in that example is not really being met - if you do a VBV ceiling and you hit it, then you aren't meeting the target. Again, that's the whole point. But yes, if you are CRF-encoding and spend nearly all your time being capped, you're effectively doing CBR at that point.

A 2-pass ABR at the cap rate will not help for the same reason - ABR can and will spike up to around 3x the average rate, so you'll still have to cap, and you'll still be in exactly the same boat. Either way, if you need 4500kbps bitrate and need to cap at same, you're better off doing CBR (though 4500kbps is staggeringly high for most content).

Rodney

[dynaflash]

I'd love it if someone could sum up the theory of VBV in a paragraph or so?

So would I . It would be in our wiki. Thought you seem to be using it to limit file size, it is actually designed to limit local bitrates to stay within a given video buffer. Used as much for network's as well as playback devices. If you know the exact specs of the network, playback device's buffer, you can absolutely calculate for it accurately to set the desired vbv-maxrate and vbv-bufsize. In the case of the ATV 2.0 (remember, thats what this thread is about, not mac playback) we are given nothing told nothing but that the atv can withstand short term bitrates up to 12,000 kbps. Problem is we know nothing of the buffer duration, etc which is just as important to accurate setting vbv (hence the "vbv-bufsize" portion of the settings). So we are left with empiricle testing to come up with something that works in the real world.

Oh, and btw, those "vbv buffer underflow" warnings mean that the encoder cannot control the bitrate within your desired spec.

Though its an older thread, the basic concepts are covered fairly well here: http://forum.doom9.org/showthread.php?t=98386 remember, some of the limitations of x264 have since been fixed, but as I said the basic theories and ideas are covered.

[sdm]

Thought you seem to be using it to limit file size, it is actually designed to limit local bitrates to stay within a given video buffer

You're right - Lately, I've been thinking vbv isn't really the right tool for job, especially in the case where most of the encode is at the vbv ceiling, but ...

I wouldn't lower the percentage, but I would add vbv to cap off the bitrate.

made me want to investigate vbv.

In the case of the ATV 2.0 (remember, thats what this thread is about, not mac playback) we are given nothing told nothing but that the atv can withstand short term bitrates up to 12,000 kbps. Problem is we know nothing of the buffer duration, etc which is just as important to accurate setting vbv (hence the "vbv-bufsize" portion of the settings).

Sorry for hi-jacking the thread and getting so off-topic... please, just a bit more ...
Would this true - the vbv-bufsize is a measurement of time? in 1/1000s of a second? and the vbv is the bitrate allowed on average over that duration? or am I out to lunch?

4500kbps is staggeringly high for most content

No kidding! The source in question is a music video with lots of fast cuts and pans - Public Enemy's Fight the Power. I don't know the buffer of my mini, but my mini and NAS are on a gigabit LAN. I didn't have any stuttering playing the uncapped encode which had an average bitrate of 9.88 Mbits/sec. My goal in limiting file size really is due to my belief that 9.88 Mbps is ridiculous!

Dynaflash, I'll read the doom9 thread too.

Thanks,
--sdm.

edit added:
just found this enlightening description in dynaflash's link:

If you set qcomp=0 (i.e. "bitrate variability"), then it tries to make each frame use exactly the same number of bits. It's not quite that precise, but that's its goal.

If you set vbv-maxrate and vbv-bufsize to some nonzero values, then it restricts the average bitrate of a certain number of frames (the number being howevermany can fit in your specified vbv-bufsize (which is specified in kbit)). If vbv-maxrate==bitrate, then it's CBR; otherwise it just restricts the peak bitrate.

You can mix any value of qcomp with the vbv stuff, in which case qcomp controls the local allocation of bits within the averaging window. (As opposed to the non-vbv case, where qcomp distributes bits over the whole movie.)

I think the name "VBV" was taken from MPEG-2 or the like. The H.264 standard calls it "HRD". Theoretically, vbv-maxrate is the rate at which the decoder can read from the medium (DVD, network, etc) and vbv-bufsize is the amount of memory it has available to pre-buffer the incoming stream.

I see my analysis above is a little wrong but is the right direction...

[rhester]

Would this true - the vbv-bufsize is a measurement of time? in 1/1000s of a second?

No. The UOM of vbv-bufsize is in kbit.

and the vbv is the bitrate allowed on average over that duration? or am I out to lunch?

But this is correct - it's the average bitrate allowed within that size. dynaflash put together a little bitrate calculator that will calculate the number of seconds of video said buffer can hold to get to the number you're thinking of.

4500kbps is staggeringly high for most content

No kidding! The source in question is a music video with lots of fast cuts and pans - Public Enemy's Fight the Power. I don't know the buffer of my mini, but my mini and NAS are on a gigabit LAN. I didn't have any stuttering playing the uncapped encode which had an average bitrate of 9.88 Mbits/sec. My goal in limiting file size really is due to my belief that 9.88 Mbps is ridiculous!

Have you experimented with CRF?

Rodney

[sdm]

Have you experimented with CRF?

Rodney

Yes, I've encoded with various CRF settings.
I'm gonna call it a day with this encode. (playback looks fine capped using 5900/4500 with CRF=65. the average bitrate ended up about 4900kbps. I can live with that I guess)

Thanks for your insights.
--sdm.

[dynaflash]

(playback looks fine capped using 5900/4500 with CRF=65. the average bitrate ended up about 4900kbps. I can live with that I guess)

Wow, thats wicked high for 65 imho. When you go back to this source, tack on Weak Denoise, my guess is that this source is very noisy (I think you said concert footage). Weak denoise on this type of source with crf will help alot. I would not use anything stronger than weak though.

[sdm]

(playback looks fine capped using 5900/4500 with CRF=65. the average bitrate ended up about 4900kbps. I can live with that I guess)

Wow, thats wicked high for 65 imho. When you go back to this source, tack on Weak Denoise, my guess is that this source is very noisy (I think you said concert footage). Weak denoise on this type of source with crf will help alot. I would not use anything stronger than weak though.

Hi dynaflash,

A while ago I was using weak denoise on most encodes because I liked the bitrate reduction and also the way it temporally stabilizes the image in supposed-to-be static parts of video (low-detail walls for example). But then I decided I didn't like the way it (slightly) smoothes out textures like brick, ashphalt, beard stuble, fabrics etc...
Everything is a tradeoff in this game it seems.

Anyways, I had already given that a try. You're right it definitely does reduce the bitrate (and filesize).

The source doesn't appear overly noisy. It does seem really sharp though, like maybe it was sharpened before encoding to mpeg2. You're welcome to take a look at the source if you're interested. Just PM me.

Thanks for your help.
--sdm.

[eddyg]

I also used to use weak denoise all the time - and have since stopped - the lack of stubble and "plastic" looking faces turned me off of it.

If I get the time I'd like to port a different denoiser.

Cheers, Ed.

[nightstrm]

I also used to use weak denoise all the time - and have since stopped - the lack of stubble and "plastic" looking faces turned me off of it.

If I get the time I'd like to port a different denoiser.

Cheers, Ed.

Same here; I no longer use it by default in my encodes as well.

[rhester]

*cough*need temporal denoising ala TTempSmooth*cough*



Rodney

[Starhawk]

Very similar to the original poster, I am trying to use VBV to restrain bit rate for a file that will be used on a Mac exclusively and am just trying to get a grasp of how to use it in my case.

Im CRF encoding a standard def home movie that is coming out much larger in file size than I'd like. By watching the bit rate monitor in VLC, most of the file is a good 2500-3000 kbps, but some grainy night shots are around 8000 kbps. I'd like to cap the rate at around 5000kbps to see how it turns out, but knowing that this is for use on a Mac, how should I set the vbv buffer? If Im understanding this correctly, if Im not concerned with bitrate spikes, then I can just set it for 5000/5000?

Thanks for any help! Im trying to take in the info http://trac.handbrake.fr/wiki/VBVRateControl but appreciate the clarification in this case.

[Starhawk]

Maybe I shall wait until the new VBV Lookahead is available in x264...

[dynaflash]

I would wait

[tlindgren]

I wouldn't lower the percentage, but I would add vbv to cap off the bitrate.

Cavalicious, I've been looking into VBV and browsed this:
http://trac.handbrake.fr/wiki/VBVRateControl,

A non-discriminatory capping of bitrate would lead to serious image degradation in highly complex scenes - right?

The alternative might well be stuttering or even video corruptions which isn't nice either. Hardware decoders tends to be less powerful so it tends to affects them a lot more more but at higher levels this can actually overwhelm almost any hardware pretty quickly if it really goes all-out. The hope is that the level and profile you've selected will allow enough buffer and bitrate for this not to be a serious problem.

I get the impression VBV isn't blindly 'capping' the bitrate, rather using some sort of buffer and intelligently distributing bits to cap bitrate spikes? Is that true?
If so, without becoming an expert, how do I know what settings are appropriate?

From what I can understand if you're coding to a specific h.264 level and profile this gives you the VBV parameters that hardware should support, so perhaps x264 should set these for you when you set level (it doesn't).

There's conflicting claims on what is allowed but both MeGUI and x.264 seems to limit vbv-buffersize to "maximum bitrate in kbps for Profile/Level" and vbv-maxrate to "maximum bitrate in kbps for Main/Level". However, most of the MeGUI DXVA profiles that use VBV is slightly more conservative and defaults to the lower value for both (except the DXVA-SD which uses it all). Also, while Blu-ray is "mostly" Level 4.1 and quite a bit of the existing hardware devices are really "Level 4 or Level 4.1 for Blu-ray", with a variety of restrictions on bitrate, VBV, max keyframe interval and so on.

Given that a significant number of people seems uses MeGUI with these settings (and x.264 doesn't warn about them, it does if you exceed them) I think at they should be reasonable safe. If your hardware lists a maximum bitrate you can likely do the same calculation using that value instead.

On an extremely bit-hungry encode of a music video (66% crf, uncapped resulted in 9.88 mbits/s), I tried this:

Code: Select all

:vbv-maxrate=4900:vbv-bufsize=3500

at the end of my advanced x264 options. ( I just copied that from somewhere else in this thread)

Those VBV values looks low even for a Level 3 encode (max 40500 macroblocks, corresponds to 720×480@30). If this really is a SD/L3 encode I suspect that using 10000/10000 should at least stop the underflow warnings but may still limit the bitrate slightly in some scenes.

However, if all the playback devices support a higher level you could declare that and use the higher allowed max bitrate. Some examples extrapolated from MeGUI:

L3 (480p30): level=3:vbv-maxrate=10000:vbv-bufsize=10000
L3.1 (720p30): level=31:vbv-maxrate=14000:vbv-bufsize=14000 (really 14k/17.5k)
L3.1/AppleTV: level=31:vbv-maxrate=12000:vbv-bufsize=5000
L3.2 (720p60): level=32:vbv-maxrate=20000:vbv-bufsize=20000
L4 (1080p30): level=4:vbv-maxrate=20000:vbv-bufsize=20000
L4.1 (1080p30): level=41:vbv-maxrate=50000:vbv-bufsize=50000 (actual)
L4.1/PS3+XBox360: level=41:vbv-maxrate=24000:vbv-bufsize=24000 (actual)
L4.1/Blu-ray: level=41:vbv-maxrate=40000:vbv-bufsize=30000:keyint=24:min-keyint=2 (actual)
L4.1/AVC-HD: level=41:vbv-maxrate=16500:vbv-bufsize=16500:keyint=24:min-keyint=2 (actual)
L4.2 (1080p60): level=42:vbv-maxrate=50000:vbv-bufsize=50000
L5 (1080p72/1920p30): level=5:vbv-maxrate=135000:vbv-bufsize=135000
L5.1 (1080p120/2048p30): level=5:vbv-maxrate=240000:vbv-bufsize=240000

Actual means I cribbed it directly from the settings, the rest was derived from Wikipedia but several was confirmed in MeGUI settings windows (it complains if you set VBV values too high).

I suspect the AppleTV and PS3+XBox360 has been empirically derived, they may handle more on newer x.264 if it's better at following the VBV settings you've give.

http://en.wikipedia.org/wiki/H.264#Levels

Note that if you want hardware accelerated playback on modern graphics cards (and you definitely do!) there's some additional restrictions on the number of Reference Frames. However, most seems to indicate that using large number of refs isn't that useful anyway and recommends a maximum of 3 which is safe even at 1080p (max 4, on lower resolutions it's higher). But if it's true that some players only activates DXVA on L3.1 and L4.1 files that sucks but perhaps I should use 3.1 on SD content instead of L3.

http://www.avsforum.com/avs-vb/showpost ... ostcount=1

Damn, perhaps I should sit down and see if I can add a Level/Profile setting to the Advanced settings page in the Windows GUI
It should default to None to start with, hopefully switching to Auto later after debugging, this would find Main/High and Level from the other settings then set VBV based on that (possibly allow manual overrides).