h264 vs x264 in terms of color banding

[frank44]

I usually render my video projectsfirst as a lossless .avi right out of after effects, which I then import in adobe media encoder, and export with the h264 codec. No matter how high my settings are, I tend to get color banding artifacts.

If I render the same .avi video with x264 and an equivalent bitrate, can I expect better results, so less color banding?

[JohnAStebbins]

I don't use adobe, so can't compare. But banding is a result of quantization which is part of the compression process. Some encoders will more aggressively quantize to achieve the target settings than others. 8bit encoders suffer more from this than 10 or 12 bit encoders.

Pick a short sample (or use HandBrake's point to point options to encode a short segment of a larger sample) and run some tests.

[Deleted User 13735]

I usually render my video projectsfirst as a lossless .avi right out of after effects,

From what source? 8 or 10 bit?
To what AVI format? 8 or 10 bit?

[frank44]

I usually render my video projectsfirst as a lossless .avi right out of after effects,

From what source? 8 or 10 bit?
To what AVI format? 8 or 10 bit?

My AE project is only 8bpc. So I guess my .avi is also 8bit. Would it help to render my 8 bit .avi source as 10bit in Handbrake?

[mduell]

Not unless you believe in magic.

Yes, I'd expect better results from x264 than Adobe Media Encoder at the same bitrate and roughly same encoding time (i.e. a reasonable x264 preset).

[Deleted User 13735]

The banding is in the quantization (dither) to 8 bit, and is aggravated further by the transform to 4:2:0 during transcoding.

Yes, there is a difference between encoders, and x264 is the best that I, or MSU for that matter, has tested.

The banding and shadow noise is identifiably worse with AVI to Mainconcept .h264, for instance.