Very Large Input Frame Support

Archive of historical development discussions
Discussions / Development has moved to GitHub
Forum rules
*******************************
Please be aware we are now using GitHub for issue tracking and feature requests.
- This section of the forum is now closed to new topics.

*******************************
Post Reply
Techrocket9
Posts: 1
Joined: Fri Jun 12, 2015 5:38 am

Very Large Input Frame Support

Post by Techrocket9 »

I am working on a project involving video files with very large frames (5760x2176). I needed to compress the raw avi file for transmission over the internet to its intended audience, and I turned to HandBrakeCLI and x265 to handle the job. I did this because the h.264 standard does not actually support frame sizes that large, but h.265 does.

When I went to encode my video I got errors about "ffmpeg_read: pkt too big".

Digging into the HandBrake source I discovered in \libhb\stream.c line 5723 code that checks for absurd sizes from ffmpeg and declines to enlarge the buffer if they occur. The check performed is

Code: Select all

// sometimes we get absurd sizes from ffmpeg
if ( stream->ffmpeg_pkt->size >= (1 << 25) )
This is fine for video that fits the constraints of the h.264 standard, but not for h.265's extreme limits. A single packet of my source file exceeds 2^25 bytes and is not in error. To encode my project I increased the limit to 2^28 bytes and recompiled from source, which allowed my encode to proceed as expected.

Code: Select all

// sometimes we get absurd sizes from ffmpeg, but we still need to work with the very large resolutions supported by x265
if ( stream->ffmpeg_pkt->size >= (1 << 28) )
I am suggesting that the size of this absurdity check needs to be re-evaluated in light of the new frame size options exposed by the x265 encoder. 2^28 worked for my project, but my test case is a far cry from h.265's theoretical max of 8192×4320
User avatar
JohnAStebbins
HandBrake Team
Posts: 5712
Joined: Sat Feb 09, 2008 7:21 pm

Re: Very Large Input Frame Support

Post by JohnAStebbins »

This code was added back in 2008 to handle bugs in ffmpeg's VC1 decoder.

Commit message:
van wrote: Don't abort when the ffmpeg vc1 decoder hands us a bogus packet size
There have been many bug fixes to the VC1 decoder since then. I think it is likely fixed by now. My suggestion would be to change this to increase the threshold to something that can accommodate 16K raw 10bit images (~2^30) and then just log the occurrence and continue. Or eliminate it entirely and hope for the best :mrgreen:
Mortimer
Posts: 8
Joined: Tue May 19, 2015 8:51 pm

Re: Very Large Input Frame Support

Post by Mortimer »

JohnAStebbins wrote:This code was added back in 2008 to handle bugs in ffmpeg's VC1 decoder.

Commit message:
van wrote: Don't abort when the ffmpeg vc1 decoder hands us a bogus packet size
There have been many bug fixes to the VC1 decoder since then. I think it is likely fixed by now. My suggestion would be to change this to increase the threshold to something that can accommodate 16K raw 10bit images (~2^30) and then just log the occurrence and continue. Or eliminate it entirely and hope for the best :mrgreen:
Potentially one log entry per large frame? Yikes!
User avatar
JohnAStebbins
HandBrake Team
Posts: 5712
Joined: Sat Feb 09, 2008 7:21 pm

Re: Very Large Input Frame Support

Post by JohnAStebbins »

Mortimer wrote:
JohnAStebbins wrote:This code was added back in 2008 to handle bugs in ffmpeg's VC1 decoder.

Commit message:
van wrote: Don't abort when the ffmpeg vc1 decoder hands us a bogus packet size
There have been many bug fixes to the VC1 decoder since then. I think it is likely fixed by now. My suggestion would be to change this to increase the threshold to something that can accommodate 16K raw 10bit images (~2^30) and then just log the occurrence and continue. Or eliminate it entirely and hope for the best :mrgreen:
Potentially one log entry per large frame? Yikes!
We have a logging function that strips duplicates and prints "Last error repeated X times" just for such eventualities.
Post Reply