[Fixed]segmentation faults with flv videos

[kevin83]

Hello! I'm new here, so first I'd like to thank the developers for this awesome program that has helped me with watching movies on my ipod nano quite a bit.
Recently I've been having issues with the svn builds of handbrake gtk and cli versions on linux. When I try to encode an flv formatted video with handbrake, it works on my mac but fails completely on my netbook running ubuntu 9.04 (I use an external monitor to fix the screen size issues). From what the log says I think it's been segfaulting. Also, this has been happening since I started using svn around build 2450 and this log is from 2533.
Here's the log:

Code: Select all

mcgeehan@mcgeehan-laptop:~$ HandBrakeCLI -i /media/EEEPC\ \BOOT/Amish\ \Paradise.flv -o /home/mcgeehan/Desktop/Amish\ \Paradise.mp4 -e x264 -b 750 -v
[13:53:38] hb_init: checking cpu count
[13:53:38] hb_init: starting libhb thread
HandBrake svn2533 (2009061401) - Linux i686 - http://handbrake.fr
2 CPUs detected
Opening /media/EEEPC BOOT/Amish Paradise.flv...
[13:53:38] hb_scan: path=/media/EEEPC BOOT/Amish Paradise.flv, title_index=1
[13:53:38] scan: trying to open with libdvdread
libdvdread: Encrypted DVD support unavailable.
libdvdnav:DVDOpenFileUDF:UDFFindFile /VIDEO_TS/VIDEO_TS.IFO failed
libdvdnav:DVDOpenFileUDF:UDFFindFile /VIDEO_TS/VIDEO_TS.BUP failed
libdvdread: Can't open file VIDEO_TS.IFO.
ERROR: dvd: ifoOpen failed
[flv @ 0x9460830]skipping flv packet: type 18, size 294, flags 0
Input #0, flv, from '/media/EEEPC BOOT/Amish Paradise.flv':
  Duration: 00:03:25.12, start: 0.000000, bitrate: N/A
    Stream #0.0: Video: h264, yuv420p, 320x180 [PAR 1:1 DAR 16:9], 29.92 tbr, 1k tbn, 2k tbc
    Stream #0.1: Audio: aac, 22050 Hz, stereo, s16
[13:53:38] scan: decoding previews for title 1
[13:53:38] scan: 10 previews, 320x180, 29.917 fps, autocrop = 2/4/32/32, aspect 1.78:1, PAR 1:1
[13:53:38] scan: title (0) job->width:256, job->height:176
[13:53:38] libhb: scan thread found 1 valid title(s)
+ title 1:
  + vts 0, ttn 0, cells 0->0 (0 blocks)
  + duration: 00:03:25
  + size: 320x180, pixel aspect: 1/1, display aspect: 1.78, 29.917 fps
  + autocrop: 2/4/32/32
  + chapters:
    + 1: cells 0->0, 0 blocks, duration 00:03:25
  + audio tracks:
    + 1, Unknown (AAC)
  + subtitle tracks:
[13:53:38] 1 job(s) to process
[13:53:38] starting job
[13:53:38] work: sanitizing track 0 mixdown Dolby Pro Logic II to Stereo
[13:53:38] job configuration:
[13:53:38]  * source
[13:53:38]    + /media/EEEPC BOOT/Amish Paradise.flv
[13:53:38]    + title 1, chapter(s) 1 to 1
[13:53:38]    + container: flv
[13:53:38]  * destination
[13:53:38]    + /home/mcgeehan/Desktop/Amish Paradise.mp4
[13:53:38]    + container: MPEG-4 (.mp4 and .m4v)
[13:53:38]  * video track
[13:53:38]    + decoder: h264
[13:53:38]    + frame rate: same as source (around 29.917 fps)
[13:53:38]    + dimensions: 320 * 180 -> 256 * 176, crop 2/4/32/32
[13:53:38]    + encoder: x264
[13:53:38]      + bitrate: 750 kbps, pass: 0
[13:53:38]  * audio track 0
[13:53:38]    + decoder: Unknown (AAC) (track 1, id 1)
[13:53:38]    + mixdown: Stereo
[13:53:38]    + encoder: faac
[13:53:38]      + bitrate: 160 kbps, samplerate: 22050 Hz
libdvdread: Encrypted DVD support unavailable.
libdvdnav:DVDOpenFileUDF:UDFFindFile /VIDEO_TS/VIDEO_TS.IFO failed
libdvdnav:DVDOpenFileUDF:UDFFindFile /VIDEO_TS/VIDEO_TS.BUP failed
libdvdread: Can't open file VIDEO_TS.IFO.
ERROR: dvd: ifoOpen failed
[flv @ 0x947bb00]skipping flv packet: type 18, size 294, flags 0
[13:53:38] reader: first SCR 0
[13:53:38] encx264: keyint-min: 30, keyint-max: 300
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile Main, level 2.0
Segmentation fault

and the gtk fail log:

Code: Select all

Handbrake Version: svn2533 (2009061401)
[14:26:18] hb_scan: path=/media/EEEPC BOOT/Amish Paradise.flv, title_index=1
[14:26:18] scan: trying to open with libdvdread
libdvdread: Encrypted DVD support unavailable.
libdvdnav:DVDOpenFileUDF:UDFFindFile /VIDEO_TS/VIDEO_TS.IFO failed
libdvdnav:DVDOpenFileUDF:UDFFindFile /VIDEO_TS/VIDEO_TS.BUP failed
libdvdread: Can't open file VIDEO_TS.IFO.
[14:26:18] dvd: ifoOpen failed
[flv @ 0xadacce0]skipping flv packet: type 18, size 294, flags 0
Input #0, flv, from '/media/EEEPC BOOT/Amish Paradise.flv':
  Duration: 00:03:25.12, start: 0.000000, bitrate: N/A
    Stream #0.0: Video: h264, yuv420p, 320x180 [PAR 1:1 DAR 16:9], 29.92 tbr, 1k tbn, 2k tbc
    Stream #0.1: Audio: aac, 22050 Hz, stereo, s16
[14:26:18] scan: decoding previews for title 1
[14:26:18] scan: 10 previews, 320x180, 29.917 fps, autocrop = 2/4/32/32, aspect 1.78:1, PAR 1:1
[14:26:18] scan: title (0) job->width:256, job->height:176
[14:26:18] libhb: scan thread found 1 valid title(s)
[14:26:20] lingui: Modified Preset: Regular->Normal
[14:26:20] 2 job(s) to process
[14:26:20] starting job
[14:26:20] job configuration:
[14:26:20]  * source
[14:26:20]    + /media/EEEPC BOOT/Amish Paradise.flv
[14:26:20]    + title 1, chapter(s) 1 to 1
[14:26:20]    + container: flv
[14:26:20]  * destination
[14:26:20]    + /home/mcgeehan/Desktop/Amish Paradise.m4v
[14:26:20]    + container: MPEG-4 (.mp4 and .m4v)
[14:26:20]  * video track
[14:26:20]    + decoder: h264
[14:26:20]    + frame rate: same as source (around 29.917 fps)
[14:26:20]    + dimensions: 320 * 180 -> 256 * 160, crop 9/11/32/32
[14:26:20]    + encoder: x264
[14:26:20]      + bitrate: 750 kbps, pass: 1
[14:26:20]  * audio track 0
[14:26:20]    + decoder: Unknown (AAC) (track 1, id 1)
[14:26:20]    + mixdown: Stereo
[14:26:20]    + encoder: faac
[14:26:20]      + bitrate: 160 kbps, samplerate: 22050 Hz
[14:26:20] encx264: keyint-min: 30, keyint-max: 300
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile Main, level 2.0
libdvdread: Encrypted DVD support unavailable.
libdvdnav:DVDOpenFileUDF:UDFFindFile /VIDEO_TS/VIDEO_TS.IFO failed
libdvdnav:DVDOpenFileUDF:UDFFindFile /VIDEO_TS/VIDEO_TS.BUP failed
libdvdread: Can't open file VIDEO_TS.IFO.
[14:26:20] dvd: ifoOpen failed
[flv @ 0xa39af20]skipping flv packet: type 18, size 294, flags 0
[14:26:20] sync: expecting 6166 video frames
[14:26:20] reader: first SCR 0
[14:26:20] sync: first pts is 2970
[14:28:47] reader: done. 1 scr changes
[14:28:53] sync: got 6134 frames, 6166 expected
[14:28:53] work: average encoding speed for job is 39.935131 fps
[14:28:53] h264-decoder done: 6135 frames, 0 decoder errors, 0 drops
[14:28:53] render: lost time: 0 (0 frames)
[14:28:53] render: gained time: 0 (0 frames) (0 not accounted for)
x264 [info]: slice I:74    Avg QP:10.71  size:  6296  PSNR Mean Y:54.18 U:59.36 V:58.79 Avg:55.19 Global:54.01
x264 [info]: slice P:6060  Avg QP:11.09  size:  2950  PSNR Mean Y:51.09 U:54.22 V:53.79 Avg:51.83 Global:51.34
x264 [info]: mb I  I16..4: 26.9%  0.0% 73.1%
x264 [info]: mb P  I16..4:  1.4%  0.0%  6.5%  P16..4: 31.3% 23.8% 24.6%  0.0%  0.0%    skip:12.4%
x264 [info]: final ratefactor: 9.37
x264 [info]: coded y,uvDC,uvAC intra:80.4% 88.9% 78.8% inter:52.5% 66.9% 18.2%
x264 [info]: SSIM Mean Y:0.9960286
x264 [info]: PSNR Mean Y:51.125 U:54.284 V:53.851 Avg:51.869 Global:51.364 kb/s:715.66
[14:28:53] aac-decoder done: 0 frames, 0 decoder errors, 0 drops
[14:28:53] starting job
[14:28:53] job configuration:
[14:28:53]  * source
[14:28:53]    + /media/EEEPC BOOT/Amish Paradise.flv
[14:28:53]    + title 1, chapter(s) 1 to 1
[14:28:53]    + container: flv
[14:28:53]  * destination
[14:28:53]    + /home/mcgeehan/Desktop/Amish Paradise.m4v
[14:28:53]    + container: MPEG-4 (.mp4 and .m4v)
[14:28:53]  * video track
[14:28:53]    + decoder: h264
[14:28:53]    + frame rate: same as source (around 29.917 fps)
[14:28:53]    + dimensions: 320 * 180 -> 256 * 160, crop 9/11/32/32
[14:28:53]    + encoder: x264
[14:28:53]      + options: ref=2:me=umh:cabac=0
[14:28:53]      + bitrate: 750 kbps, pass: 2
[14:28:53]  * audio track 0
[14:28:53]    + decoder: Unknown (AAC) (track 1, id 1)
[14:28:53]    + mixdown: Stereo
[14:28:53]    + encoder: faac
[14:28:53]      + bitrate: 160 kbps, samplerate: 22050 Hz
libdvdread: Encrypted DVD support unavailable.
libdvdnav:DVDOpenFileUDF:UDFFindFile /VIDEO_TS/VIDEO_TS.IFO failed
libdvdnav:DVDOpenFileUDF:UDFFindFile /VIDEO_TS/VIDEO_TS.BUP failed
libdvdread: Can't open file VIDEO_TS.IFO.
[14:28:53] dvd: ifoOpen failed
[flv @ 0xada7eb0]skipping flv packet: type 18, size 294, flags 0

I
when the gtk crashes the whole program just quits without warning. I'm really clueless as to why this would happen.

[JohnAStebbins]

Can't tell what's going on there. How would you like to run it under gdb and give us a backtrace?
First, build it with debugging enabled:

Code: Select all

cd [your handbrake sources]
mkdir build.debug
cd build.debug
../configure --debug=max --optimize=none
make

Have a coffee... then

Code: Select all

gdb ./HandBrakeCLI
run -i /media/EEEPC\ \BOOT/Amish\ \Paradise.flv -o /home/mcgeehan/Desktop/Amish\ \Paradise.mp4 -e x264 -b 750 -v

if it hasn't magically fixed itself, it will crash and give you back a gdb prompt

Code: Select all

bt

should provide some indication where it crashed and how it got there.

[kevin83]

wow. that took a long time.
here's the once again segfaulting output from gdb.

Code: Select all

(gdb) run -i /media/EEEPC\ \BOOT/Amish\ \Paradise.flv -o /home/mcgeehan/Desktop/Amish\ \Paradise.mp4 -e x264 -b 750 -v
Starting program: /home/mcgeehan/hb-trunk/build.debug/HandBrakeCLI -i /media/EEEPC\ \BOOT/Amish\ \Paradise.flv -o /home/mcgeehan/Desktop/Amish\ \Paradise.mp4 -e x264 -b 750 -v
[Thread debugging using libthread_db enabled]
[15:45:45] hb_init: checking cpu count
[15:45:45] hb_init: starting libhb thread
[New Thread 0xb7d626d0 (LWP 21929)]
[New Thread 0xb7d61b90 (LWP 21930)]
HandBrake svn2533 (2009061401) - Linux i686 - http://handbrake.fr
2 CPUs detected
Opening /media/EEEPC BOOT/Amish Paradise.flv...
[15:45:45] hb_scan: path=/media/EEEPC BOOT/Amish Paradise.flv, title_index=1
[New Thread 0xb7560b90 (LWP 21931)]
[15:45:45] scan: trying to open with libdvdread
libdvdread: Encrypted DVD support unavailable.
libdvdnav:DVDOpenFileUDF:UDFFindFile /VIDEO_TS/VIDEO_TS.IFO failed
libdvdnav:DVDOpenFileUDF:UDFFindFile /VIDEO_TS/VIDEO_TS.BUP failed
libdvdread: Can't open file VIDEO_TS.IFO.
ERROR: dvd: ifoOpen failed
[flv @ 0x9fc7860]skipping flv packet: type 18, size 294, flags 0
Input #0, flv, from '/media/EEEPC BOOT/Amish Paradise.flv':
  Duration: 00:03:25.12, start: 0.000000, bitrate: N/A
    Stream #0.0: Video: h264, yuv420p, 320x180 [PAR 1:1 DAR 16:9], 29.92 tbr, 1k tbn, 2k tbc
    Stream #0.1: Audio: aac, 22050 Hz, stereo, s16
[15:45:45] scan: decoding previews for title 1
Scanning title 1...
Scanning title 1...
[15:45:46] scan: 10 previews, 320x180, 29.917 fps, autocrop = 2/4/32/32, aspect 1.78:1, PAR 1:1
[15:45:46] scan: title (0) job->width:256, job->height:176
[Thread 0xb7560b90 (LWP 21931) exited]
[15:45:46] libhb: scan thread found 1 valid title(s)
+ title 1:
  + vts 0, ttn 0, cells 0->0 (0 blocks)
  + duration: 00:03:25
  + size: 320x180, pixel aspect: 1/1, display aspect: 1.78, 29.917 fps
  + autocrop: 2/4/32/32
  + chapters:
    + 1: cells 0->0, 0 blocks, duration 00:03:25
  + audio tracks:
    + 1, Unknown (AAC)
  + subtitle tracks:
[New Thread 0xb7560b90 (LWP 21932)]
[15:45:46] 1 job(s) to process
[15:45:46] starting job
[15:45:46] work: sanitizing track 0 mixdown Dolby Pro Logic II to Stereo
[15:45:46] job configuration:
[15:45:46]  * source
[15:45:46]    + /media/EEEPC BOOT/Amish Paradise.flv
[15:45:46]    + title 1, chapter(s) 1 to 1
[15:45:46]    + container: flv
[15:45:46]  * destination
[15:45:46]    + /home/mcgeehan/Desktop/Amish Paradise.mp4
[15:45:46]    + container: MPEG-4 (.mp4 and .m4v)
[15:45:46]  * video track
[15:45:46]    + decoder: h264
[15:45:46]    + frame rate: same as source (around 29.917 fps)
[15:45:46]    + dimensions: 320 * 180 -> 256 * 176, crop 2/4/32/32
[15:45:46]    + encoder: x264
[15:45:46]      + bitrate: 750 kbps, pass: 0
[15:45:46]  * audio track 0
[15:45:46]    + decoder: Unknown (AAC) (track 1, id 1)
[15:45:46]    + mixdown: Stereo
[15:45:46]    + encoder: faac
[15:45:46]      + bitrate: 160 kbps, samplerate: 22050 Hz
[New Thread 0xb6d5fb90 (LWP 21933)]
[New Thread 0xb655eb90 (LWP 21934)]
libdvdread: Encrypted DVD support unavailable.
libdvdnav:DVDOpenFileUDF:UDFFindFile /VIDEO_TS/VIDEO_TS.IFO failed
libdvdnav:DVDOpenFileUDF:UDFFindFile /VIDEO_TS/VIDEO_TS.BUP failed
libdvdread: Can't open file VIDEO_TS.IFO.
ERROR: dvd: ifoOpen failed
[New Thread 0xb5d5db90 (LWP 21935)]
[15:45:46] encx264: keyint-min: 30, keyint-max: 300
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile Main, level 2.0
[New Thread 0xb5199b90 (LWP 21936)]
[flv @ 0xa135f80]skipping flv packet: type 18, size 294, flags 0
[New Thread 0xb4998b90 (LWP 21937)]
[15:45:46] reader: first SCR 0

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xb4998b90 (LWP 21937)]
0x08364ca5 in float_to_int16_interleave_sse2 (dst=0xb3692008, src=0xa197e30, len=-4096, channels=2) at libavcodec/x86/dsputil_mmx.c:2486
2486	FLOAT_TO_INT16_INTERLEAVE(sse2,
(gdb) bt
#0  0x08364ca5 in float_to_int16_interleave_sse2 (dst=0xb3692008, src=0xa197e30, len=-4096, channels=2) at libavcodec/x86/dsputil_mmx.c:2486
#1  0x080a7472 in aac_decode_frame (avccontext=0xa136f20, data=0xb3691008, data_size=0xb49982f8, avpkt=0xb499819c) at libavcodec/aac.c:1696
#2  0x080a3f9a in avcodec_decode_audio2 (avctx=0xa136f20, samples=0xb3691008, frame_size_ptr=0xb49982f8, buf=0xa166400 "!", buf_size=9)
    at libavcodec/utils.c:620
#3  0x080809e1 in decodeAudio (pv=0xa135db8, data=0xa166400 "!", size=9) at ../libhb/decavcodec.c:1154
#4  0x08080f2b in decavcodecaiWork (w=0x9fbf580, buf_in=0xb4998368, buf_out=0xb4998364) at ../libhb/decavcodec.c:1260
#5  0x0807dc89 in work_loop (_w=0x9fbf580) at ../libhb/work.c:1103
#6  0x0806ec07 in hb_thread_func (_t=0x9ffb208) at ../libhb/ports.c:291
#7  0xb7ff54ff in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
#8  0xb7e4849e in clone () from /lib/tls/i686/cmov/libc.so.6

[JohnAStebbins]

Thanks. Seems to be a bug in decoding aac audio with ffmpeg. Either they have a new bug, or we are not respecting some buffer alignment constraints that they require. This gives us a lead to work with.

[kevin83]

some quick questions:
since this is a bug in the aac decoder, does it even have anything to do with flv videos?
Is this a problem with handbrake or with ffmpeg?
How big of a problem does this seem to be?

I also have another issue. It seems that handbrake sees mp3 audio in the flv container as aac, and it doesn't treat it any differently than aac audio. I have no clue why it does this, but it might be because handbrake never seems to compile with lame, theora, or vorbis enabled.

[JohnAStebbins]

I'm trying to reproduce this, but have not been able to find any flv files with aac audio. I tried aac in other containers, and it worked fine. Any chance you could supply a sample of a video that causes this crash?

[JohnAStebbins]

I just did a test transcode of flv with mp3 to mkv with aac. Handbrake properly detected the mp3 and the encode worked perfectly. You'll have to show us an activity log of your mp3 problem.

Handbrake does not rely on your system libs for any of the codecs it uses. During build, it downloads and builds local copies of the specific versions of all the codec libraries that we test it against. So there can not be any missing support for codecs.

[KonaBlend]

Should be fixed as of changeset 2548 .
See also changeset 2549 .

[kevin83]

I can confirm that this is fixed for me on multiple videos as of r2550. Thanks for the help guys. This also fixed the 2nd pass crashing in the gtk build, along with other odd crashing issues when I encoded video.