No valid Source when opening H.265 file

[SteveBu]

Description of problem or question:
I have some Video recorded from a HikVision CCTV camera which handbrake will not open - They will however play in VLC



Steps to reproduce the problem (If Applicable):
Just try opening the file



HandBrake version (e.g., 1.0.0):
1.1.1 and 1.1.2



Operating system and version (e.g., Ubuntu 16.04 LTS, macOS 10.13 High Sierra, Windows 10 Creators Update):
macOS 10.13.6 High Sierra



HandBrake Activity Log ***required*** (see How-to get an activity log)

Code: Select all

[12:15:20] macgui: Handbrake Version: 1.1.2  (2018090500)
[12:15:20] hb_init: starting libhb thread
[12:15:20] hb_init: starting libhb thread
[12:15:31] macgui: trying to open a folder or file
[12:15:31] macgui: ScanCore scanning titles with a duration of 10 seconds or more
[12:15:31] CPU: Intel(R) Core(TM) i5-4260U CPU @ 1.40GHz
[12:15:31]  - Intel microarchitecture Haswell
[12:15:31]  - logical processor count: 4
[12:15:31] hb_scan: path=/Volumes/BitBucket3/Camera04_20180901163951_20180901182959_52164.mp4, title_index=0
udfread ERROR: ECMA 167 Volume Recognition failed
disc.c:323: failed opening UDF image /Volumes/BitBucket3/Camera04_20180901163951_20180901182959_52164.mp4
disc.c:424: error opening file BDMV/index.bdmv
disc.c:424: error opening file BDMV/BACKUP/index.bdmv
[12:15:31] bd: not a bd - trying as a stream/file instead
libdvdnav: Using dvdnav version 6.0.0
libdvdread: Encrypted DVD support unavailable.
libdvdread:DVDOpenFileUDF:UDFFindFile /VIDEO_TS/VIDEO_TS.IFO failed
libdvdread:DVDOpenFileUDF:UDFFindFile /VIDEO_TS/VIDEO_TS.BUP failed
libdvdread: Can't open file VIDEO_TS.IFO.
libdvdnav: vm: failed to read VIDEO_TS.IFO
[12:15:31] dvd: not a dvd - trying as a stream/file instead
[12:15:31] file is MPEG Program Stream
[12:15:31] Found program stream map
.... MANY DUPLICATE LINES REMOVED ....
[12:15:31] Found program stream map
[12:15:32] Probing 3 unknown streams
[12:15:35] Found the following streams
[12:15:35]     Video Streams : 
[12:15:35]     Audio Streams : 
[12:15:35]     Subtitle Streams : 
[12:15:35]     Other Streams : 
[12:15:35]       0xe0-0x0 type Unknown (0x24)
[12:15:35]       0xc0-0x0 type AC3/IGS (0x91)
[12:15:35]       0xbd-0x0 type Unknown (0x0)
[12:15:35] libhb: scan thread found 0 valid title(s)
[12:15:35] macgui: ScanCore scan done

[SteveBu]

For Info this is what VLC reports about the file when playing :

Code: Select all

Stream 0
Type                     Video
Video Resolution    3072 x 2048
Buffer dimensions  3072 x 2048
Frame Rate            20
Decoded Format     Planar 4:2:0 YUV Full Scale
Orientation            Top Left
Color Primaries      ITU-R BT.709
Color transfer function   ITU-R BT.709
Color space            ITU-R BT.709 Range
Stream 1
Codec                    CVD subtitles (cvd)
Type                      Subtitle

[mduell]

Try the nightly.

CCTV systems sometimes produce not-quite-spec-compliant streams.

[SteveBu]

Same result

Code: Select all

[16:43:19] macgui: Handbrake Version: 20180909191723-09ed0b5-master  (2018091301)
[16:43:19] hb_init: starting libhb thread
[16:43:19] hb_init: starting libhb thread
[16:43:41] macgui: trying to open a folder or file
[16:43:41] macgui: ScanCore scanning titles with a duration of 10 seconds or more
[16:43:41] CPU: Intel(R) Core(TM) i5-4260U CPU @ 1.40GHz
[16:43:41]  - Intel microarchitecture Haswell
[16:43:41]  - logical processor count: 4
[16:43:41] hb_scan: path=/Volumes/BitBucket3/Camera04_20180901092953_20180901100217_69474.mp4, title_index=0
udfread ERROR: ECMA 167 Volume Recognition failed
disc.c:323: failed opening UDF image /Volumes/BitBucket3/Camera04_20180901092953_20180901100217_69474.mp4
disc.c:424: error opening file BDMV/index.bdmv
disc.c:424: error opening file BDMV/BACKUP/index.bdmv
[16:43:41] bd: not a bd - trying as a stream/file instead
libdvdnav: Using dvdnav version 6.0.0
libdvdread: Encrypted DVD support unavailable.
libdvdread:DVDOpenFileUDF:UDFFindFile /VIDEO_TS/VIDEO_TS.IFO failed
libdvdread:DVDOpenFileUDF:UDFFindFile /VIDEO_TS/VIDEO_TS.BUP failed
libdvdread: Can't open file VIDEO_TS.IFO.
libdvdnav: vm: failed to read VIDEO_TS.IFO
[16:43:41] dvd: not a dvd - trying as a stream/file instead
[16:43:41] file is MPEG Program Stream
[16:43:41] Found program stream map
[16:43:41] Found program stream map
[16:43:41] Found program stream map
..........REPEATED LINES REMOVED ..........
[16:43:42] Found program stream map
[16:43:42] Probing 3 unknown streams
[16:43:43] Found the following streams
[16:43:43]     Video Streams : 
[16:43:43]     Audio Streams : 
[16:43:43]     Subtitle Streams : 
[16:43:43]     Other Streams : 
[16:43:43]       0xe0-0x0 type Unknown (0x24)
[16:43:43]       0xc0-0x0 type AC3/IGS (0x91)
[16:43:43]       0xbd-0x0 type Unknown (0x0)
[16:43:43] libhb: scan thread found 0 valid title(s)
[16:43:43] macgui: ScanCore scan done

[Rodeo]

This does look like it might be our bug in the MPEG demuxer. Assuming it is indeed, should be fixable, but could you provide a short sample (just a few seconds) source for us to test? You can use Dropbox or similar.

[SteveBu]

Its a bit more than a few seconds unfortunately but here is a file https://www.dropbox.com/s/14qqipd5cufsz ... 4.mp4?dl=0
Many Thanks
Steve

[SteveBu]

Was the file I sent OK or do you need something smaller ?

[Rodeo]

Oh the file is OK, I haven't had time to look at it yet, sorry.

Thanks for the heads-up!

[JohnAStebbins]

This is another case of our PS demux mistakenly recognizing an mp4 file as PS. I won't have time to do a deeper dive to see if there is anything we can do about it for a couple of weeks. But this should be added to a github issue so it doesn't get overlooked.

[BradleyS]

The file does not open in QuickTime Player on macOS 10.13.6. I'm not seeing anything of substance indicating it's an MP4 container aside from the file extension.

[JohnAStebbins]

The file does not open in QuickTime Player on macOS 10.13.6. I'm not seeing anything of substance indicating it's an MP4 container aside from the file extension.

Ah, I just assumed it was that old problem again. Seems it is a mis-named PS file that we just fail to see any video stream in.

[JohnAStebbins]

After a quick look, our PS demux doesn't support h.265. Some investigation into the proper stream type(s) and iframe recognition will be required to add support.

[Rodeo]

Yeah, stream types should be straightforward, keyframes probably a bit trickier. If I get started, maybe I can figure something out

[BradleyS]

Does this get us closer? https://git.ffmpeg.org/gitweb/ffmpeg.gi ... 4fec656ead

[Rodeo]

Does this get us closer? https://git.ffmpeg.org/gitweb/ffmpeg.gi ... 4fec656ead

Well the stream type was already in the log TBH (just recognized as unknown, but it easily adds up)

[JohnAStebbins]

I've had some time to dig deeper into this.

First, we don't recognize h.265 in PS streams. It's an omission that's pretty easily solved by enabling stream probing for any unidentified stream. Right now we only enable probing for specific streams that have ambiguous stream IDs.

Second, there's a bug in how we decode PS packets. This stream delivers partial frames in each PS packet. It breaks it down so far that the VPS, SPS, and PPS of the hevc stream are each delivered in their own individual PS packets. We are expecting these NAL units to all be in a single PS packet which causes us to fail to initialize the decoder.

I've created a pull request that fixes these problems, but it requires some testing before we can commit it.
https://github.com/HandBrake/HandBrake/pull/1629

[JohnAStebbins]

FYI, we still don't parse keyframes for HEVC in PS (or TS). This causes errors in the log during preview generation since the decoder doesn't receive data starting at a clean keyframe after the seeks during scan. It will result in some unclean preview frames in some cases. For this particular source I got a few unreadable preview frames.