Can't Read Large MagicYUV 2.0rc1 Files

Archive of historical bug reports.
Please use the GitHub link above to report issues.
Forum rules
*******************************
Please be aware we are now using GitHub for issue tracking and feature requests.
- This section of the forum is now closed to new topics.

*******************************
Post Reply
Parallax Abstraction
Posts: 23
Joined: Thu Nov 12, 2015 4:41 am

Can't Read Large MagicYUV 2.0rc1 Files

Post by Parallax Abstraction »

Please detail your question or problem in as much detail as possible:
I run a gaming YouTube channel and recently started capturing using the MagicYUV 2.0rc1 lossless codec as I find it performs much better than Lagarith. I use HandBrake to transcode the lossless footage to a much smaller file for archiving after I've uploaded my project to YouTube. It does a great job encoding MagicYUV files until they become a certain size, then it says it can't read them when I try to load them in. Right before the file in question here, I converted a 57GB file without issue. This file is 126GB and says no valid source or titles are found. I was able to open and edit this file in Vegas Pro 13 without issue, as well as play it in PotPlayer.

Tell us about your environment. What version of HandBrake? What version of Windows are you running. etc.
HandBrake 1.0.1 (problem started with 1.0.0)
Windows 10 Pro AU x64
Intel Core i7-6700K @ 4.6Ghz (not using Quick Sync)

If there was any exception or error displayed, please copy it and paste it here:
Upon opening the file, it says "No valid source or titles found."

Please include the scan or encode log:

Code: Select all

HandBrake 1.0.1 (2016122900) - 64bit
OS: Microsoft Windows NT 10.0.14393.0 - 64bit
CPU: Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz
Ram: 16347 MB, 
GPU Information:
  NVIDIA GeForce GTX 1080 - 21.21.13.7648
Screen: 1920x1080
Temp Dir: C:\Users\Parallax Abstraction\AppData\Local\Temp\
Install Dir: C:\Program Files\Handbrake
Data Dir: C:\Users\Parallax Abstraction\AppData\Roaming\HandBrake Team\HandBrake\1.0.1.0

-------------------------------------------

[23:50:33] hb_init: starting libhb thread
[23:50:33] hb_init: starting libhb thread
[23:50:33] hb_init: starting libhb thread
[23:50:33] hb_init: starting libhb thread
[23:50:37] hb_init: starting libhb thread

 # Starting Scan ...

[23:50:37] CPU: Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz
[23:50:37]  - Intel microarchitecture Skylake
[23:50:37]  - logical processor count: 8
[23:50:37] OpenCL device #1: NVIDIA Corporation GeForce GTX 1080
[23:50:37]  - OpenCL version: 1.2 CUDA
[23:50:37]  - driver version: 376.48
[23:50:37]  - device type:    GPU
[23:50:37]  - supported:      no
[23:50:37] Intel Quick Sync Video support: no
[23:50:37] hb_scan: path=X:\Capture\Book of Demons.avi, title_index=0
udfread ERROR: ECMA 167 Volume Recognition failed
src/libbluray/disc/disc.c:274: failed opening UDF image X:\Capture\Book of Demons.avi
src/libbluray/disc/disc.c:352: error opening file BDMV\index.bdmv
src/libbluray/disc/disc.c:352: error opening file BDMV\BACKUP\index.bdmv
[23:50:37] bd: not a bd - trying as a stream/file instead
libdvdnav: Using dvdnav version 5.0.1
libdvdread: Encrypted DVD support unavailable.
libdvdread:DVDOpenFileUDF:UDFFindFile /VIDEO_TS/VIDEO_TS.IFO failed
libdvdread:DVDOpenFileUDF:UDFFindFile /VIDEO_TS/VIDEO_TS.BUP failed
libdvdread: Can't open file VIDEO_TS.IFO.
libdvdnav: vm: failed to read VIDEO_TS.IFO
[23:50:37] dvd: not a dvd - trying as a stream/file instead
Input #0, avi, from 'X:\Capture\Book of Demons.avi':
  Metadata:
    encoder         : DxtoryCore ver2.0.0.141
    ISRC            : Video:MagicYUV - YUV 4:2:0 Audio0:Speakers (Realtek High Definition Audio) Audio1:VoiceMeeter Output (VB-Audio VoiceMeeter VAIO)
  Duration: 00:41:02.36, start: 0.000000, bitrate: 441837 kb/s
    Stream #0:0: Video: magicyuv [M8Y0 / 0x3059384D]
      yuv420p, 1920x1080
      60 fps, 60 tbn
    Stream #0:1: Audio: pcm_s16le [[1][0][0][0] / 0x0001]
      48000 Hz, 2 channels, s16, 1536 kb/s
    Stream #0:2: Audio: pcm_s16le [[1][0][0][0] / 0x0001]
      48000 Hz, 2 channels, s16, 1536 kb/s
[23:50:37] scan: decoding previews for title 1
[23:50:37] scan: audio 0x1: pcm_s16le, rate=48000Hz, bitrate=1536000 Unknown (pcm_s16le) (2.0 ch)
[23:50:37] scan: audio 0x2: pcm_s16le, rate=48000Hz, bitrate=1536000 Unknown (pcm_s16le) (2.0 ch)
[23:50:37] Warning: Could not read data for preview 1, skipped
[23:50:37] scan: decoding previews for title 1
[23:50:37] Warning: Could not read data for preview 1, skipped
[23:50:38] libhb: scan thread found 0 valid title(s)

 # Scan Finished ...

Thanks all. If any more information is needed, please ask and I'll provide what I can.
User avatar
s55
HandBrake Team
Posts: 10350
Joined: Sun Dec 24, 2006 1:05 pm

Re: Can't Read Large MagicYUV 2.0rc1 Files

Post by s55 »

This isn't the first post I've seen with >2GB files causing some problems. I'll look into it further.

Does it process OK with the previous 0.10.5 release? (Old versions are available from the download page. There is a link on that page to old releases)
Parallax Abstraction
Posts: 23
Joined: Thu Nov 12, 2015 4:41 am

Re: Can't Read Large MagicYUV 2.0rc1 Files

Post by Parallax Abstraction »

I'm not sure what the ceiling is that causes this issue but it seems like it's over 100GB for me because stuff below that seems to work well. I'm not sure if 100GB is the actual ceiling but that's been my experience.

I will try the older release when I get home tonight and let you know the results. Thanks for responding so fast!
Parallax Abstraction
Posts: 23
Joined: Thu Nov 12, 2015 4:41 am

Re: Can't Read Large MagicYUV 2.0rc1 Files

Post by Parallax Abstraction »

So, I download 0.10.5 as you suggested and tried it. It still won't read the file but I got a different result. If I try to load the 100GB+ file, I don't get an error message, it just returns to the main application window and no options are available. So it still fails to load in the file, it just doesn't explain it this time.

Maybe something with that particular file just messed up. I'll do another lengthy test recording today to generate a similar size file and see if that makes any difference. I'd offer up the test file for your use but well, getting 100GB to you might not be super practical. :)
Parallax Abstraction
Posts: 23
Joined: Thu Nov 12, 2015 4:41 am

Re: Can't Read Large MagicYUV 2.0rc1 Files

Post by Parallax Abstraction »

So this is really a head scratcher. I recorded another long test clip tonight. This one was almost twice the size at 197GB and Handbrake reads it no problem. I upgraded to 1.02 and is still won't read my original 100GB clip, which still opens and works fine in Vegas Pro or any player I've tried. Granted, I recorded this clip out of a different game but I did do it with the same capture software (Dxtory) and the same codec (MagicYUV 4:2:0). Kind of at a loss as to why this is the case but perhaps there's just something with the way that one file was captured. If it happens with future videos I do, I'll post in here again.
Parallax Abstraction
Posts: 23
Joined: Thu Nov 12, 2015 4:41 am

Re: Can't Read Large MagicYUV 2.0rc1 Files

Post by Parallax Abstraction »

So I have some more information on this, which came from the MagicYUV forums and which I confirmed myself. It's not just the codec itself, it's what application the footage is recorded in as well. Anything recorded using MagicYUV in Dxtory isn't readable by Handbrake. Anything recorded in Bandicam becomes unreadable sometimes if it goes over 100GB, but not all the time. This is very strange to me as this is the only codec I've seen that happen with. I used to record with Lagarith in both of these applications and never had an issue. I don't know if this provides any assistance to you. Or should I take it up with the authors of the recording applications? Thanks!
GenericPerson#375
Posts: 2
Joined: Wed Feb 15, 2017 1:48 am

Re: Can't Read Large MagicYUV 2.0rc1 Files

Post by GenericPerson#375 »

Parallax Abstraction wrote: Thu Feb 16, 2017 4:36 pm So I have some more information on this, which came from the MagicYUV forums and which I confirmed myself. It's not just the codec itself, it's what application the footage is recorded in as well. Anything recorded using MagicYUV in Dxtory isn't readable by Handbrake. Anything recorded in Bandicam becomes unreadable sometimes if it goes over 100GB, but not all the time. This is very strange to me as this is the only codec I've seen that happen with. I used to record with Lagarith in both of these applications and never had an issue. I don't know if this provides any assistance to you. Or should I take it up with the authors of the recording applications? Thanks!
If you have recorded the MAGY file with Dxtory and/or can't put it in to Handbrake, convert it to the exact same MAGY type with Adobe Media Encoder/Some other program that can access VFW codecs. Even if it's 200GB+, Handbrake should be able to read it properly.
Parallax Abstraction
Posts: 23
Joined: Thu Nov 12, 2015 4:41 am

Re: Can't Read Large MagicYUV 2.0rc1 Files

Post by Parallax Abstraction »

Someone on the MagicYUV forums mentioned this too. I'm definitely not prepared to buy Adobe Media Encoder just to convert files into something that works with Handbrake. :) However, the creator of MagicYUV has also been working with me on the issue. He's asked me to send him a 47GB recording I made using Dxtory that won't open in Handbrake. We figured out a way to get that large of a file to him and he's going to analyse it and see if he can find anything out.
Deleted User 11865

Re: Can't Read Large MagicYUV 2.0rc1 Files

Post by Deleted User 11865 »

Cool. If he thinks it's a bug in libavformat or libavcodec, at least he should be able to explain where the issue lies.
cehoyos
Posts: 8
Joined: Sat Feb 04, 2017 9:55 am

Re: Can't Read Large MagicYUV 2.0rc1 Files

Post by cehoyos »

I suspect this is not reproducible with FFmpeg / an avconv regression.
Parallax Abstraction
Posts: 23
Joined: Thu Nov 12, 2015 4:41 am

Re: Can't Read Large MagicYUV 2.0rc1 Files

Post by Parallax Abstraction »

So, I sent the file I created yesterday to the developer of MagicYUV. It's a 47GB MagicYUV file, created in Dxtory. I didn't mention before but my files also have two audio tracks, one for game audio and one for my commentary. Here are the responses I got from him when he tried to use the file with Handbrake:

"OK, so to the looks of it, the AVI itself is fine. VirtualDub, MPC-BE, even ffmpeg command line read it just fine, so it's some issue with Handbrake specifically.
If you look at the "Activity Log" in handbrake, it even recognizes the file and video as magicyuv correctly, but still croaks for some reason."

"If I remux with VirtualDub in Direct Stream Copy mode (without re-encoding the video), HB reads it fine (though it'll have only 1 audio channel of the 2). So I think it's a muxing issue, though it's unclear where it fails, but definitely not at the video decoding phase."

Hopefully this provides some measure of assistance in nailing down the source of the issue. The developer said he suspects the Handbrake developers might need to see the file as well. It's still on my Mega drive and I can provide a share link if requested. Mega is also kind of a pain with large single files so I'm also happy to setup a private FTP on my home server, though I only have 10Mbps upload so it will take 12+ hours to download the file that way. Thanks all!
Deleted User 13735

Re: Can't Read Large MagicYUV 2.0rc1 Files

Post by Deleted User 13735 »

Please set me straight; I wasn't aware that MagicYUV is officially supported in libav?
We tested it and reported on the Vegas forums sometime back that it is not supported in Handbrake.

So is it supported of not supported?
Yes or No would be fantastic!
Parallax Abstraction
Posts: 23
Joined: Thu Nov 12, 2015 4:41 am

Re: Can't Read Large MagicYUV 2.0rc1 Files

Post by Parallax Abstraction »

So, it wasn't supported before. I don't know it it either became supported when MagicYUV came out or in a more recent release of Handbrake. When I first tried it back in MagicYUV v1.x, it wasn't supported at all. Now it works, it just has this issue.

Rodeo, the MagicYUV dev suggested I sent you the same file I sent him to analyse. Are you interested in that? If so, I can either send you a link to my Mega drive to download it or if that doesn't work, I'm happy to temporarily setup an FileZilla FTP on my home server to get it to you as well. Anything I can do to help figure this out.

Thanks!
Parallax Abstraction
Posts: 23
Joined: Thu Nov 12, 2015 4:41 am

Re: Can't Read Large MagicYUV 2.0rc1 Files

Post by Parallax Abstraction »

Saw 1.0.3 and got excited but sadly, problem persists. :(

Rodeo, are you interested in checking out a file that won't open in Handbrake to see if you can glean anything from it? The MagicYUV developer seems to think that's the best next step.
Deleted User 13735

Re: Can't Read Large MagicYUV 2.0rc1 Files

Post by Deleted User 13735 »

Er, I don't think the MagicYUV developer understands how busy Rodeo and the other volunteer Handbrake developers are with their own project :shock:

I suggest you try submitting a ticket and the sample file with the folks over at LIBAV. It looks like there is already interest at ffmpeg, and libav won't be far behind esp with expressions of interest from you and a few others i know.

When more robust support for Magic in libav, I can promise that Handbrake will be early adopters; in the meantime, high quality codecs such as UT and Sony YUV have been working just fine with Handbrake...
Parallax Abstraction
Posts: 23
Joined: Thu Nov 12, 2015 4:41 am

Re: Can't Read Large MagicYUV 2.0rc1 Files

Post by Parallax Abstraction »

I understand they're busy but they seemed very interested in figuring this out, which is why I offered to send them the file to check. I didn't expect an instant resolution or anything but it seemed like the next logical step to solving this.

As for other codecs, UT doesn't work properly with my capture tools (the resulting footage is always black & white when I try) and Sony YUV isn't a codec I can use with capture tools at all.
Deleted User 11865

Re: Can't Read Large MagicYUV 2.0rc1 Files

Post by Deleted User 11865 »

I can check it out. You can send me a link via private message.
Parallax Abstraction
Posts: 23
Joined: Thu Nov 12, 2015 4:41 am

Re: Can't Read Large MagicYUV 2.0rc1 Files

Post by Parallax Abstraction »

Awesome, thanks! So I'll send you the Mega link but the file is so large, it may be difficult to download. If so, I'm happy to setup a temporarily FileZilla FTP on my home server to get you the file. I'll send you the link shortly. Thanks again!
Deleted User 13735

Re: Can't Read Large MagicYUV 2.0rc1 Files

Post by Deleted User 13735 »

Fwiw, MagicYUV works perfectly as a system codec with Vegas ...
EXCEPT when the files come from one of the screencap apps like DxTory etc.
Lots of misflagged misreported headers it seems...

I haven't heard your particular complaint with UT before, however...

IF your >4GB files were saved to any drive formatted FAT32, of course there is no way they will ever open. It's a fairly regular problem with capture apps. Do you know if your drive is fat or ntfs?

One other thing I can mention is that your pcm audio appears as 2 ch discrete mono rather than a normal l/r pair. Exactly the same thing that caused problems with Sony MXF in Handbrake.
ALbino
Posts: 19
Joined: Fri Sep 20, 2013 7:26 pm

Re: Can't Read Large MagicYUV 2.0rc1 Files

Post by ALbino »

Release Candidate 2 was released a week or so ago for MagicYUV, and in the notes it says:
2.0.0rc2
Released: 2017.03.21.

This is a bugfix release fixing issues with some recorder programs and BattlEye.

FIX: Core: Added decoder checks against invalid data which caused the codec to crash. This happened if the decoder received garbage data when opening invalid AVI files produced by some recorder programs (most notably Bandicam).
https://www.magicyuv.com/change-log/

I wonder if that's related to your issue, or if potentially solves it.

FWIW, I've completely replaced Lagarith with MagicYUV in my workflow, as it's faster and uses less space. I don't do any game recording or anything, so I can't speak to whether it works with those kinds of programs or not.
Parallax Abstraction
Posts: 23
Joined: Thu Nov 12, 2015 4:41 am

Re: Can't Read Large MagicYUV 2.0rc1 Files

Post by Parallax Abstraction »

Unfortunately, rc2 hasn't resolved the issue, at least for me. One of the Handbrake developers said he has a theory on what the issue is and it might be something they can fix, though it might take a while.
ALbino
Posts: 19
Joined: Fri Sep 20, 2013 7:26 pm

Re: Can't Read Large MagicYUV 2.0rc1 Files

Post by ALbino »

Sorry to hear that. I hope it works out :)
Parallax Abstraction
Posts: 23
Joined: Thu Nov 12, 2015 4:41 am

Re: Can't Read Large MagicYUV 2.0rc1 Files

Post by Parallax Abstraction »

At least for now, I can use Lagarith or I can remux the footage before I load it into Handbrake. Extra time but not the end of the world. Hopefully the Handbrake ninjas can work their magic (no pun intended, OK maybe) soon. :)
Post Reply