Freezing computer - 100% memory usage

Support for HandBrake on Linux, Solaris, and other Unix-like platforms
Forum rules
An Activity Log is required for support requests. Please read How-to get an activity log? for details on how and why this should be provided.
Post Reply
TrogdorMenoo
Posts: 17
Joined: Thu Nov 08, 2018 11:14 pm

Freezing computer - 100% memory usage

Post by TrogdorMenoo » Sun Dec 30, 2018 4:47 am

Description of problem or question:

Handbrake causes computer to freeze totally.


Steps to reproduce the problem (If Applicable):

Load a Bluray, try to encode.

I suspect that Handbrake is running itself out of memory and running the OS out of memory. Can't it work on much smaller portions of the material without having to load the entire thing into memory? The computer has 6GB of memory and this should be more than enough.

What kind of swapfile settings should I have? I'm a former Windows user, just barely migrated to Linux.

Here's memory usage just after encode is initiated.
https://postimg.cc/dLQvH7n5

Here's memory usage when it freezes the whole computer.

https://postimg.cc/xq3n3GGk

HandBrake version (e.g., 1.0.0):
1.2.0 (x86_64)

Operating system and version (e.g., Ubuntu 16.04 LTS, macOS 10.13 High Sierra, Windows 10 Creators Update):

Linux Mint 19 Cinnamon


HandBrake Activity Log ***required*** (see How-to get an activity log)
I'd love to give you one but the log file is 1.4 GIGABYTES! I'm not sure it will help for this case....

Code: Select all

1.4 GB big!

Woodstock
Veteran User
Posts: 2971
Joined: Tue Aug 27, 2013 6:39 am

Re: Freezing computer - 100% memory usage

Post by Woodstock » Sun Dec 30, 2018 5:03 am

Seeing what handbrake saw is imperative to helping out. "Load a Bluray, try to encode" is something handbrake, by itself, cannot do, so you have multiple programs involved here.

mduell
Veteran User
Posts: 6401
Joined: Sat Apr 21, 2007 8:54 pm

Re: Freezing computer - 100% memory usage

Post by mduell » Sun Dec 30, 2018 6:01 am

An Activity Log is required for support requests. Please read How-to get an activity log? for details on how and why this should be provided.

TrogdorMenoo
Posts: 17
Joined: Thu Nov 08, 2018 11:14 pm

Re: Freezing computer - 100% memory usage

Post by TrogdorMenoo » Sun Dec 30, 2018 2:57 pm

mduell wrote:
Sun Dec 30, 2018 6:01 am
An Activity Log is required for support requests. Please read How-to get an activity log? for details on how and why this should be provided.
I did read it... but what can I do about the 1.4 GB file size!?

TrogdorMenoo
Posts: 17
Joined: Thu Nov 08, 2018 11:14 pm

Re: Freezing computer - 100% memory usage

Post by TrogdorMenoo » Sun Dec 30, 2018 3:04 pm

Woodstock wrote:
Sun Dec 30, 2018 5:03 am
Seeing what handbrake saw is imperative to helping out. "Load a Bluray, try to encode" is something handbrake, by itself, cannot do, so you have multiple programs involved here.
The logfile is extremely big (1.4GB) so I can't really upload it easily. Workflow: Used the "backup" function in MakeMKV which output a BR filesystem. I then I did "Open Source" in Handbrake, and selected the folder. I specified output options including, MKV, .h265, quality 25, added subtitle track, used passthrough audio option for 2 tracks. Then I added to Queue, and executed the Queue. It got as far as "Encoding Job 1 of 2, pass 1 (subtitle scan) of 2, 0.10% (ETA 00h10m20s) 1 encode(s) pending".

rollin_eng
Veteran User
Posts: 3024
Joined: Wed May 04, 2011 11:06 pm

Re: Freezing computer - 100% memory usage

Post by rollin_eng » Sun Dec 30, 2018 3:23 pm

Delete all your logs then start an encode, let it run for a minute then kill Handbrake.

Then upload your log.

Where did you get your copy of Handbrake from?

TrogdorMenoo
Posts: 17
Joined: Thu Nov 08, 2018 11:14 pm

Re: Freezing computer - 100% memory usage

Post by TrogdorMenoo » Sun Dec 30, 2018 7:28 pm

rollin_eng wrote:
Sun Dec 30, 2018 3:23 pm
Delete all your logs then start an encode, let it run for a minute then kill Handbrake.

Then upload your log.

Where did you get your copy of Handbrake from?
Logfile https://stellar7.space/wp-content/uploa ... og.8427.7z:

I retrieved it from the Linux Mint Cinnamon repository... the one GUI package installer that is part of Mint.

mduell
Veteran User
Posts: 6401
Joined: Sat Apr 21, 2007 8:54 pm

Re: Freezing computer - 100% memory usage

Post by mduell » Sun Dec 30, 2018 8:02 pm

You should contact Mint for support with their build of HB; Linux distros like to break the software for ideological/political reasons. This forum is for official builds, or builds from HB source.

User avatar
s55
HandBrake Team
Posts: 9376
Joined: Sun Dec 24, 2006 1:05 pm

Re: Freezing computer - 100% memory usage

Post by s55 » Sun Dec 30, 2018 8:05 pm

I'd recommend you look at giving our flatpak build a go. (available on the download page and via flathub)

https://handbrake.fr/docs/en/1.2.0/get- ... brake.html

Distro packages are 9 of of 10 times, deliberately broken by their maintainers for the reasons mduell stated. Which leaves you the end user with software that doesn't work. :/ Go figure but not a lot we can do to stop them. Asking nicely doesn't work.

Rule out any 3rd party issues before folks waste a lot of time going down the rabbit hole again :roll:

rollin_eng
Veteran User
Posts: 3024
Joined: Wed May 04, 2011 11:06 pm

Re: Freezing computer - 100% memory usage

Post by rollin_eng » Sun Dec 30, 2018 8:06 pm

Get HB from here:

https://handbrake.fr/downloads.php

And see if you get the same issues.

User avatar
JohnAStebbins
HandBrake Team
Posts: 5315
Joined: Sat Feb 09, 2008 7:21 pm

Re: Freezing computer - 100% memory usage

Post by JohnAStebbins » Mon Dec 31, 2018 3:28 pm

Also if the log file was 1.4G, then it's almost certainly repeating some error over and over again. You could probably look at it and post a few lines to show us the errors.

But if you literally did what you said in your original post "Load a Bluray, try to encode" and nothing else, then you are trying to decode encrypted video and are getting decode errors. HandBrake is not a ripper. You must decrypt any encrypted video before feeding it to HandBrake.

TrogdorMenoo
Posts: 17
Joined: Thu Nov 08, 2018 11:14 pm

Re: Freezing computer - 100% memory usage

Post by TrogdorMenoo » Tue Jan 01, 2019 3:19 am

JohnAStebbins wrote:
Mon Dec 31, 2018 3:28 pm
Also if the log file was 1.4G, then it's almost certainly repeating some error over and over again. You could probably look at it and post a few lines to show us the errors.

But if you literally did what you said in your original post "Load a Bluray, try to encode" and nothing else, then you are trying to decode encrypted video and are getting decode errors. HandBrake is not a ripper. You must decrypt any encrypted video before feeding it to HandBrake.
Please see above for log file. Indeed there were lots of errors. Nevertheless, the problem still occurs on discs that did not generate errors. It seems the program is using an inordinate amount of memory.

TrogdorMenoo
Posts: 17
Joined: Thu Nov 08, 2018 11:14 pm

Re: Freezing computer - 100% memory usage

Post by TrogdorMenoo » Tue Jan 01, 2019 3:37 am

s55 wrote:
Sun Dec 30, 2018 8:05 pm
I'd recommend you look at giving our flatpak build a go. (available on the download page and via flathub)

https://handbrake.fr/docs/en/1.2.0/get- ... brake.html

Distro packages are 9 of of 10 times, deliberately broken by their maintainers for the reasons mduell stated. Which leaves you the end user with software that doesn't work. :/ Go figure but not a lot we can do to stop them. Asking nicely doesn't work.

Rule out any 3rd party issues before folks waste a lot of time going down the rabbit hole again :roll:
I just tried the flatpack and ran into a completely different problem. In whatever version I had installed before (GTK something), I could select the parent folder of the BR disc rip and it would analyze the BR structure. Now when I select the parent structure, it appears to do nothing. "No Title Found" remains visible and it doesn't appear to have done anything when I try "open source". No matter what I try, I cannot get it to open source anymore.

rollin_eng
Veteran User
Posts: 3024
Joined: Wed May 04, 2011 11:06 pm

Re: Freezing computer - 100% memory usage

Post by rollin_eng » Tue Jan 01, 2019 8:31 am

Could you please post your HB logs for this new issue, instructions can be found here:

https://handbrake.fr/docs/en/latest/hel ... y-log.html

TrogdorMenoo
Posts: 17
Joined: Thu Nov 08, 2018 11:14 pm

Re: Freezing computer - 100% memory usage

Post by TrogdorMenoo » Tue Jan 01, 2019 8:34 pm

rollin_eng wrote:
Tue Jan 01, 2019 8:31 am
Could you please post your HB logs for this new issue, instructions can be found here:

https://handbrake.fr/docs/en/latest/hel ... y-log.html
Here's the log from after being unable to open any BR rip, when it previously worked. Note: The Windows version does it open it.

Code: Select all

[21:29:08] gtkgui: HandBrake 1.2.0 (2018122200) - Linux x86_64 - https://handbrake.fr
Cannot load libnvidia-encode.so.1
Cannot load libnvidia-encode.so.1
Cannot load libnvidia-encode.so.1
Cannot load libnvidia-encode.so.1
[21:29:08] hb_init: starting libhb thread
[21:29:08] hb_init: starting libhb thread
[21:29:08] hb_init: starting libhb thread
[21:29:49] CPU: Intel(R) Core(TM) i7 CPU       X 980  @ 3.33GHz
[21:29:49]  - logical processor count: 12
[21:29:49] hb_scan: path=/run/user/1000/gvfs/smb-share:server=flower,share=_rips/MakeMKV/backup/PeaceCorps2017, title_index=0
disc.c:323: failed opening UDF image /run/user/1000/gvfs/smb-share:server=flower,share=_rips/MakeMKV/backup/PeaceCorps2017
disc.c:424: error opening file BDMV/index.bdmv
disc.c:424: error opening file BDMV/BACKUP/index.bdmv
[21:29:49] bd: not a bd - trying as a stream/file instead
libdvdnav: Using dvdnav version 6.0.0
libdvdread: Encrypted DVD support unavailable.
libdvdread: Could not open input: No such file or directory
libdvdread: Can't open /run/user/1000/gvfs/smb-share:server=flower,share=_rips/MakeMKV/backup/PeaceCorps2017 for reading
libdvdnav: vm: failed to open/read the DVD
[21:29:49] dvd: not a dvd - trying as a stream/file instead
[21:29:49] hb_stream_open: open /run/user/1000/gvfs/smb-share:server=flower,share=_rips/MakeMKV/backup/PeaceCorps2017 failed
[21:29:49] scan: unrecognized file type
[21:29:49] libhb: scan thread found 0 valid title(s)
[21:31:19] CPU: Intel(R) Core(TM) i7 CPU       X 980  @ 3.33GHz
[21:31:19]  - logical processor count: 12
[21:31:19] hb_scan: path=/run/user/1000/gvfs/smb-share:server=flower,share=_rips/MakeMKV/backup/PeaceCorps2017/BDMV, title_index=0
disc.c:323: failed opening UDF image /run/user/1000/gvfs/smb-share:server=flower,share=_rips/MakeMKV/backup/PeaceCorps2017/BDMV
disc.c:424: error opening file BDMV/index.bdmv
disc.c:424: error opening file BDMV/BACKUP/index.bdmv
[21:31:19] bd: not a bd - trying as a stream/file instead
libdvdnav: Using dvdnav version 6.0.0
libdvdread: Encrypted DVD support unavailable.
libdvdread: Could not open input: No such file or directory
libdvdread: Can't open /run/user/1000/gvfs/smb-share:server=flower,share=_rips/MakeMKV/backup/PeaceCorps2017/BDMV for reading
libdvdnav: vm: failed to open/read the DVD
[21:31:19] dvd: not a dvd - trying as a stream/file instead
[21:31:19] hb_stream_open: open /run/user/1000/gvfs/smb-share:server=flower,share=_rips/MakeMKV/backup/PeaceCorps2017/BDMV failed
[21:31:19] scan: unrecognized file type
[21:31:19] libhb: scan thread found 0 valid title(s)
[21:31:57] CPU: Intel(R) Core(TM) i7 CPU       X 980  @ 3.33GHz
[21:31:57]  - logical processor count: 12
[21:31:57] hb_scan: path=/run/user/1000/gvfs/smb-share:server=flower,share=_rips/MakeMKV/backup/PeaceCorps2017, title_index=0
disc.c:323: failed opening UDF image /run/user/1000/gvfs/smb-share:server=flower,share=_rips/MakeMKV/backup/PeaceCorps2017
disc.c:424: error opening file BDMV/index.bdmv
disc.c:424: error opening file BDMV/BACKUP/index.bdmv
[21:31:57] bd: not a bd - trying as a stream/file instead
libdvdnav: Using dvdnav version 6.0.0
libdvdread: Encrypted DVD support unavailable.
libdvdread: Could not open input: No such file or directory
libdvdread: Can't open /run/user/1000/gvfs/smb-share:server=flower,share=_rips/MakeMKV/backup/PeaceCorps2017 for reading
libdvdnav: vm: failed to open/read the DVD
[21:31:57] dvd: not a dvd - trying as a stream/file instead
[21:31:57] hb_stream_open: open /run/user/1000/gvfs/smb-share:server=flower,share=_rips/MakeMKV/backup/PeaceCorps2017 failed
[21:31:57] scan: unrecognized file type
[21:31:57] libhb: scan thread found 0 valid title(s)
[21:33:29] CPU: Intel(R) Core(TM) i7 CPU       X 980  @ 3.33GHz
[21:33:29]  - logical processor count: 12
[21:33:29] hb_scan: path=/run/user/1000/gvfs/smb-share:server=flower,share=_rips/MakeMKV/backup/PeaceCorps2017, title_index=0
disc.c:323: failed opening UDF image /run/user/1000/gvfs/smb-share:server=flower,share=_rips/MakeMKV/backup/PeaceCorps2017
disc.c:424: error opening file BDMV/index.bdmv
disc.c:424: error opening file BDMV/BACKUP/index.bdmv
[21:33:29] bd: not a bd - trying as a stream/file instead
libdvdnav: Using dvdnav version 6.0.0
libdvdread: Encrypted DVD support unavailable.
libdvdread: Could not open input: No such file or directory
libdvdread: Can't open /run/user/1000/gvfs/smb-share:server=flower,share=_rips/MakeMKV/backup/PeaceCorps2017 for reading
libdvdnav: vm: failed to open/read the DVD
[21:33:29] dvd: not a dvd - trying as a stream/file instead
[21:33:29] hb_stream_open: open /run/user/1000/gvfs/smb-share:server=flower,share=_rips/MakeMKV/backup/PeaceCorps2017 failed
[21:33:29] scan: unrecognized file type
[21:33:29] libhb: scan thread found 0 valid title(s)
[21:36:33] CPU: Intel(R) Core(TM) i7 CPU       X 980  @ 3.33GHz
[21:36:33]  - logical processor count: 12
[21:36:33] hb_scan: path=/run/user/1000/gvfs/smb-share:server=flower,share=_rips/MakeMKV/backup/PeaceCorps2017/BDMV, title_index=0
disc.c:323: failed opening UDF image /run/user/1000/gvfs/smb-share:server=flower,share=_rips/MakeMKV/backup/PeaceCorps2017/BDMV
disc.c:424: error opening file BDMV/index.bdmv
disc.c:424: error opening file BDMV/BACKUP/index.bdmv
[21:36:33] bd: not a bd - trying as a stream/file instead
libdvdnav: Using dvdnav version 6.0.0
libdvdread: Encrypted DVD support unavailable.
libdvdread: Could not open input: No such file or directory
libdvdread: Can't open /run/user/1000/gvfs/smb-share:server=flower,share=_rips/MakeMKV/backup/PeaceCorps2017/BDMV for reading
libdvdnav: vm: failed to open/read the DVD
[21:36:33] dvd: not a dvd - trying as a stream/file instead
[21:36:33] hb_stream_open: open /run/user/1000/gvfs/smb-share:server=flower,share=_rips/MakeMKV/backup/PeaceCorps2017/BDMV failed
[21:36:33] scan: unrecognized file type
[21:36:33] libhb: scan thread found 0 valid title(s)


User avatar
JohnAStebbins
HandBrake Team
Posts: 5315
Joined: Sat Feb 09, 2008 7:21 pm

Re: Freezing computer - 100% memory usage

Post by JohnAStebbins » Tue Jan 01, 2019 11:02 pm

Looks like a flatpak sandbox problem with network shares. This was working not too long ago. I tested it myself. So something has changed recently in flatpak's sandbox. It should work with the files local. I'll keep digging to see if I can figure out what they changed and if there is some new permission necessary.

TrogdorMenoo
Posts: 17
Joined: Thu Nov 08, 2018 11:14 pm

Re: Freezing computer - 100% memory usage

Post by TrogdorMenoo » Tue Jan 01, 2019 11:31 pm

JohnAStebbins wrote:
Tue Jan 01, 2019 11:02 pm
Looks like a flatpak sandbox problem with network shares. This was working not too long ago. I tested it myself. So something has changed recently in flatpak's sandbox. It should work with the files local. I'll keep digging to see if I can figure out what they changed and if there is some new permission necessary.
Can you please post here whenever the network share issue has been resolved? I'll try to get a log once it can open the source from the network share.

TrogdorMenoo
Posts: 17
Joined: Thu Nov 08, 2018 11:14 pm

Re: Freezing computer - 100% memory usage

Post by TrogdorMenoo » Fri Jan 04, 2019 5:02 am

JohnAStebbins wrote:
Tue Jan 01, 2019 11:02 pm
Looks like a flatpak sandbox problem with network shares. This was working not too long ago. I tested it myself. So something has changed recently in flatpak's sandbox. It should work with the files local. I'll keep digging to see if I can figure out what they changed and if there is some new permission necessary.
I tried doing something local and I noticed that the computer freezes because it runs out of memory. I was able to encode 1 video and attempt a second one using the flatpack build. Initially I could see memory usage creeping up as the BR is loaded and it rose very slowly during the encode. This was using .h264. Then, I did the same encode again but using .h265 with a slow encode quality setting. Memory usage never really decreased from the first encode to the second. During the second encode, memory usage kept increasing over the course of an hour or more before it finally got to 99%+ usage. At that point everything froze.

About 24+ hrs later, the computer was accessible again. Everything was exactly in the same state as before HB froze the computer. It appears that the computer killed HB and everything else was back to normal.

I would think that 6GB of RAM would be more than adequate for a Linux Mint computer that does nothing more than encode. I don't really have anything else running.

Here are the 3 log files: https://we.tl/t-Q7q8DHIvdU

User avatar
JohnAStebbins
HandBrake Team
Posts: 5315
Joined: Sat Feb 09, 2008 7:21 pm

Re: Freezing computer - 100% memory usage

Post by JohnAStebbins » Sat Jan 05, 2019 3:54 pm

As far as I can tell, there must have been an unintended access hole in the flatpak sandbox when I last tested access to network file systems. To enable, you have to add a new access permission to the HandBrake flatpak. I'll be adding it to the nighly builds and next release, but you can also grant access locally with:

Code: Select all

sudo flatpak override --filesystem=xdg-run/gvfs fr.handbrake.ghb
That assumes you installed fr.handbrake.ghb *without* the '--user' option. If you used the '--user' option during install then the command would be

Code: Select all

flatpak override --user --filesystem=xdg-run/gvfs fr.handbrake.ghb
About you memory usage problem, I'm not sure what is happening there. When I encode an HD file with the same settings, HandBrake is only using around 1GB. Can you do the following and show me the output?
  1. Run HandBrake GUI

    Code: Select all

    flatpak run fr.handbrake.ghb
    
  2. In another terminal window, monitor the memory usage of the running HandBrake process 'ghb'

    Code: Select all

    while true; do ps -o rss -C ghb; sleep 600; done
    
    will produce something like

    Code: Select all

      RSS
    116572
      RSS
    1033224
      RSS
    1069344
      RSS
    1080992
      RSS
    298448
      RSS
    298448
    
    This is the output during a short encode I did. The memory usage started at about 116MB, grew to around 1GB then dropped back to around 300MB when the encode was complete. My overnight encode using x265 looked about the same with max memory usage reaching around 1GB.

Post Reply