Graceful Crash Recovery

Archive of historical feature requests.
Please use the GitHub link above to report issues.
Forum rules
*******************************
Please be aware we are now using GitHub for issue tracking and feature requests.
- This section of the forum is now closed to new topics.

*******************************
Post Reply
listrophy
Posts: 5
Joined: Fri Feb 22, 2008 8:46 pm

Graceful Crash Recovery

Post by listrophy » Fri Feb 22, 2008 8:54 pm

I've done a cursory check and haven't found this request, so here goes:

Problem:
I've gone through the trouble of slicing up a TV show DVD into episodes and queuing them... something like chapters 1-4 for episode 1, 5-9 for episode 2, etc, etc. I've done this for a whole season of shows... let's say 24 episodes: 24 items in the queue. I hit "Start Queue" and go to bed. I wake up to find that HandBrake has crashed (yes, 0.9.2) and I have to re-enter all the episode info.

Solution:
Graceful Crash Recovery. Before any encoding begins - and perhaps every time an item in the queue finishes - HandBrake takes stock of the situation and writes to disk what still needs to be done. If, in the middle of an encode, HandBrake crashes, Handbrake will still remember everything it was supposed to do.

Please. This has happened to me multiple times now. I'd really like a fix. Thanks!

Cavalicious
Moderator
Posts: 1804
Joined: Mon Mar 26, 2007 12:07 am

Re: Graceful Crash Recovery

Post by Cavalicious » Fri Feb 22, 2008 10:30 pm

This (or something similar) has been brought up by one of the Devs already (Travis I believe). Don't quote me, but I think its in the pipe.

Conner
Posts: 38
Joined: Wed Jan 03, 2007 12:58 am

Re: Graceful Crash Recovery

Post by Conner » Fri Feb 22, 2008 10:39 pm

another alternative would be to be able to save the queue, or 'batch' (what Sorenson Squeeze calls it). That way, if a crash happens, you can open up the saved queue, delete the items that are done, and set it off and running again.

listrophy
Posts: 5
Joined: Fri Feb 22, 2008 8:46 pm

Re: Graceful Crash Recovery

Post by listrophy » Fri Feb 22, 2008 10:45 pm

Conner wrote:another alternative would be to be able to save the queue, or 'batch' (what Sorenson Squeeze calls it). That way, if a crash happens, you can open up the saved queue, delete the items that are done, and set it off and running again.
I like your idea, but why not both?

Your idea really gets interesting when people start posting these "batch" files online. That way, when someone has determined that they've made a particularly good choice on a particular DVD, everyone can share.... but that's another discussion.

Conner
Posts: 38
Joined: Wed Jan 03, 2007 12:58 am

Re: Graceful Crash Recovery

Post by Conner » Sat Feb 23, 2008 3:11 am

shared batch files... interesting, that I hadn't thought of.

jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Re: Graceful Crash Recovery

Post by jbrjake » Sat Feb 23, 2008 4:32 pm

This will come when it's possible. I've suggested it to travistex several times, but he doesn't want it re-scanning the discs when the queue is loaded to set up the job struct, and I don't want the gui messing with the library's private code.

It's not coming in the immediate future.

listrophy
Posts: 5
Joined: Fri Feb 22, 2008 8:46 pm

Re: Graceful Crash Recovery

Post by listrophy » Sun Feb 24, 2008 5:57 am

jbrjake wrote:This will come when it's possible. I've suggested it to travistex several times, but he doesn't want it re-scanning the discs when the queue is loaded to set up the job struct, and I don't want the gui messing with the library's private code.

It's not coming in the immediate future.
All fair and good, I guess. But here's a workaround: an "export" of the queue to CLI script. If I have a queue set up, maybe Handbrake could give me an .sh file that would do the same thing that the GUI is about to do. That way, in the event of a crash, I can remove the successful queue operations and continue with the unfinished items. For example, if I have a queue set up, I'd have a GUI option that gives me a text file containing:

Code: Select all

HandBrakeCLI -i source1 -o destination1
HandBrakeCLI -i source1 -o destination2
HandBrakeCLI -i source2 -o destination3
HandBrakeCLI -i source2 -o destination4
I could copy (or save) this to a .sh file and execute it.

jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Re: Graceful Crash Recovery

Post by jbrjake » Sun Feb 24, 2008 4:53 pm

Considering that the MacGui doesn't interact with the CLI in any way shape or form, this would be a major introduction of new code for translating GUI widgets to the CLI. And then the inverse for re-importing. Code that would constantly break and constantly have to be fixed. Code that really makes more sense in string-and-parsing-friendly scripting languages like ruby and perl.

Doing it the right way makes a lot more sense. Output the user-set parts of the job struct in the key-value form used by the presets.plist, and then manicure.rb can handle the CLI translation as needed.

Your "workaround" is more work.

listrophy
Posts: 5
Joined: Fri Feb 22, 2008 8:46 pm

Re: Graceful Crash Recovery

Post by listrophy » Sun Feb 24, 2008 5:27 pm

jbrjake wrote:Considering that the MacGui doesn't interact with the CLI in any way shape or form...
Ah. I was not aware of that. I just assumed the Gui was a front-end.

Anyway, you folks are the developers, and I'm a user with a feature request. Implementation doesn't technically concern me, so long as the feature's functionality is added. I was just offering ideas on how to go about it. Feel free to ignore that part if you desire. :)

Cavalicious
Moderator
Posts: 1804
Joined: Mon Mar 26, 2007 12:07 am

Re: Graceful Crash Recovery

Post by Cavalicious » Sun Feb 24, 2008 6:31 pm

listrophy wrote: I just assumed the Gui was a front-end.
Blasphemy ! :o

dynaflash
Veteran User
Posts: 3820
Joined: Thu Nov 02, 2006 8:19 pm

Re: Graceful Crash Recovery

Post by dynaflash » Sun Feb 24, 2008 8:17 pm

The MacGui is as much *HandBrake* as the CLI is.

bdfortin
Posts: 17
Joined: Thu May 17, 2007 5:39 am

Re: Graceful Crash Recovery

Post by bdfortin » Wed Feb 27, 2008 6:07 am

I have a request similar to this, so I'll ask it here:

If we can pause encodes, and maybe eventually get crash recovery, would it be possible to save the progress of an encode in case we have to restart or shut down for some reason?

jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Re: Graceful Crash Recovery

Post by jbrjake » Wed Feb 27, 2008 3:10 pm

bdfortin wrote:I have a request similar to this, so I'll ask it here:

If we can pause encodes, and maybe eventually get crash recovery, would it be possible to save the progress of an encode in case we have to restart or shut down for some reason?
No, that's not similar at all. We're talking about preserving the job variables, that's it.

Metasyntactic
Posts: 17
Joined: Wed Feb 13, 2008 6:07 pm

Re: Graceful Crash Recovery

Post by Metasyntactic » Thu Mar 06, 2008 11:54 pm

As a developer, the answer seems fairly simple to me. Just take your in memory data structure and use any of the existing serialization mechanisms ot write it out to a file. In .net (as just one example) you could just create an XmlSerializer which you could save the structure to. Reading the data back in is simply a matter of telling the serializer to load and not save.

I too would welcome such an addition. 0.9.2 has been far less stable for me than 0.9.1. Having a simple save/load button on the queue would be very welcome.

User avatar
s55
HandBrake Team
Posts: 9523
Joined: Sun Dec 24, 2006 1:05 pm

Re: Graceful Crash Recovery

Post by s55 » Thu Mar 06, 2008 11:57 pm

The MacGUI is not written in a .NET langauge ;)

jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Re: Graceful Crash Recovery

Post by jbrjake » Fri Mar 07, 2008 12:22 am

Metasyntactic wrote:As a developer, the answer seems fairly simple to me. Just take your in memory data structure and use any of the existing serialization mechanisms ot write it out to a file.
Yes...that's the idea.
I wrote:Output the user-set parts of the job struct in the key-value form used by the presets.plist
But it's not "fairly simple," because of how the job struct works. Thus all the issues with queuing. Part of the struct is publicly accessible by the MacGui, but parts are private to libhb, and so when importing the exported job list, it'll have to rescan the sources, creating a proper job struct for that title in libhb, before filling in the details of the job from the stuff exported to disk. It'll happen, but, again, it's not going to happen soon since afaik the only person who's shown the slightest interest in pursuing it is travistex, and he seems to be busy with other things.

It's not anywhere near as easy as glomming on a "simple save/load button" -- there are structural changes required to write the data to disk, some tedius philosophical debates about how segmented the lib should be from its interfaces have to occur (again *groan*), and new Obj-C methods (and maybe C functions) are needed to reconstruct the job from the partial data in the exported queue. And if history is any marker, there will be plenty of bugs to work through in this process. Not to mention the usability issues of all of this (when does the rescan take place, all at once when the queue is loaded, or just as the job is called? Both differ from what people are used to in HB.)....

Metasyntactic
Posts: 17
Joined: Wed Feb 13, 2008 6:07 pm

Re: Graceful Crash Recovery

Post by Metasyntactic » Fri Mar 07, 2008 12:31 am

sr55 wrote:The MacGUI is not written in a .NET langauge ;)
Fortunately, similar constructs exist for the Mac as well :-)

Metasyntactic
Posts: 17
Joined: Wed Feb 13, 2008 6:07 pm

Re: Graceful Crash Recovery

Post by Metasyntactic » Fri Mar 07, 2008 12:35 am

jbrjake wrote:
Metasyntactic wrote:As a developer, the answer seems fairly simple to me. Just take your in memory data structure and use any of the existing serialization mechanisms ot write it out to a file.
Yes...that's the idea.
I wrote:Output the user-set parts of the job struct in the key-value form used by the presets.plist
But it's not "fairly simple," because of how the job struct works. Thus all the issues with queuing. Part of the struct is publicly accessible by the MacGui, but parts are private to libhb, and so when importing the exported job list, it'll have to rescan the sources, creating a proper job struct for that title in libhb, before filling in the details of the job from the stuff exported to disk. It'll happen, but, again, it's not going to happen soon since afaik the only person who's shown the slightest interest in pursuing it is travistex, and he seems to be busy with other things.

It's not anywhere near as easy as glomming on a "simple save/load button" -- there are structural changes required to write the data to disk, some tedius philosophical debates about how segmented the lib should be from its interfaces have to occur (again *groan*), and new Obj-C methods (and maybe C functions) are needed to reconstruct the job from the partial data in the exported queue. And if history is any marker, there will be plenty of bugs to work through in this process. Not to mention the usability issues of all of this (when does the rescan take place, all at once when the queue is loaded, or just as the job is called? Both differ from what people are used to in HB.)....
I've worked on a similar system in the past. The Q (even if it does have parts that sit outside the reach of your GUI) is populated by data provided from teh GUI. i.e. when you click 'add to Q' there is code that captures teh state of the GUI and passes that to someone else to process. It is that state that should be saved and restored. The act of loading would in essence take the data from disc, load it into the gui and the perform teh 'add to Q' action. (of course, in practice, you would bypass the actual GUI since it would be irrelevant).

This system will work regardless of if libhb is completely public, completely private, or a mix of the two.

jbrjake
Veteran User
Posts: 4805
Joined: Wed Dec 13, 2006 1:38 am

Re: Graceful Crash Recovery

Post by jbrjake » Fri Mar 07, 2008 12:44 am

Metasyntactic wrote:The Q (even if it does have parts that sit outside the reach of your GUI) is populated by data provided from teh GUI.
Not entirely. Much of the data is provided by the title scan and does not come from the GUI at all, it's all handled in libhb. That's the part that sits outside the reach of the GUI. A job struct is not just an expression of a GUI state.
i.e. when you click 'add to Q' there is code that captures teh state of the GUI and passes that to someone else to process. It is that state that should be saved and restored. The act of loading would in essence take the data from disc, load it into the gui and the perform teh 'add to Q' action. (of course, in practice, you would bypass the actual GUI since it would be irrelevant)
Yes that's what I've been saying: "when importing the exported job list, it'll have to rescan the sources, creating a proper job struct for that title in libhb, before filling in the details of the job from the stuff exported to disk."

listrophy
Posts: 5
Joined: Fri Feb 22, 2008 8:46 pm

Re: Graceful Crash Recovery

Post by listrophy » Fri Mar 14, 2008 11:05 pm

Yes that's what I've been saying: "when importing the exported job list, it'll have to rescan the sources, creating a proper job struct for that title in libhb, before filling in the details of the job from the stuff exported to disk."
If you've ever used MS Word, you'd know that the program isn't always successful in recovering after a crash. People deal with it.

What I'm saying is that performing a fragile operation such as rescanning the sources via libhb is OK. If the rescanning fails, then: oh well, crash recovery failed. But if the rescan does succeed, you have one very very happy user.

In short, a sometimes-working crash recovery feature is better than no crash recovery feature.

dynaflash
Veteran User
Posts: 3820
Joined: Thu Nov 02, 2006 8:19 pm

Re: Graceful Crash Recovery

Post by dynaflash » Sat Mar 15, 2008 4:39 am

right. As jbrjake said, its been bandied about but is not in the top ten of the to do list. Actually not in the top twenty to be totally honest.

Post Reply