Page 1 of 1

Graceful Crash Recovery

Posted: Fri Feb 22, 2008 8:54 pm
by listrophy
I've done a cursory check and haven't found this request, so here goes:

Problem:
I've gone through the trouble of slicing up a TV show DVD into episodes and queuing them... something like chapters 1-4 for episode 1, 5-9 for episode 2, etc, etc. I've done this for a whole season of shows... let's say 24 episodes: 24 items in the queue. I hit "Start Queue" and go to bed. I wake up to find that HandBrake has crashed (yes, 0.9.2) and I have to re-enter all the episode info.

Solution:
Graceful Crash Recovery. Before any encoding begins - and perhaps every time an item in the queue finishes - HandBrake takes stock of the situation and writes to disk what still needs to be done. If, in the middle of an encode, HandBrake crashes, Handbrake will still remember everything it was supposed to do.

Please. This has happened to me multiple times now. I'd really like a fix. Thanks!

Re: Graceful Crash Recovery

Posted: Fri Feb 22, 2008 10:30 pm
by Cavalicious
This (or something similar) has been brought up by one of the Devs already (Travis I believe). Don't quote me, but I think its in the pipe.

Re: Graceful Crash Recovery

Posted: Fri Feb 22, 2008 10:39 pm
by Conner
another alternative would be to be able to save the queue, or 'batch' (what Sorenson Squeeze calls it). That way, if a crash happens, you can open up the saved queue, delete the items that are done, and set it off and running again.

Re: Graceful Crash Recovery

Posted: Fri Feb 22, 2008 10:45 pm
by listrophy
Conner wrote:another alternative would be to be able to save the queue, or 'batch' (what Sorenson Squeeze calls it). That way, if a crash happens, you can open up the saved queue, delete the items that are done, and set it off and running again.
I like your idea, but why not both?

Your idea really gets interesting when people start posting these "batch" files online. That way, when someone has determined that they've made a particularly good choice on a particular DVD, everyone can share.... but that's another discussion.

Re: Graceful Crash Recovery

Posted: Sat Feb 23, 2008 3:11 am
by Conner
shared batch files... interesting, that I hadn't thought of.

Re: Graceful Crash Recovery

Posted: Sat Feb 23, 2008 4:32 pm
by jbrjake
This will come when it's possible. I've suggested it to travistex several times, but he doesn't want it re-scanning the discs when the queue is loaded to set up the job struct, and I don't want the gui messing with the library's private code.

It's not coming in the immediate future.

Re: Graceful Crash Recovery

Posted: Sun Feb 24, 2008 5:57 am
by listrophy
jbrjake wrote:This will come when it's possible. I've suggested it to travistex several times, but he doesn't want it re-scanning the discs when the queue is loaded to set up the job struct, and I don't want the gui messing with the library's private code.

It's not coming in the immediate future.
All fair and good, I guess. But here's a workaround: an "export" of the queue to CLI script. If I have a queue set up, maybe Handbrake could give me an .sh file that would do the same thing that the GUI is about to do. That way, in the event of a crash, I can remove the successful queue operations and continue with the unfinished items. For example, if I have a queue set up, I'd have a GUI option that gives me a text file containing:

Code: Select all

HandBrakeCLI -i source1 -o destination1
HandBrakeCLI -i source1 -o destination2
HandBrakeCLI -i source2 -o destination3
HandBrakeCLI -i source2 -o destination4
I could copy (or save) this to a .sh file and execute it.

Re: Graceful Crash Recovery

Posted: Sun Feb 24, 2008 4:53 pm
by jbrjake
Considering that the MacGui doesn't interact with the CLI in any way shape or form, this would be a major introduction of new code for translating GUI widgets to the CLI. And then the inverse for re-importing. Code that would constantly break and constantly have to be fixed. Code that really makes more sense in string-and-parsing-friendly scripting languages like ruby and perl.

Doing it the right way makes a lot more sense. Output the user-set parts of the job struct in the key-value form used by the presets.plist, and then manicure.rb can handle the CLI translation as needed.

Your "workaround" is more work.

Re: Graceful Crash Recovery

Posted: Sun Feb 24, 2008 5:27 pm
by listrophy
jbrjake wrote:Considering that the MacGui doesn't interact with the CLI in any way shape or form...
Ah. I was not aware of that. I just assumed the Gui was a front-end.

Anyway, you folks are the developers, and I'm a user with a feature request. Implementation doesn't technically concern me, so long as the feature's functionality is added. I was just offering ideas on how to go about it. Feel free to ignore that part if you desire. :)

Re: Graceful Crash Recovery

Posted: Sun Feb 24, 2008 6:31 pm
by Cavalicious
listrophy wrote: I just assumed the Gui was a front-end.
Blasphemy ! :o

Re: Graceful Crash Recovery

Posted: Sun Feb 24, 2008 8:17 pm
by dynaflash
The MacGui is as much *HandBrake* as the CLI is.

Re: Graceful Crash Recovery

Posted: Wed Feb 27, 2008 6:07 am
by bdfortin
I have a request similar to this, so I'll ask it here:

If we can pause encodes, and maybe eventually get crash recovery, would it be possible to save the progress of an encode in case we have to restart or shut down for some reason?

Re: Graceful Crash Recovery

Posted: Wed Feb 27, 2008 3:10 pm
by jbrjake
bdfortin wrote:I have a request similar to this, so I'll ask it here:

If we can pause encodes, and maybe eventually get crash recovery, would it be possible to save the progress of an encode in case we have to restart or shut down for some reason?
No, that's not similar at all. We're talking about preserving the job variables, that's it.

Re: Graceful Crash Recovery

Posted: Thu Mar 06, 2008 11:54 pm
by Metasyntactic
As a developer, the answer seems fairly simple to me. Just take your in memory data structure and use any of the existing serialization mechanisms ot write it out to a file. In .net (as just one example) you could just create an XmlSerializer which you could save the structure to. Reading the data back in is simply a matter of telling the serializer to load and not save.

I too would welcome such an addition. 0.9.2 has been far less stable for me than 0.9.1. Having a simple save/load button on the queue would be very welcome.

Re: Graceful Crash Recovery

Posted: Thu Mar 06, 2008 11:57 pm
by s55
The MacGUI is not written in a .NET langauge ;)

Re: Graceful Crash Recovery

Posted: Fri Mar 07, 2008 12:22 am
by jbrjake
Metasyntactic wrote:As a developer, the answer seems fairly simple to me. Just take your in memory data structure and use any of the existing serialization mechanisms ot write it out to a file.
Yes...that's the idea.
I wrote:Output the user-set parts of the job struct in the key-value form used by the presets.plist
But it's not "fairly simple," because of how the job struct works. Thus all the issues with queuing. Part of the struct is publicly accessible by the MacGui, but parts are private to libhb, and so when importing the exported job list, it'll have to rescan the sources, creating a proper job struct for that title in libhb, before filling in the details of the job from the stuff exported to disk. It'll happen, but, again, it's not going to happen soon since afaik the only person who's shown the slightest interest in pursuing it is travistex, and he seems to be busy with other things.

It's not anywhere near as easy as glomming on a "simple save/load button" -- there are structural changes required to write the data to disk, some tedius philosophical debates about how segmented the lib should be from its interfaces have to occur (again *groan*), and new Obj-C methods (and maybe C functions) are needed to reconstruct the job from the partial data in the exported queue. And if history is any marker, there will be plenty of bugs to work through in this process. Not to mention the usability issues of all of this (when does the rescan take place, all at once when the queue is loaded, or just as the job is called? Both differ from what people are used to in HB.)....

Re: Graceful Crash Recovery

Posted: Fri Mar 07, 2008 12:31 am
by Metasyntactic
sr55 wrote:The MacGUI is not written in a .NET langauge ;)
Fortunately, similar constructs exist for the Mac as well :-)

Re: Graceful Crash Recovery

Posted: Fri Mar 07, 2008 12:35 am
by Metasyntactic
jbrjake wrote:
Metasyntactic wrote:As a developer, the answer seems fairly simple to me. Just take your in memory data structure and use any of the existing serialization mechanisms ot write it out to a file.
Yes...that's the idea.
I wrote:Output the user-set parts of the job struct in the key-value form used by the presets.plist
But it's not "fairly simple," because of how the job struct works. Thus all the issues with queuing. Part of the struct is publicly accessible by the MacGui, but parts are private to libhb, and so when importing the exported job list, it'll have to rescan the sources, creating a proper job struct for that title in libhb, before filling in the details of the job from the stuff exported to disk. It'll happen, but, again, it's not going to happen soon since afaik the only person who's shown the slightest interest in pursuing it is travistex, and he seems to be busy with other things.

It's not anywhere near as easy as glomming on a "simple save/load button" -- there are structural changes required to write the data to disk, some tedius philosophical debates about how segmented the lib should be from its interfaces have to occur (again *groan*), and new Obj-C methods (and maybe C functions) are needed to reconstruct the job from the partial data in the exported queue. And if history is any marker, there will be plenty of bugs to work through in this process. Not to mention the usability issues of all of this (when does the rescan take place, all at once when the queue is loaded, or just as the job is called? Both differ from what people are used to in HB.)....
I've worked on a similar system in the past. The Q (even if it does have parts that sit outside the reach of your GUI) is populated by data provided from teh GUI. i.e. when you click 'add to Q' there is code that captures teh state of the GUI and passes that to someone else to process. It is that state that should be saved and restored. The act of loading would in essence take the data from disc, load it into the gui and the perform teh 'add to Q' action. (of course, in practice, you would bypass the actual GUI since it would be irrelevant).

This system will work regardless of if libhb is completely public, completely private, or a mix of the two.

Re: Graceful Crash Recovery

Posted: Fri Mar 07, 2008 12:44 am
by jbrjake
Metasyntactic wrote:The Q (even if it does have parts that sit outside the reach of your GUI) is populated by data provided from teh GUI.
Not entirely. Much of the data is provided by the title scan and does not come from the GUI at all, it's all handled in libhb. That's the part that sits outside the reach of the GUI. A job struct is not just an expression of a GUI state.
i.e. when you click 'add to Q' there is code that captures teh state of the GUI and passes that to someone else to process. It is that state that should be saved and restored. The act of loading would in essence take the data from disc, load it into the gui and the perform teh 'add to Q' action. (of course, in practice, you would bypass the actual GUI since it would be irrelevant)
Yes that's what I've been saying: "when importing the exported job list, it'll have to rescan the sources, creating a proper job struct for that title in libhb, before filling in the details of the job from the stuff exported to disk."

Re: Graceful Crash Recovery

Posted: Fri Mar 14, 2008 11:05 pm
by listrophy
Yes that's what I've been saying: "when importing the exported job list, it'll have to rescan the sources, creating a proper job struct for that title in libhb, before filling in the details of the job from the stuff exported to disk."
If you've ever used MS Word, you'd know that the program isn't always successful in recovering after a crash. People deal with it.

What I'm saying is that performing a fragile operation such as rescanning the sources via libhb is OK. If the rescanning fails, then: oh well, crash recovery failed. But if the rescan does succeed, you have one very very happy user.

In short, a sometimes-working crash recovery feature is better than no crash recovery feature.

Re: Graceful Crash Recovery

Posted: Sat Mar 15, 2008 4:39 am
by dynaflash
right. As jbrjake said, its been bandied about but is not in the top ten of the to do list. Actually not in the top twenty to be totally honest.