U-Net & Handbrake?

[KenithO]

Have some MP4 transfers of old B&W Films that would like to restore (dust and scratch removal plus image stabilization and sharpening) before using HandBrake to convert them to H.265.

Recently watched "Research at NVIDIA: AI Can Now Fix Your Grainy Photos by Only Looking at Grainy Photos" https://www.youtube.com/watch?v=pp7HdI0-MIo and was impressed!

Did some searching and was able to get PDF "Noise2Noise: Learning Image Restoration without Clean Data" https://arxiv.org/pdf/1803.04189.pdf which mentions "Our baseline is a recent state-of-the-art method ”RED30” (Mao et al., 2016), a 30-layer hierarchical residual network with 128 feature maps, which has been demonstrated to be very effective in a wide range of image restoration tasks, including Gaussian noise....For all further tests, we switch from RED30 to a shallower U-Net (Ronneberger et al., 2015) that is roughly 10× faster to train and gives similar results (−0.2 dB in Gaussian noise)."

Since U-Net is so much faster only checked it and found PDF "U-Net: Convolutional Networks for Biomedical Image Segmentation" https://pdfs.semanticscholar.org/0704/5 ... 912281.pdf which mentions "The u-net architecture achieves very good performance on very different biomedical segmentation applications...We provide the full Caffe[6]-based implementation and the trained networks4. We are sure that the u-net architecture can be applied easily to many more tasks."

Continued search and also found "U-Net: Convolutional Networks for Biomedical Image Segmentation" https://lmb.informatik.uni-freiburg.de/ ... index.html there is a short video and at bottom of page "We provide the u-net for download in the following archive: u-net-release-2015-10-02.tar.gz (185MB). It contains the ready trained network, the source code, the matlab binaries of the modified caffe network, all essential third party libraries, the matlab-interface for overlap-tile segmentation and a greedy tracking algorithm used for our submission for the ISBI cell tracking challenge 2015. Everything is compiled and tested only on Ubuntu Linux 14.04 and Matlab 2014b (x64)" Hope it will be ported to Windows!!! it also mentions "If you do not have a CUDA-capable GPU or your GPU is smaller than mine, edit segmentAndTrack.sh accordingly (see there for documentation)."

I do not read either German so do not know if this has been addressed in these HandBrake Forums but did search the English HandBrake Forum and did not find any mention of U-Net.

Question: I am not a Linux coder so am interested to find out from others how difficult it would be to add U-Net to Handbrake (Linux) so it could be used before H.265 conversion?

Thanks

Ken

[Woodstock]

Question back at you - have you played with the existing NLmeans filter in handbrake yet, which is also intended to do this?

[KenithO]

Hi Woodstock,

"have you played with the existing NLmeans filter in handbrake yet..." No am new to Handbrake and have only tried H.265 with defaults.

Googled using keywords "NLmeans filter" and got 484 results. Also searched Youtube using same but got nothing.

Do you have any before/after images or tutorials/guides how to use it?

Thanks

Ken

[s55]

https://dirk-farin.net/projects/nlmeans/index.html

NLMeans can be activated on the Filters tab by choosing NLMeans on Denoise. Experiment with the preset until you get the desired result. Tune probably want Tape

Sharpen is right below it but you may not need that.

We don't support stabilisation.