Further image scaling algorithms

[Djfe]

What do you think about implementing further image/video scaling algorithms?

There are so called "super resolution" scalers out there with and without subpixel motion estimation which could enhance video Quality a lot and guess informations from nearby pixels so that scaled videos would look a lot better than Lanczos:
Links to get an idea what I'm talking about:
http://en.wikipedia.org/wiki/Superresolution
http://www.wisdom.weizmann.ac.il/~visio ... age_SR.pdf
https://www.youtube.com/watch?v=181c6DxDs6k
https://www.youtube.com/watch?v=PnqY-YIglQM
http://users.soe.ucsc.edu/~milanfar/IS0 ... aPaper.pdf (really interesting paper!)
http://citeseerx.ist.psu.edu/viewdoc/do ... 1&type=pdf
http://jeeshenlee.files.wordpress.com/2 ... emgucv.pdf

And there is another type of scaling which would be nice:
so called hqx and brz image scalers, they are ideal for scaling pixelated (retro) styled stuff and fonts (maybe animation/cartoons as well)
http://en.wikipedia.org/wiki/2xSaI
http://www.imagemagick.org/discourse-se ... 22&t=23046
http://sourceforge.net/projects/hqmame/
http://scale2x.sourceforge.net/index.html

Quote from the second link:

The implementation of these pixel art scaling algorithms can be found in following websites:
1. Super Eagle and Super2xSaI : Main Site;
2. Hqnx : Main site, Source Code or Source Code with support transparency ;
3. xBR : Main Site, and it's source code;
4. Almost all pixel art scaling algorithm : Main site, Source code collections. The author of this site (Hawkynt) is very gracious as he always reply my private message regarding his software.

My suggestion is, please visit the main site first then see their sample library at the number four (http://code.google.com/p/2dimagefilter/), because it was the only project that I know which tried to unifying and even improving these algorithm outside emulator project (unfortunately it was GPL and implemented in C Sharp). This site also provide stand alone application, so we can see how the algorithm works.

EDIT: updatet the quote, to show the links better

[Djfe]

Another interesting link about this topic:
http://www.extremetech.com/extreme/1329 ... t-yeeaaaah

And a German paper about it:
http://www.informatik.hu-berlin.de/fors ... o10_15.pdf

[Deleted User 13735]

Looks interesting; post back when you have some working open source libraries.

[Deleted User 11865]

We're going to allow a choice of scaling algorithms eventually. At first, it will be limited to what libswscale supports, though it's always possible that they add support for new ones to swscale (or whatever replaces it someday, libav<something>scale).

[Deleted User 13735]

Of course, this stuff has its most appeal for upscaling.
For downscaling for most video purposes, bicubic is just fine, however Lanczos show a little less moire, when present.

[BradleyS]

Having some experience with super resolution algorithms, I can say that the results are typically not dramatic if you're going for a natural looking upscale. Animation excepted, of course.

That said it's cool tech, and patches as always, are welcome. I've not yet found an open source implementation in C.

[Djfe]

Will look out for open-source super resolution filters.

But the open-source-code for all scalers for animations are available at the sites presented in the quote above, it's all GPL or LGPL, I think
(Just click on the links and your browsers takes you to the Google code pages containing the source code/linking to the svn)

EDIT: It might not be all C though but also C#.

[Djfe]

Having some experience with super resolution algorithms, I can say that the results are typically not dramatic if you're going for a natural looking upscale. Animation excepted, of course.

That said it's cool tech, and patches as always, are welcome. I've not yet found an open source implementation in C.

As far as I know super resolution filters, which combine several images by using subpixel motion estimation, manage to reduce noise as well, which sounds awesome to me.
http://en.wikipedia.org/wiki/File:Super ... loseup.png

Open Source super resolution image scalers:
OpenCV (I think you can use it with C):
http://docs.opencv.org/modules/superres ... ution.html

An OpenCV plugin(?) (C++):
http://opencv.jp/opencv2-x-samples/usag ... resolution

Supreme (Not C but Python):
https://github.com/stefanv/supreme

GraspJ (Java7+ and OpenCL 1.1+):
http://www.optnano.com/content/1/1/11
http://link.springer.com/article/10.118 ... -2853-1-11
Source: https://code.google.com/p/graspj/

Multithreaded Plugin for ImageJ:
Parallel Super-Resolution (Java)
https://sites.google.com/site/piotrwend ... resolution
according to this website it's only C++ : http://www.ohloh.net/p/superresolution <--somebody else should take a look at this, I'm not that familiar with looking that up (the screenshots on the google code page looked like a Java GUI though)

Sadly haven't found more scalers using Google, which are Open Source (especially not in C). -> is it too complex to translate an image/video scaler from (for example) Java to C?

Another interesting paper about SR-scalers:
http://www.sipl.technion.ac.il/new/Teac ... erview.pdf


What about supporting more of those FFMPEG inbuilt scalers?:
https://www.ffmpeg.org/ffmpeg-scaler.html
Or is Lanczos better than any of them? (In terms of quality) ->so that it doesn't make sense to include them

[Deleted User 13735]

At first, it will be limited to what libswscale supports,

That was your answer. Of the not-so-many choices libswscale offers, Lanczos is considered the go-to choice most of the time.
Also mentioned, Handbrake GUI doesn't really offer upscaling, so something as basic as bicubic works just fine for downscaling.
There may be some advantages to opening up some more libswscale options in the Handbrake CLI, as newer algorithms find their way into the mainstream.

[Djfe]

Yep, thx
Just wanted to list further scalers and info about scaling algorithms,
If somebody who is interested in implementing it into Handbrake/FMPEG, he might find it useful

A book about a super resolution algorithm with subpixel motion estimation (starts at page 229, but you need to buy the book for the complete algorithm, I think):
http://books.google.de/books?id=cdjzYLNgs_kC&pg=PA229


I start studying computer science after summer this year, so I might come back to this thread again and work on a C implementation for FFMPEG/Handbrake
if I'm more familiar with C and algorithms. (Will see about that, but at least I can remember the notes here)

[Deleted User 13735]

I thought you knew, Handbrake does not use ffmpeg.
It does, however, use the Libav implementation of libswcale as pointed out above (probably quite similar).