Ai upscaling - 1080p to 8k to 1080p
-
- Posts: 48
- Joined: Fri May 29, 2020 5:17 pm
Ai upscaling - 1080p to 8k to 1080p
Anyone experimented with ai upscaling? Using topaz video enhancer or some other similar technology to convert a 1080p bluray up to 8k, to be reencoded back to 1080p? This kind of upscaling seems quite promising from i have seen. Mabe even keeping a higher res like 4k?
-
- Veteran User
- Posts: 4859
- Joined: Wed May 04, 2011 11:06 pm
Re: Ai upscaling - 1080p to 8k to 1080p
Why would you want to upscale 1080 to 8k, then convert to 1080?
-
- Posts: 48
- Joined: Fri May 29, 2020 5:17 pm
Re: Ai upscaling - 1080p to 8k to 1080p
Upscale to 8k to improve quality, downsample to 1080p to save space. Downsampled content looks better than native content. So like 4k video, downsampled to 1080p will look better than if it was shot natively at 1080p. Obviously its not going to retain the same sharpness as if it stayed at 4/8k but it should, at least in theory look better than before.rollin_eng wrote: ↑Sun Aug 23, 2020 3:51 pm Why would you want to upscale 1080 to 8k, then convert to 1080?
-
- Veteran User
- Posts: 4859
- Joined: Wed May 04, 2011 11:06 pm
Re: Ai upscaling - 1080p to 8k to 1080p
Upscaling wont improve quality.
-
- Posts: 48
- Joined: Fri May 29, 2020 5:17 pm
Re: Ai upscaling - 1080p to 8k to 1080p
From what ive seen topaz video enhancer ai works pretty well, it takes a while, sometimes a few seconds for each frame, but they do look very good. Its not the kind of upscaling your tv does, where its just mapping a lower res image to more pixels. I dunno what it does but it really does work well.
-
- Veteran User
- Posts: 4859
- Joined: Wed May 04, 2011 11:06 pm
Re: Ai upscaling - 1080p to 8k to 1080p
If thats the case just buy the DVD and upscale to 32k
Re: Ai upscaling - 1080p to 8k to 1080p
Wishful thinking, otherwise called "alchemy."Upscale to 8k to improve quality,
Re: Ai upscaling - 1080p to 8k to 1080p
I understand the idea of "enhanced" upscaling for playback on a high-resolution display, but upscaling then downscaling sounds a lot like just applying a sharpening filter. I wonder whether the results would be any better than using a dedicated sharpening filter; I suspect not consistently so, but maybe sometimes, who knows? The optimal amount of sharpening will also depend on the size of your display and viewing distance, so not something I'd recommend applying generically to all your encodes.
Re: Ai upscaling - 1080p to 8k to 1080p
Despite what you see in crime dramas, you really cannot get better quality by upscaling. You can make educated guesses as to what a single pixel converted to 16 might look like, but it is just a guess. It is influenced not by what was really recorded, but what the algorithms used can create that "mostly doesn't disagree" with the original.
An interesting experiment would be to take an actual 8K source, shrink it to 1080p, then compare an AI-upscaled-to-8k picture looks like. But, you will probably find that whatever quality the original 8K image had, it's suffered badly from the journey.
An interesting experiment would be to take an actual 8K source, shrink it to 1080p, then compare an AI-upscaled-to-8k picture looks like. But, you will probably find that whatever quality the original 8K image had, it's suffered badly from the journey.
-
- Posts: 48
- Joined: Fri May 29, 2020 5:17 pm
Re: Ai upscaling - 1080p to 8k to 1080p
No doubt an upscaled image is worse than an image at that res natively, but the program i used works very well. Its not a simple upscaler to map pixels to larger display, but actually "adds detail". Obviously it cannot "enhance" the image and extract true detail that isnt present in the image in the first place, but its pretty good at taking, for example, a few darkened pixels that are supposed to be a pore, and actually making look correct.Woodstock wrote: ↑Sun Aug 23, 2020 6:54 pm Despite what you see in crime dramas, you really cannot get better quality by upscaling. You can make educated guesses as to what a single pixel converted to 16 might look like, but it is just a guess. It is influenced not by what was really recorded, but what the algorithms used can create that "mostly doesn't disagree" with the original.
An interesting experiment would be to take an actual 8K source, shrink it to 1080p, then compare an AI-upscaled-to-8k picture looks like. But, you will probably find that whatever quality the original 8K image had, it's suffered badly from the journey.
-
- Posts: 48
- Joined: Fri May 29, 2020 5:17 pm
Re: Ai upscaling - 1080p to 8k to 1080p
That was kind of what i was getting at. Not sure tbh. I did try and the image does look noticably sharper, and i cant see any artifacts that are associates with sharpening. So maybe its a good way to go about it.Rodeo wrote: ↑Sun Aug 23, 2020 6:41 pm I understand the idea of "enhanced" upscaling for playback on a high-resolution display, but upscaling then downscaling sounds a lot like just applying a sharpening filter. I wonder whether the results would be any better than using a dedicated sharpening filter; I suspect not consistently so, but maybe sometimes, who knows? The optimal amount of sharpening will also depend on the size of your display and viewing distance, so not something I'd recommend applying generically to all your encodes.
-
- Veteran User
- Posts: 4859
- Joined: Wed May 04, 2011 11:06 pm
Re: Ai upscaling - 1080p to 8k to 1080p
Re: Ai upscaling - 1080p to 8k to 1080p
I've experimented a bit with "TopazLabs Video Enhance AI". It's pretty promising but I think it works best on animated/CGI material at the moment. I upscaled one of my animated DVD's to 1080p for a test and it looks really good. It has pretty good sharpening/noise-removing-stuff implemented that works very well for animations.
-
- Posts: 17
- Joined: Fri Aug 14, 2020 4:13 am
Re: Ai upscaling - 1080p to 8k to 1080p
I've heard the AI upscaling do enhance the graphic quality, it using a certain algorithm to blur the pixel and add noise to the graphic, perhaps do dithering too. No doubt that the AI upscaled video does no longer represent its original object which been filmed, it is somehow distorted, but make the graphic looks pretty & enhanced for sure.
Re: Ai upscaling - 1080p to 8k to 1080p
Much like the x264 psychovisual optimizations, you can do things that look better subjectively, but objectively have reduced the quality of the video.
Re: Ai upscaling - 1080p to 8k to 1080p
Test Setup: I took a 2712x2400 original, reduced it to 50% (no sharpening), and then upscaled to original size using Bicubic, Lanczos 4, and Topaz AI.Re: Ai upscaling - 1080p to 8k to 1080p
Now that you're as impressed as I was by the sharpening, Here are the enhanced Difference Composites. You can see that Topaz is definitely an enhancement, and not an accurate representation.
Re: Ai upscaling - 1080p to 8k to 1080p
Hi everyone, dunno if can help in any way, but I collected some open source resources about "AI upscaling":
- Video2X: Machine learning video/GIF/image upscaling
- Waifu2x-Extension-GUI: Image, GIF and Video enlarger/upscaler(super-resolution[/url] achieved with Waifu2x, SRMD, RealSR, Anime4K and ACNet.
- Video Super Resolution: A collection of state-of-the-art video or single-image super-resolution architectures, reimplemented in tensorflow.
- AviSynth AiUpscale: An AviSynth+ implementation of some Super-Resolution Convolutional Neural Networks.
- Fast-SRGAN: enable real time super resolution for upsampling low resolution videos.
- Anime4K: a set of open-source, high-quality real-time anime upscaling/denoising algorithms that can be implemented in any programming language.
- Zooming-Slow-Mo: Fast and Accurate One-Stage Space-Time Video Super-Resolution
- Video-Super-Resolution list: collection of papers
Of course I also opened a "feature request" @ Intel Media SDK's github...
Hope that inspires !
- Video2X: Machine learning video/GIF/image upscaling
- Waifu2x-Extension-GUI: Image, GIF and Video enlarger/upscaler(super-resolution[/url] achieved with Waifu2x, SRMD, RealSR, Anime4K and ACNet.
- Video Super Resolution: A collection of state-of-the-art video or single-image super-resolution architectures, reimplemented in tensorflow.
- AviSynth AiUpscale: An AviSynth+ implementation of some Super-Resolution Convolutional Neural Networks.
- Fast-SRGAN: enable real time super resolution for upsampling low resolution videos.
- Anime4K: a set of open-source, high-quality real-time anime upscaling/denoising algorithms that can be implemented in any programming language.
- Zooming-Slow-Mo: Fast and Accurate One-Stage Space-Time Video Super-Resolution
- Video-Super-Resolution list: collection of papers
Of course I also opened a "feature request" @ Intel Media SDK's github...
Hope that inspires !
-
- Posts: 2
- Joined: Wed Jan 20, 2021 8:23 am
Re: Ai upscaling - 1080p to 8k to 1080p
Have you tried just using one of the AI models at 100% (Denoise/DeBlock).
I did a small test using the Artemis HQv9 model and it did a great job of maintaining the fine details while stripping the noise from my source material.
My only issue was it took hours to process a short scene on my system so it's not worth my time to do a full movie.
I did a small test using the Artemis HQv9 model and it did a great job of maintaining the fine details while stripping the noise from my source material.
My only issue was it took hours to process a short scene on my system so it's not worth my time to do a full movie.
Re: Ai upscaling - 1080p to 8k to 1080p
Here's the result of my 1st Video 2X upscale test on SHELLSHOCK AMV:
Author introduction (from this page):
By far this is my most austere video, a work of intense significance - tumultuously political and passionately opinionated. Like drugs and alcohol, war can infect even the most righteous of souls, and as seen through the eyes of Shiro Amada, chaos sides with neither friend nor foe.
Every decision in making this video, from song to scene selection, was totally deliberate. My intention was to emulate old war footage (WWII, Vietnam, etc.), which the raw quality of my MJPEG codec aptly provides. For research, I began watching The History Channel as well as every piece of film from Apocalypse Now to Saving Private Ryan. That direction proved more of a challenge than I realized. Countless revisions were done to many parts before settling on a sequence that worked.
There is a seemingly random dimension to the video, amplifying the nightmare of war. What I enforce upon the viewer is combat, terror, fear, blood, contempt, and death – juxtaposed with a message, a warning, and a lesson to learn.
Source file (download):
Upscaled result (download):
"Side-to-side" (original vs upscaled):
Approx. - Google Collab - processing time (waifu2x ncnn Vulkan algorithm) 1 hour and 30 minutes
Author introduction (from this page):
By far this is my most austere video, a work of intense significance - tumultuously political and passionately opinionated. Like drugs and alcohol, war can infect even the most righteous of souls, and as seen through the eyes of Shiro Amada, chaos sides with neither friend nor foe.
Every decision in making this video, from song to scene selection, was totally deliberate. My intention was to emulate old war footage (WWII, Vietnam, etc.), which the raw quality of my MJPEG codec aptly provides. For research, I began watching The History Channel as well as every piece of film from Apocalypse Now to Saving Private Ryan. That direction proved more of a challenge than I realized. Countless revisions were done to many parts before settling on a sequence that worked.
There is a seemingly random dimension to the video, amplifying the nightmare of war. What I enforce upon the viewer is combat, terror, fear, blood, contempt, and death – juxtaposed with a message, a warning, and a lesson to learn.
Source file (download):
Code: Select all
Generale
Nome completo : Shellshock.mpg
Formato : MPEG-PS
Dimensione : 67,1MiB
Durata : 5 min 29s
Modo bitrate generale : Variabile
Bitrate totale : 1.708 kb/s
Compressore : encoded by TMPGEnc (ver. 2.02.31.119)
Video
ID : 224 (0xE0)
Formato : MPEG Video
Versione formato : Version 1
Impostazioni formato : CustomMatrix / BVOP
Impostazioni formato, BVOP : Si
Impostazioni formato, Matrix : Personalizzato
Impostazioni formato, GOP : M=3, N=18
Durata : 5 min 29s
Modalità bitrate : Variabile
Bitrate : 1.482 kb/s
Larghezza : 352 pixel
Altezza : 240 pixel
Rapporto aspetto visualizzazione : 4:3
Frame rate : 29,970 (30000/1001) FPS
Standard : NTSC
Spazio colore : YUV
Croma subsampling : 4:2:0
Profondità bit : 8 bit
Tipo scansione : Progressivo
Modo compressione : Con perdita
Bit/(pixel*frame) : 0.585
Timecode del primo frame : 00:00:00:00
Timecode sorgente : Group of pictures header
GOP, Open/Closed : Open
GOP, Open/Closed of first frame : Closed
Dimensione della traccia : 58,2MiB (87%)
Compressore : TMPGEnc 2.02.31.119
Audio
ID : 192 (0xC0)
Formato : MPEG Audio
Versione formato : Version 1
Profilo formato : Layer 2
Durata : 5 min 29s
Modalità bitrate : Costante
Bitrate : 192 kb/s
Canali : 2 canali
Frequenza campionamento : 44,1 kHz
Frame rate : 38,281 FPS (1152 SPF)
Modo compressione : Con perdita
Dimensione della traccia : 7,54MiB (11%)
Code: Select all
Generale
ID univoco : 113452538163437116898826089180663258847 (0x555A29FF489FA9ECBE61E603C0821EDF)
Nome completo : Shellshock_2X.mkv
Formato : Matroska
Versione formato : Version 4
Dimensione : 168MiB
Durata : 5 min 32s
Bitrate totale : 4.238 kb/s
Data codifica : UTC 2021-01-16 15:55:24
Creato con : mkvmerge v52.0.0 ('Secret For The Mad') 64-bit
Compressore : libebml v1.4.1 + libmatroska v1.6.2
Video
ID : 1
Formato : AVC
Formato/Informazioni : Advanced Video Codec
Profilo formato : High@L4
Impostazioni formato : CABAC / 6 Ref Frames
Impostazioni formato, CABAC : Si
Impostazioni formato, RefFrames : 6 frame
ID codec : V_MPEG4/ISO/AVC
Durata : 5 min 29s
Bitrate : 4.147 kb/s
Larghezza : 1.408 pixel
Altezza : 960 pixel
Rapporto aspetto visualizzazione : 3:2
Modalità frame rate : Costante
Frame rate : 29,970 (30000/1001) FPS
Spazio colore : YUV
Croma subsampling : 4:2:0
Profondità bit : 8 bit
Tipo scansione : Progressivo
Bit/(pixel*frame) : 0.102
Dimensione della traccia : 163MiB (97%)
Compressore : x264 core 152 r2854 e9a5903
Impostazioni compressione : cabac=1 / ref=6 / deblock=1:1:1 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=0.40:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=3 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=5 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=17.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:0.60
Default : Si
Forced : No
Audio
ID : 2
Formato : AAC LC
Formato/Informazioni : Advanced Audio Codec Low Complexity
ID codec : A_AAC-2
Durata : 5 min 24s
Bitrate : 127 kb/s
Canali : 2 canali
Channel layout : L R
Frequenza campionamento : 44,1 kHz
Frame rate : 43,066 FPS (1024 SPF)
Modo compressione : Con perdita
Ritardo video : 7s 954 ms
Dimensione della traccia : 4,92MiB (3%)
Default : Si
Forced : No
Approx. - Google Collab - processing time (waifu2x ncnn Vulkan algorithm) 1 hour and 30 minutes
Re: Ai upscaling - 1080p to 8k to 1080p
Sorry, image links:
https://i.servimg.com/u/f31/16/32/04/85/gundam10.png
https://i.servimg.com/u/f31/16/32/04/85/gundam11.png
https://i.servimg.com/u/f31/16/32/04/85/shot_310.png
https://i.servimg.com/u/f31/16/32/04/85/shot_110.png
https://i.servimg.com/u/f31/16/32/04/85/mirino10.png
https://i.servimg.com/u/f31/16/32/04/85/mirino10.jpg
https://i.servimg.com/u/f31/16/32/04/85/abbrac12.png
https://i.servimg.com/u/f31/16/32/04/85/abbrac13.png
https://i.servimg.com/u/f31/16/32/04/85/bn_36010.png
https://i.servimg.com/u/f31/16/32/04/85/bn_10810.png
https://i.servimg.com/u/f31/16/32/04/85/cockpi10.png
https://i.servimg.com/u/f31/16/32/04/85/cockpi11.png
https://i.servimg.com/u/f31/16/32/04/85/tempes10.png
https://i.servimg.com/u/f31/16/32/04/85/tempes11.png
https://i.servimg.com/u/f31/16/32/04/85/explos10.png
https://i.servimg.com/u/f31/16/32/04/85/explos10.jpg
https://i.servimg.com/u/f31/16/32/04/85/fratel10.png
https://i.servimg.com/u/f31/16/32/04/85/fratel11.png
https://i.servimg.com/u/f31/16/32/04/85/acchia11.png
https://i.servimg.com/u/f31/16/32/04/85/acchia10.jpg
https://i.servimg.com/u/f31/16/32/04/85/final_11.png
https://i.servimg.com/u/f31/16/32/04/85/final_11.jpg
https://i.servimg.com/u/f31/16/32/04/85/gundam10.png
https://i.servimg.com/u/f31/16/32/04/85/gundam11.png
https://i.servimg.com/u/f31/16/32/04/85/shot_310.png
https://i.servimg.com/u/f31/16/32/04/85/shot_110.png
https://i.servimg.com/u/f31/16/32/04/85/mirino10.png
https://i.servimg.com/u/f31/16/32/04/85/mirino10.jpg
https://i.servimg.com/u/f31/16/32/04/85/abbrac12.png
https://i.servimg.com/u/f31/16/32/04/85/abbrac13.png
https://i.servimg.com/u/f31/16/32/04/85/bn_36010.png
https://i.servimg.com/u/f31/16/32/04/85/bn_10810.png
https://i.servimg.com/u/f31/16/32/04/85/cockpi10.png
https://i.servimg.com/u/f31/16/32/04/85/cockpi11.png
https://i.servimg.com/u/f31/16/32/04/85/tempes10.png
https://i.servimg.com/u/f31/16/32/04/85/tempes11.png
https://i.servimg.com/u/f31/16/32/04/85/explos10.png
https://i.servimg.com/u/f31/16/32/04/85/explos10.jpg
https://i.servimg.com/u/f31/16/32/04/85/fratel10.png
https://i.servimg.com/u/f31/16/32/04/85/fratel11.png
https://i.servimg.com/u/f31/16/32/04/85/acchia11.png
https://i.servimg.com/u/f31/16/32/04/85/acchia10.jpg
https://i.servimg.com/u/f31/16/32/04/85/final_11.png
https://i.servimg.com/u/f31/16/32/04/85/final_11.jpg