CodeWalrus

General => Tech, Science, IT discussion & News => Topic started by: gameblabla on September 09, 2016, 02:05:54 am

Title: Scaling algorithms
Post by: gameblabla on September 09, 2016, 02:05:54 am
Everyone knows scaling algorithms such as linear, cubic or lanczos or even specific ones like hqx or xBRZ.
But i thought i would talk about some of the lesser known scaling algorithms i think are pretty interesting.

Let's start with Waifu2x first.
Waifu2x is an upscaling algorithm for anime-style art that was developped at a chinese university in Hong Kong, inspired by SRCNN.
(That's probably the only good thing China has done, read on)
It also supports noise cancellation but a picture with little to no noise gives me the best result.
The goal of the algorithm is to give sharper details to the art, compared to traditional algorithms.
Perhaps the best thing to explain it is to show it.

(https://gameblabla.nl/img/comparaison.png)
Left is cubic upscaling, Right is Waifu2x

As you can see, the Waifu2x processed image has much less blur and has sharper edges than the cubic one.
It is overall much prettier to look at than the cubic processed one.

Another test, here's the reference image. (it comes from School Days)
(https://gameblabla.nl/img/reference.png)

What i have done next is to upscale it using the cubic algorithm to 1920x1080.
Then, i tried to upscale it twice (since waifu2x only upscales two times) with waifu2x.

Here's the result (You need Firefox or Webkit to view the APNG picture)
https://gameblabla.nl/img/waifu2x_result.png (https://gameblabla.nl/img/waifu2x_result.png)

This time though, it is not as significant as the previous one, probably due to many details in the background.
But the edges around the boy are still much sharper than the cubic one, once again and the overall picture is less blurry.

I tried with other anime pictures and i got pretty good results, so far i am very happy.
Maybe i should try to upscale an old anime to 2k using waifu2x ?
Sadly, i do not have the processing power or space for that.

Moving on to...

Seam Carving
This algorithm has a completely different purpose.
It was made in 2005 by a team in MERL with the goal of making an upscaler/downscaler that would keep most of details
in the picture when changing aspect ratios.

ImageMagick has an implementation of Seam Carving so i thought trying it out.
Our reference image is still the same :
(https://gameblabla.nl/img/reference.png)

I will reduce to 75% of its horizontal size. (out of 100%)
(https://gameblabla.nl/img/reference_seam.png)

Here's what it would like when downscaled with Lanczos
(https://gameblabla.nl/img/reference_lanczos.png)

You can see how it tries to keep important details like the main character without changing the proportions too much.
It results in a funny and non-consistent look though, especially for humans.
Let's try it with Obama instead :

Original:
(https://gameblabla.nl/img/president_ref.jpg)

Seam carved to 70% of its horizontal size :
(https://gameblabla.nl/img/president_seam.jpg)

Our president looks much thinner, he almost looks like a kid now !

Now let's try to downscale using this algorihm with our School Days picture :
(https://gameblabla.nl/img/seamcarving_downscale.png)

Everything we knew about the picture before is now lost, it looks like something right from a nightmare...
Overall, it is best suited for paysages and aspect ratio changes.

Scalable Function Graphics
If you are looking for the best upscaling algorithm for image recovery, this is the one.
Designed for photos, the results are simply stunning :
The picture is much less blurry than any other algorithms while managing to be fairly accurate to
the original picture.
Have a look :
(https://m1.behance.net/rendition/modules/146595327/disp/43a1c13d7083146eeb2abbf18376d96d.png)

Photofunction, the only software based around it, can upscale such an image using the SFG algorithm.
However this comes at a cost : on a AMD FX chip, upscaling a 64x32 picture can take up to 12 hours !
It is thus not very practical but it does not take advantage of graphics cards, which could help.
It is also crap for aurora beams. (where a simple algorithm such as cubic fairs better)

It makes me wish someone else would reimplement it using CUDA or OpenCL because the results are great.
Sadly, due to overheating, i can't demonstrate it to you.
But you can download Photofunction here (requires Java 8.0) :
https://gameblabla.nl/files/photofunction.zip (https://gameblabla.nl/files/photofunction.zip)
Title: Re: Scaling algorithms
Post by: Juju on September 09, 2016, 05:15:03 am
I heard wonders about Waifu2x, works best with vector art, like anime and cartoon art. (Which is pretty much made for.)

Seam Carving looks pretty interesting too, I think I heard of it while reading the ImageMagick docs.

And Scalable Function Graphics looks pretty interesting too, wish someone makes an implementation in CUDA or something like they did with Waifu2x.
Title: Re: Scaling algorithms
Post by: DJ Omnimaga on September 25, 2016, 04:58:50 pm
Woah, I didn't know about those first two algorithms. The first one, Waifu2x, looks especially nice. Something I always wondered, though, is if there is a tool that can bulk resize an entire sprite sheet or a bunch of images rather than one by one. That could be handy for pixel art resizing because resizing an entire sprite sheet causes each sprite to bleed on the corners of the one next to it.
Title: Re: Scaling algorithms
Post by: catastropher on September 25, 2016, 05:04:18 pm
We had to implement seam carving in my algorithms class (which I took like 3 years ago). It's a fun dynamic programming problem! :D
Title: Re: Scaling algorithms
Post by: DJ Omnimaga on September 25, 2016, 05:06:50 pm
I don't want to see what :walrii: would look like if seam carving was applied to it O.O
Title: Re: Scaling algorithms
Post by: catastropher on September 25, 2016, 05:08:03 pm
Quote from: DJ Omnimaga on September 25, 2016, 05:06:50 pm
I don't want to see what :walrii: would look like if seam carving was applied to it O.O

I still have my source code, maybe I should run Walrii through it? :P
Title: Re: Scaling algorithms
Post by: DJ Omnimaga on September 25, 2016, 05:13:41 pm
Oh noes D:. But yeah this algorithm probably works best with people photos with the right coloring/gamma/contrast, right?
Title: Re: Scaling algorithms
Post by: gameblabla on September 25, 2016, 05:16:03 pm
Quote from: DJ Omnimaga on September 25, 2016, 04:58:50 pm
Woah, I didn't know about those first two algorithms. The first one, Waifu2x, looks especially nice. Something I always wondered, though, is if there is a tool that can bulk resize an entire sprite sheet or a bunch of images rather than one by one. That could be handy for pixel art resizing because resizing an entire sprite sheet causes each sprite to bleed on the corners of the one next to it.

All the waifu2x converters such as waifu2x-converter-cpp are command-line so yes, you can convert multiple sprites by simply doing a loop
like this (bash) :
for f in $(find . -type f); do
./waifu2x-converter-cpp -i $f;
done

And yes, it could be pretty hand for a video game if you want to work on a higher-resolution remake or something.
Or you suck badly at drawing high-resolution graphics.

QuoteI still have my source code, maybe I should run Walrii through it? :P

I would like to see it, especially if it's written in C/C++ !
It could be implemented to an emulator.

QuoteOh noes D:. But yeah this algorithm probably works best with people photos with the right coloring/gamma/contrast, right?

It works only best without any humans in it or else they will be badly deformed.
Title: Re: Scaling algorithms
Post by: DJ Omnimaga on September 25, 2016, 05:54:53 pm
Yeah my main goal at one point was to port Reuben to PC or higher-res, but I didn't want to redo all sprites from scratch. :P (except the main character, which would have used the battle sprite facing all 4 directions)
Title: Re: Scaling algorithms
Post by: VMike on March 28, 2017, 03:15:20 pm
Hello.
First I want to say sorry that I dug up this old thread. I just accidently stumbled on it in my persue for info.
So I registered here to write a reply in this thread.
It is about Waifu2x that I think makes old games look soooo good. Here arw some pictures of Waifu2x 32x with highest noise reduction over Final Fantasy VII. It is first rendered in the standard resolution of 320x240 unstretched wich gives Waifu2x best results.

https://m.imgur.com/a/Zef82
And also here
https://m.imgur.com/a/LOkFG

Now, I know there are other filters for Ps1 emulation but none I have ever seen gives such nice result.
I have asked around everywhere but always get thevsame answer; No, Waifu2x is not possible to run in realtime.
And I have kinda given up. But since I got nothing to lose I tought I might as well try here. If anyone is up for the task, I would even pay a little to have Waifu2x filtering my Ps1 emulators in real time.
Oh well. Thanx for reading anyway :)
Title: Re: Scaling algorithms
Post by: DJ Omnimaga on March 28, 2017, 06:34:53 pm
Hi @VMike and welcome to the forums. To be honest, I am not too sure if I like how the Final Fantasy VII screenshots turned out with Waifu2x, as there seems to be too much filtering (probably due to the original 240p resolution being too low), but it actually gives the game a different feel, like if it was painted by hand. I am definitively curious if computers are powerful enough to render such filter in real time. I wonder if it would be possible to add less filtering, though, so that the game looks closer to PS1 graphics but at higher resolution? What about 480p?
Title: Re: Scaling algorithms
Post by: VMike on March 28, 2017, 08:20:05 pm
Thank you for your answer.
In those pictures I have taken an image from the emulator while it runs in 320x240. Its wierd, because when it is pre-rendered games like Final Fantasy and Resident Evil they actualy looks better if the source is in original resolution. I have contacted a guy on twitter as well who claims that it should be easy to make Waifu2x run in 30fps if the image is small enough. I hope 320x240 is small enough.

Could someone make like an overlay you just could add over an emulator window wich uses Waifu2x? Its wierd, because everywhere else I have asked they have said that it is impossible to make it run fast enough for Ps1 emulation.

Yeah I really, really like that look it gives the game. Would be extremely nice if someone could make it work.

Here are some pictures of Resident Evil 2 320x240 screenshots upscaled with Waifu2x.
(https://codewalr.us/proxy.php?request=http%3A%2F%2Fi.imgur.com%2Fs1kSktY.png&hash=dcd8cccb03dd289f94c52821ed9aac2e)
(https://codewalr.us/proxy.php?request=http%3A%2F%2Fi.imgur.com%2FuLwLwa3.png&hash=0cd3fe2261b2e2193016410c4c374491)
(https://codewalr.us/proxy.php?request=http%3A%2F%2Fi.imgur.com%2F9u6DG55.png&hash=819b69fed7b9ee05d9b08e4bd0db469d)
(https://codewalr.us/proxy.php?request=http%3A%2F%2Fi.imgur.com%2FW6H8xw7.png&hash=0627e8c9bac71e376d11c4105d539e65)
I also added some minor ReShade to them. Gaussian flare and some sharpening. Well, compared to the originals.. this looks like gold. At least to me :)
Title: Re: Scaling algorithms
Post by: c4ooo on March 29, 2017, 01:06:20 am
What does a waifu2x walrus look like? :3
Title: Re: Scaling algorithms
Post by: gameblabla on March 29, 2017, 06:37:02 am
Hey @VMike,
as you may have noticed when using Waifu2x, it is very CPU-intensive !
My AMD FX 6300 can take up to 2 minutes to upscale to a 2k image, let alone for 4k...
For real-time upscaling, it needs to be much faster than that, at least 30 images per second.

The only way for it to be practical way would be to make use of GPU acceleration (via CUDA or OpenCL) but even then,
it would require a powerful GPU (at least a Titan), fast storage (Samsung SSD), fast DDR4 memory etc...
Not impossible but very impractical for most people.

Also, Waifu2x would be best suited for visual games and games with anime-styled graphics.
Resident Evil looks interesting with Waifu2x, almost like a painting.
But for now, it's best suited for still pictures and maybe movies (if you are doing frame-by-frame).
Title: Re: Scaling algorithms
Post by: VMike on March 29, 2017, 08:34:45 am
Quote from: gameblabla on March 29, 2017, 06:37:02 am
Hey @VMike,
as you may have noticed when using Waifu2x, it is very CPU-intensive !
My AMD FX 6300 can take up to 2 minutes to upscale to a 2k image, let alone for 4k...
For real-time upscaling, it needs to be much faster than that, at least 30 images per second.

The only way for it to be practical way would be to make use of GPU acceleration (via CUDA or OpenCL) but even then,
it would require a powerful GPU (at least a Titan), fast storage (Samsung SSD), fast DDR4 memory etc...
Not impossible but very impractical for most people.

Also, Waifu2x would be best suited for visual games and games with anime-styled graphics.
Resident Evil looks interesting with Waifu2x, almost like a painting.
But for now, it's best suited for still pictures and maybe movies (if you are doing frame-by-frame).


There is a version using cuda. I think that is also open sourse. Yes it is slow, with my 980ti it takes about 20 seconds to scale 320x240 32 times the size. It was just that I read that someone claims he has it got running for 30 fps. And I dont think it is impossible.

I really want this for those old pre-rendered psone games. I have not seen any other filter that makes them look so good.
Title: Re: Scaling algorithms
Post by: p2 on March 29, 2017, 11:50:59 am
could someone maybe try this on a lil 20sec movie or something?
I'd love to see how this style (u called it painting) looks like if its an animated thing O.O

*currently doing SFG Sharpen Edge in the walrii
Edit01: resembles original image 1m) 99.553474%
Edit01: resembles original image 2m) 99.43323%
Edit: it didnt change much at all... looked like some jpeg artifacts all over the pic, nothing special... :/
now trying some more fancy stuff (parts of AoE2 screenshots)
Title: Re: Scaling algorithms
Post by: VMike on March 29, 2017, 12:05:31 pm
Quote from: p2 on March 29, 2017, 11:50:59 am
could someone maybe try this on a lil 20sec movie or something?
I'd love to see how this style (u called it painting) looks like if its an animated thing O.O

*currently doing SFG Sharpen Edge in the walrii


Edit01: resembles original image 1m) 99.553474%
Edit01: resembles original image 2m) 99.43323%


Someone put the frames from the original Tomb Raider 1 intro thru it.

Here is the original.
https://www.youtube.com/watch?v=_tOnWSV4Thk
Here is the Waifu one.
https://www.youtube.com/watch?v=UFZe7zAweRM

Am going to do a little video myself maybe. I will url it here then.

The more I see of it the more I need it, haha.


Edit by p2: fixed youtube links
Title: Re: Scaling algorithms
Post by: p2 on March 29, 2017, 01:22:52 pm
My first try of an AgeOfEmpires2 screenshot:
Original Image:
(https://img.ourl.ca/Step00.png)(https://img.ourl.ca/Bildschirmfoto%202017-03-29%20um%2015.16.51.png)
always resizing it from 54x66 to 540x714













(https://img.ourl.ca/Step01.png)(https://img.ourl.ca/Step02.png)(https://img.ourl.ca/Step03.png)(https://img.ourl.ca/Step04.png)(https://img.ourl.ca/Step05.png)(https://img.ourl.ca/Step06.png)(https://img.ourl.ca/Step07.png)(https://img.ourl.ca/Step08.png)(https://img.ourl.ca/Step09.png)(https://img.ourl.ca/Step10.png)(https://img.ourl.ca/Step11.png)(https://img.ourl.ca/Step12.png)(https://img.ourl.ca/Step13.png)
98,218445%98,73424%98,84584%99,29145%99,50681%99,70652%99,81886%99,93266%100,0%From now on taking an image every few minutesnüflast round half way done...
(https://img.ourl.ca/Step14.png)(https://img.ourl.ca/Step15.png)(https://img.ourl.ca/Step16.png)(https://img.ourl.ca/Step17.png)(https://img.ourl.ca/Step18.png)(https://img.ourl.ca/Step19.png)(https://img.ourl.ca/Step20.png)(https://img.ourl.ca/Step21.png)(https://img.ourl.ca/Step22.png)(https://img.ourl.ca/Step23png)(https://img.ourl.ca/Step24.png)(https://img.ourl.ca/Step25.png)(https://img.ourl.ca/Step26.png)
Step2 - 97,653%halfway through98,10193%98,38588%%%%%%%%%...



For the whole process my CPU was below 20% - while running Firefox, mail client, Telegram, iTunes, Appstore, iMessage, ...
Pls fix this <_<
Except if it's only taking the GPU which would explain a lot of things... ::)
but still it takes much longer than I expected on my machine :ninja:
Title: Re: Scaling algorithms
Post by: DJ Omnimaga on March 29, 2017, 04:36:23 pm
Wow, those Resident Evil pictures look great. As for a video, I guess what could be done is split the video into multiple PNG files, bulk-apply Waifu2x to all of them then re-merge them into a video.
Title: Re: Scaling algorithms
Post by: p2 on March 29, 2017, 05:03:53 pm
yep that's exavtly what I'm looking forward to O.O
but doing it manually is an awful amount of work >.<
Title: Re: Scaling algorithms
Post by: VMike on March 29, 2017, 05:20:09 pm
I am not happy until it runs flawless in real time over the game :)
Title: Re: Scaling algorithms
Post by: p2 on March 29, 2017, 08:08:54 pm
well Waifu2x is indeed most impressive snd pretty fast but still...
on an amd radeon pro 460 (macbook pro 15" late 2016 with max available hardware) it takes about 3 seconds for a pic to upscale from 765x335 to 6120x2680 and apply it.
so for regular videos I'd say 2seconds per frame which is 0.5FPS... we need 60x the speed of that to get it running....
And it's gonna be rather hard to find hardware 60x faster (i7 at 3.8GHz and Inter 460 here).

But still I only tested the thing inside the browser from http://waifu2x.me/ so doing it directly with an app might be a bit faster, that I do not know.

Still I really REALLY hope it'll work one day! ^.^


EDIT:
Some linux (ubuntu) pro should read through this, it looks like there IS a method to apply it to videos O.O
https://github.com/nagadomi/waifu2x
Title: Re: Scaling algorithms
Post by: VMike on March 29, 2017, 08:14:23 pm
Quote from: p2 on March 29, 2017, 08:08:54 pm
well Waifu2x is indeed most impressive snd pretty fast but still...
on an Intel pro 460 (macbook pro 15" late 2016 with max available hardware) it takes about 3 seconds for a pic to upscale from 765x335 to 6120x2680 and apply it.
so for regular videos I'd say 2seconds per frame which is 0.5FPS... we need 60x the speed of that to get it running....
And it's gonna be rather hard to find hardware 60x faster (i7 at 3.8GHz and Inter 460 here).

But still I only tested the thing inside the browser from http://waifu2x.me/ so doing it directly with an app might be a bit faster, that I do not know.

Still I really REALLY hope it'll work one day! ^.^

EDIT:
Some linux (ubuntu) pro should read through this, it looks like there IS a method to apply it to videos O.O
https://github.com/nagadomi/waifu2x


Here is a glsl version, if that helps.
https://github.com/ues h i t a/waifu2x-converter-glsl

A Open GL port.
https://github.com/MaverickTse/aviutl-waifu2x-gl

Here is the Windows version that can run on your gpu if you have Nvidia.
https://github.com/lltcggie/waifu2x-caffe

As I said, someone did mention that optimization to achieve 30fps(Ps1 runs in like 24 fps or so) is not impossible. But I am no coder, I dont know this stuff. Otherwise I would convert it into a emu upscaler at once :)
Title: Re: Scaling algorithms
Post by: p2 on March 29, 2017, 08:22:35 pm
Looks like mpost of the material on Waifu2x is Japanese which means we need somone who can read Japanese... :ninja:
Maybe @Juju could help us out with this? <3 (Japanese + Ubuntu) But I doubt he'd have the time for this...  :-\
Title: Re: Scaling algorithms
Post by: Juju on March 30, 2017, 02:26:05 am
I'm not that good, and pretty sure you can find docs in English.
Title: Re: Scaling algorithms
Post by: p2 on March 30, 2017, 10:26:29 pm
Doing SFG in Detail Recovery Mode on a 765x335px picture since 2 days.....  :ninja: Can't wait for the reults  :thumbsup:
Title: Re: Scaling algorithms
Post by: p2 on March 31, 2017, 01:27:48 pm
still calculating...
Round 96, which means 96 Million rounds of machine learning so far.
The formula resembles the picture 99.15898%
Title: Re: Scaling algorithms
Post by: DJ Omnimaga on March 31, 2017, 04:16:38 pm
I wonder what would be the minimum requirements for a PC  to render Waifu2x videos at 30 fps...
Title: Re: Scaling algorithms
Post by: VMike on March 31, 2017, 07:18:04 pm
Quote from: DJ Omnimaga on March 31, 2017, 04:16:38 pm
I wonder what would be the minimum requirements for a PC  to render Waifu2x videos at 30 fps...

I think we will have what is needed in ten years.
Title: Re: Scaling algorithms
Post by: aeTIos on March 31, 2017, 07:47:01 pm
Hey @VMike welcome aboard and thanks for digging up this thread, I hadn't seen it yet haha.
By the way would you mind introducing yourself in the [link incoming] appropriate topic? Looking forward to seeing you around :D (although *I* probaby should be around more <_<)
Title: Re: Scaling algorithms
Post by: gameblabla on March 31, 2017, 08:11:49 pm
Quote from: p2 on March 30, 2017, 10:26:29 pm
Doing SFG in Detail Recovery Mode on a 765x335px picture since 2 days.....  :ninja: Can't wait for the reults  :thumbsup:

You're going to be hugely disappointed... it's not designed for that.
Oh well

QuoteI think we will have what is needed in ten years.

I agree, especially if someone builds an accelerator card for Waifu2x !  ;D

Also, if you want to waifu2x a video, it's quite simple (if not CPU and/or GPU intensive)
1. Convert your video into individual frames with ffmpeg
2. Use that version of waifu2x, its a command line only app that supports CUDA :
https://github.com/tanakamura/waifu2x-converter-cpp (https://github.com/tanakamura/waifu2x-converter-cpp)
3. Write a script to process the png files
4. ???
5. PROFIT
Title: Re: Scaling algorithms
Post by: semiprocoder on April 01, 2017, 01:18:34 am
Waifu isn't that great for Miraculous: Tales of Ladybug and Cat Noir (never thought I would like this show, but it is AWESOME). Not that bad, either. The one thing I would say is that I screenshot netflix as I could not find any good online images, so the image quality on these was pretty bad to start off with.
https://www.dropbox.com/sh/r0x6t2khang7tzl/AAC7xfc8nvpZwaKfkbyX3TAaa?dl=0
Title: Re: Scaling algorithms
Post by: VMike on April 01, 2017, 07:51:22 pm
Quote from: p2 on March 29, 2017, 11:50:59 am
could someone maybe try this on a lil 20sec movie or something?
I'd love to see how this style (u called it painting) looks like if its an animated thing O.O

*currently doing SFG Sharpen Edge in the walrii
Edit01: resembles original image 1m) 99.553474%
Edit01: resembles original image 2m) 99.43323%
Edit: it didnt change much at all... looked like some jpeg artifacts all over the pic, nothing special... :/
now trying some more fancy stuff (parts of AoE2 screenshots)


Here is some video on some gameplay I made :)
https://www.youtube.com/watch?v=Bt8jKC_Nc0U
Title: Re: Scaling algorithms
Post by: semiprocoder on April 02, 2017, 08:26:16 pm
A video of minecraft with a resource pack upscaled by waifu2x. The resource pack I used is faithful 32x, and I upscaled (only) blocks and items to 256x. I think the result looks pretty nice. Forgive the low fps; minecraft is being weird rn (Ex: when I take a screenshot, the res is 2304x1202, which is higher than my monitor. I have the res set to 1080p in optifine, so idk whats wrong.). Anyways, here is the video:
https://www.youtube.com/watch?v=ccZh_5YKVqY
Title: Re: Scaling algorithms
Post by: VMike on April 02, 2017, 08:43:28 pm
Quote from: semiprocoder on April 02, 2017, 08:26:16 pm
A video of minecraft with a resource pack upscaled by waifu2x. The resource pack I used is faithful 32x, and I upscaled (only) blocks and items to 256x. I think the result looks pretty nice. Forgive the low fps; minecraft is being weird rn (Ex: when I take a screenshot, the res is 2304x1202, which is higher than my monitor. I have the res set to 1080p in optifine, so idk whats wrong.). Anyways, here is the video:
https://www.youtube.com/watch?v=ccZh_5YKVqY


Internal resolution can be higher than your monitor. Eq I play some games in 4k on my 1080p tv. I have also taken some screenshots at 8k for fun, also that even though my tv is only 1080.
Title: Re: Scaling algorithms
Post by: p2 on April 02, 2017, 09:07:11 pm
so thats how those 2k minecraft texturepacks are made O.O
jk those vids (especially the FF one) look really beautyful!  :love:
Title: Re: Scaling algorithms
Post by: VMike on April 02, 2017, 09:08:45 pm
Quote from: p2 on April 02, 2017, 09:07:11 pm
so thats how those 2k minecraft texturepacks are made O.O
jk those vids (especially the FF one) look really beautyful!  :love:

Have one more :)
https://www.youtube.com/watch?v=OvKkmQm_O_w
Title: Re: Scaling algorithms
Post by: p2 on April 02, 2017, 09:11:52 pm
omg it's just too adorable how the character model looks like its getting simplyfied when further away  :thumbsup:

btw that's EXACTLY how every game engine should do it!! No quality loss, no annoying blur effects, no enemies suddenly appearing once they're close enough, just simplyfied models ^.^
Title: Re: Scaling algorithms
Post by: VMike on April 02, 2017, 09:15:19 pm
Quote from: p2 on April 02, 2017, 09:11:52 pm
omg it's just too adorable how the character model looks like its getting simplyfied when further away  :thumbsup:

btw that's EXACTLY how every game engine should do it!! No quality loss, no annoying blur effects, no enemies suddenly appearing once they're close enough, just simplyfied models ^.^


The reason it looks like that is because the game is rendered at psOne native res, wich is 320x240 pixels. It gives best result with Waifu2x if the image is unstretched and not have anything added to it with the emulator. That is one problem we could have if one day Waifu is available for real time. The 3d rendered elements of the game such as field view and boss fights (and characters) are rendered extremely low. I don't know how it would be if I did this to high rendered images. All I know is that it would not make the pre-rendered backgrounds look as good as they do now.
Title: Re: Scaling algorithms
Post by: _iPhoenix_ on April 02, 2017, 09:47:39 pm
Or you could spend an hour with some close friends and GIMP

(https://image.ibb.co/m52ibF/149116931965761.png)
Title: Re: Scaling algorithms
Post by: gameblabla on April 02, 2017, 11:33:12 pm
@VMike, i don't understand why you want to apply Waifu2x to PS1 games, especially 3D games.
For 3D games, most people are better off using the OpenGL renderer and use Widescreen hacks.
It won't look as odd as Waifu2x, at least for games like Resident Evil.

But if you insist... it can be done but we would have to use the software renderer.
In that case, it's better to experiment results using pcsx4all or a more primitive console like the Sega Megadrive/Genesis.
Title: Re: Scaling algorithms
Post by: VMike on April 02, 2017, 11:42:57 pm
Quote from: gameblabla on April 02, 2017, 11:33:12 pm
@VMike, i don't understand why you want to apply Waifu2x to PS1 games, especially 3D games.
For 3D games, most people are better off using the OpenGL renderer and use Widescreen hacks.
It won't look as odd as Waifu2x, at least for games like Resident Evil.

But if you insist... it can be done but we would have to use the software renderer.
In that case, it's better to experiment results using pcsx4all or a more primitive console like the Sega Megadrive/Genesis.

The reason is how it fixes the pre-rendered backgrounds in games that uses those. And those games are about all I play on Ps1. Like most rpgs, Resident Evil etc.
I know it wont do much good to the 3d elements.
May I ask why software rendering would be better? Since waifu uses cuda. Well, I am not good at this. But I really would like Waifu2x for ps1 emu, yes  :P
Title: Re: Scaling algorithms
Post by: gameblabla on April 02, 2017, 11:47:31 pm
Quote from: VMike on April 02, 2017, 11:42:57 pm
May I ask why software rendering would be better? Since waifu uses cuda. Well, I am not good at this. But I really would like Waifu2x for ps1 emu, yes  :P

We could use OpenGL rendering but we would have to render it again to a framebuffer so we treat it again with Waifu2x.
With software rendering, we could simply pass the internal framebuffer to waifu2x to another bigger framebuffer.
There's probably an easier way to do it though.
Title: Re: Scaling algorithms
Post by: VMike on April 02, 2017, 11:53:19 pm
Quote from: gameblabla on April 02, 2017, 11:47:31 pm
Quote from: VMike on April 02, 2017, 11:42:57 pm
May I ask why software rendering would be better? Since waifu uses cuda. Well, I am not good at this. But I really would like Waifu2x for ps1 emu, yes  :P

We could use OpenGL rendering but we would have to render it again to a framebuffer so we treat it again with Waifu2x.
With software rendering, we could simply pass the internal framebuffer to waifu2x to another bigger framebuffer.
There's probably an easier way to do it though.

What about speed? To get the result I have got in the videos I have made, each frame took almost 0.5 seconds to render (980ti). I don't know how well optimized waifu is of course.
Title: Re: Scaling algorithms
Post by: gameblabla on April 03, 2017, 12:18:15 am
Quote from: VMike on April 02, 2017, 11:53:19 pm
What about speed? To get the result I have got in the videos I have made, each frame took almost 0.5 seconds to render (980ti). I don't know how well optimized waifu is of course.

This means that the emulator would only be able to render at 2 frames per second... Not much if you ask me
I doubt Waifu2x can be improved that much. Not with significant differences anyway
Title: Re: Scaling algorithms
Post by: VMike on April 03, 2017, 12:47:31 am
Quote from: gameblabla on April 03, 2017, 12:18:15 am
Quote from: VMike on April 02, 2017, 11:53:19 pm
What about speed? To get the result I have got in the videos I have made, each frame took almost 0.5 seconds to render (980ti). I don't know how well optimized waifu is of course.

This means that the emulator would only be able to render at 2 frames per second... Not much if you ask me
I doubt Waifu2x can be improved that much. Not with significant differences anyway

Yeah, that is sad to hear.
Title: Re: Scaling algorithms
Post by: p2 on April 03, 2017, 09:56:20 am
im im not mistsken waifu2x only takes parts of the computing resources, not the maximum. So shouldnt it be possible ot use multiple instances of waifu2x at the same time, looping through them, each grabbing a frame?
Title: Re: Scaling algorithms
Post by: VMike on April 03, 2017, 10:57:26 am
Quote from: p2 on April 03, 2017, 09:56:20 am
im im not mistsken waifu2x only takes parts of the computing resources, not the maximum. So shouldnt it be possible ot use multiple instances of waifu2x at the same time, looping through them, each grabbing a frame?

That is true. I did look at Afterburner and I THINK it was around 50%. I am not sure thou, it was not using it max at least like some games. I will check that later.
Now if someone did make this now it would be fun.
Waifu2x minimum requirements - GTX 1080Ti
Recomended - N/A
Title: Re: Scaling algorithms
Post by: VMike on April 13, 2017, 05:46:55 pm
Apperently, a neural network called FSRCNN is suppose to be much faster than waifu2x, easily 30fps, and still almost look as good as it (if trained correct), or better in some cases.

http://mmlab.ie.cuhk.edu.hk/projects/FSRCNN.html