Yeah.. the problem is that it needa to be exactly lile the Photoshop one to get the same quality in real time while gaming.

Hello again!
About a year ago we had a topic about Waifu2x in real time. Now I am wondering if someone is interested in something that is probably way easier. And this will be good for those who likes the look of Waifu2x. These images are made from Retroarch shaders in real time then extra filtered with Photoshops Oil Paint filter. So I wonder if someone has the code for the oil paint filter or could make it? It would then be quite easy to port to Retroarch.

Apperently, a neural network called FSRCNN is suppose to be much faster than waifu2x, easily 30fps, and still almost look as good as it (if trained correct), or better in some cases.

http://mmlab.ie.cuhk.edu.hk/projects/FSRCNN.html

Quote from: p2 on April 03, 2017, 09:56:20 AM
im im not mistsken waifu2x only takes parts of the computing resources, not the maximum. So shouldnt it be possible ot use multiple instances of waifu2x at the same time, looping through them, each grabbing a frame?

That is true. I did look at Afterburner and I THINK it was around 50%. I am not sure thou, it was not using it max at least like some games. I will check that later.
Now if someone did make this now it would be fun.
Waifu2x minimum requirements - GTX 1080Ti
Recomended - N/A

Quote from: gameblabla on April 03, 2017, 12:18:15 AM

Quote from: VMike on April 02, 2017, 11:53:19 PM
What about speed? To get the result I have got in the videos I have made, each frame took almost 0.5 seconds to render (980ti). I don't know how well optimized waifu is of course.

This means that the emulator would only be able to render at 2 frames per second... Not much if you ask me
I doubt Waifu2x can be improved that much. Not with significant differences anyway

Yeah, that is sad to hear.

Quote from: gameblabla on April 02, 2017, 11:47:31 PM

Quote from: VMike on April 02, 2017, 11:42:57 PM
May I ask why software rendering would be better? Since waifu uses cuda. Well, I am not good at this. But I really would like Waifu2x for ps1 emu, yes 

We could use OpenGL rendering but we would have to render it again to a framebuffer so we treat it again with Waifu2x.
With software rendering, we could simply pass the internal framebuffer to waifu2x to another bigger framebuffer.
There's probably an easier way to do it though.

What about speed? To get the result I have got in the videos I have made, each frame took almost 0.5 seconds to render (980ti). I don't know how well optimized waifu is of course.

Quote from: gameblabla on April 02, 2017, 11:33:12 PM
@VMike, i don't understand why you want to apply Waifu2x to PS1 games, especially 3D games.
For 3D games, most people are better off using the OpenGL renderer and use Widescreen hacks.
It won't look as odd as Waifu2x, at least for games like Resident Evil.

But if you insist... it can be done but we would have to use the software renderer.
In that case, it's better to experiment results using pcsx4all or a more primitive console like the Sega Megadrive/Genesis.

The reason is how it fixes the pre-rendered backgrounds in games that uses those. And those games are about all I play on Ps1. Like most rpgs, Resident Evil etc.
I know it wont do much good to the 3d elements.
May I ask why software rendering would be better? Since waifu uses cuda. Well, I am not good at this. But I really would like Waifu2x for ps1 emu, yes 

Quote from: p2 on April 02, 2017, 09:11:52 PM
omg it's just too adorable how the character model looks like its getting simplyfied when further away 

btw that's EXACTLY how every game engine should do it!! No quality loss, no annoying blur effects, no enemies suddenly appearing once they're close enough, just simplyfied models ^.^

The reason it looks like that is because the game is rendered at psOne native res, wich is 320x240 pixels. It gives best result with Waifu2x if the image is unstretched and not have anything added to it with the emulator. That is one problem we could have if one day Waifu is available for real time. The 3d rendered elements of the game such as field view and boss fights (and characters) are rendered extremely low. I don't know how it would be if I did this to high rendered images. All I know is that it would not make the pre-rendered backgrounds look as good as they do now.

Quote from: p2 on April 02, 2017, 09:07:11 PM
so thats how those 2k minecraft texturepacks are made
jk those vids (especially the FF one) look really beautyful! 

Have one more

Quote from: semiprocoder on April 02, 2017, 08:26:16 PM
A video of minecraft with a resource pack upscaled by waifu2x. The resource pack I used is faithful 32x, and I upscaled (only) blocks and items to 256x. I think the result looks pretty nice. Forgive the low fps; minecraft is being weird rn (Ex: when I take a screenshot, the res is 2304x1202, which is higher than my monitor. I have the res set to 1080p in optifine, so idk whats wrong.). Anyways, here is the video:

Internal resolution can be higher than your monitor. Eq I play some games in 4k on my 1080p tv. I have also taken some screenshots at 8k for fun, also that even though my tv is only 1080.

Quote from: p2 on March 29, 2017, 11:50:59 AM
could someone maybe try this on a lil 20sec movie or something?
I'd love to see how this style (u called it painting) looks like if its an animated thing

*currently doing SFG Sharpen Edge in the walrii
Edit01: resembles original image 1m) 99.553474%
Edit01: resembles original image 2m) 99.43323%
Edit: it didnt change much at all... looked like some jpeg artifacts all over the pic, nothing special... :/
now trying some more fancy stuff (parts of AoE2 screenshots)

Here is some video on some gameplay I made

Quote from: DJ Omnimaga on March 31, 2017, 04:16:38 PM
I wonder what would be the minimum requirements for a PC  to render Waifu2x videos at 30 fps...

I think we will have what is needed in ten years.

Quote from: p2 on March 29, 2017, 08:08:54 PM
well Waifu2x is indeed most impressive snd pretty fast but still...
on an Intel pro 460 (macbook pro 15" late 2016 with max available hardware) it takes about 3 seconds for a pic to upscale from 765x335 to 6120x2680 and apply it.
so for regular videos I'd say 2seconds per frame which is 0.5FPS... we need 60x the speed of that to get it running....
And it's gonna be rather hard to find hardware 60x faster (i7 at 3.8GHz and Inter 460 here).

But still I only tested the thing inside the browser from http://waifu2x.me/ so doing it directly with an app might be a bit faster, that I do not know.

Still I really REALLY hope it'll work one day! ^.^

EDIT:
Some linux (ubuntu) pro should read through this, it looks like there IS a method to apply it to videos
https://github.com/nagadomi/waifu2x

Here is a glsl version, if that helps.
https://github.com/ues h i t a/waifu2x-converter-glsl

A Open GL port.
https://github.com/MaverickTse/aviutl-waifu2x-gl

Here is the Windows version that can run on your gpu if you have Nvidia.
https://github.com/lltcggie/waifu2x-caffe

As I said, someone did mention that optimization to achieve 30fps(Ps1 runs in like 24 fps or so) is not impossible. But I am no coder, I dont know this stuff. Otherwise I would convert it into a emu upscaler at once

I am not happy until it runs flawless in real time over the game

Quote from: p2 on March 29, 2017, 11:50:59 AM
could someone maybe try this on a lil 20sec movie or something?
I'd love to see how this style (u called it painting) looks like if its an animated thing

*currently doing SFG Sharpen Edge in the walrii


Edit01: resembles original image 1m) 99.553474%
Edit01: resembles original image 2m) 99.43323%

Someone put the frames from the original Tomb Raider 1 intro thru it.

Here is the original.

Here is the Waifu one.


Am going to do a little video myself maybe. I will url it here then.

The more I see of it the more I need it, haha.


Edit by p2: fixed youtube links